You are here:
Understand Named Credentials, Connections, and Pipelines
When you create a data pipeline in Marketing Intelligence, three components work together to ingest data: Named Credential, Connection, and a Pipeline. This architecture separates authentication, connectivity, and data processing so you can reuse credentials and configure pipelines efficiently.
Named Credentials
A Named Credential specifies the authentication details and the endpoint for an external system. It acts as a centralized configuration that services reference so you don't have to define authentication details every time you create a connection.
A Named Credential typically includes:
- The external system endpoint URL
- The authentication method, such as OAuth or basic authentication
- Required credentials, such as tokens, usernames, or passwords
Connections
A Connection links Data Cloud to an external data source. It defines how Data Cloud interacts with the source system and prepares the data for ingestion.
A Connection typically includes:
- Connector configuration for the source system
- Authentication using a Named Credential
Pipelines
A Pipeline manages the end-to-end data ingestion and processing workflow. It retrieves data from the Connection and prepares it for use in Data Cloud.
A Pipeline handles:
- Data ingestion from the source system
- Data transformation
- Data harmonization
- Loading processed data into Data Cloud objects
Component Relationships
| Relationship | Description |
|---|---|
| Named Credential → Connection | One Named Credential can be used by multiple Connections. |
| Named Credential → Pipeline | One Named Credential can support multiple Pipelines. |
| Connection → Pipeline | Each Connection is associated with one Pipeline. |
How the Components Work Together
When a pipeline runs:
- The Named Credential provides the authentication and endpoint to access the external system.
- The Connection uses that credential to establish the link between Data Cloud and the external data source.
- The Pipeline retrieves data through the connection and performs ingestion, transformation, and loading into Data Cloud.
Example: Connecting Multiple Google Ads Accounts
Suppose you want to ingest data from three different Google Ads accounts for your organization. Instead of configuring authentication three times, you can:
- Create one Named Credential called "Google Ads Auth" that contains your OAuth token and the Google Ads API endpoint.
- Create three separate Connections—one for each account—that all reference "Google Ads Auth" for authentication.
- Create three Pipelines—one per Connection—to ingest, transform, and load account data into Data Cloud.
If your Google Ads authentication token expires, you only need to update it in one place: the Named Credential. All three Connections and Pipelines automatically use the updated credential without requiring individual reconfiguration.
