> ## Documentation Index
> Fetch the complete documentation index at: https://docs.domino.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Manage Data Sources
> A Data Source is a reusable connection to an external database, data service, or object store that an admin configures once and practitioners query from any execution.
A Data Source is a reusable connection to an external database, data service, or object store. You configure the connection once, and practitioners query it from a Workspace, Job, or app without handling hostnames, ports, or drivers, and write results back to a Domino Dataset.
Admins can expose a Data Source to everyone in the deployment. When a practitioner [creates a Data Source](/6.3/platform-capabilities/core-concepts/data/data-source-connectors/use-data-sources), they must name the users or organizations that can access it.
## Create a Data Source
Go to **Manage Resources** > **Data Sources**, then click **Connect to External Data**.
Select your Data Source, then enter the configuration details for accessing it. The details vary depending on the Data Source type.
| Credential type | Who supplies the credentials | Use it when |
| :------------------ | :------------------------------------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------------------------------- |
| **Individual** | Each user provides their own credentials before using the Data Source. | Access must be attributable to a person, or the underlying system enforces per-user permissions. |
| **Service Account** | You provide one set of static credentials that Domino applies on behalf of every user with permission to the Data Source. | The team shares one identity against the external system. |
End users cannot access or extract Service Account credentials, nor can they manage user-level permissions on such Data Sources. Domino stores Data Source credentials in the Domino secret store, which is backed by [HashiCorp Vault](https://www.vaultproject.io/).
OAuth authentication is not supported for Service Accounts. The Data Source itself does not distinguish end users, because the same credentials authenticate every query.
Enter your credentials for Data Source authentication, then click **Test Credentials** to verify that authentication works, or click **Skip for Now** to continue.
Select whether **Everyone** can use this Data Source or just **Specific users or organizations**, then click **Finish Setup**.
## Authentication methods
Data Sources support multiple authentication methods to meet different security requirements. Which methods are available depends on the Data Source type, and you choose which of them your organization can use. Enable OAuth or IAM for more secure connections than credentials, enable several methods at once when different teams need different ones, and require certificate authentication where enterprise security standards demand it.
Enabling a method takes one configuration record, and requires the **SysAdmin** role.
In the **Admin panel**, go to **Platform settings** > **Configuration records**, then click **Add Record**.
Add a key for your Data Source type. The [Data Source authentication](/6.3/admin/configuration/central-configuration#data-source-authentication) section of the configuration records reference lists the key for each type.
Enter the value as comma-separated authentication types, for example `certauth,basic,oauth` for PostgreSQL with certificate support, or `basic` for IBM DB2 with basic authentication only.
The first value becomes the default authentication method for all users. Put your preferred method first.
Click **Create**.
OAuth and IAM need configuration outside this page. See [IAM authentication for AWS Data Sources](#iam-authentication-for-aws-data-sources) and [OAuth for Snowflake](#oauth-for-snowflake).
To remove a method, go to **Platform settings** > **Configuration records**, click the edit icon next to the relevant [configuration record](/6.3/admin/configuration/central-configuration#data-source-authentication), remove the method from the list of values, and click **Save**.
## IAM authentication for AWS Data Sources
You can enable users to create Data Sources that authenticate with AWS Identity and Access Management (IAM) for these Data Source types:
* [MySQL](/6.3/platform-capabilities/core-concepts/data/data-source-connectors/connect-to-mysql)
* [PostgreSQL](/6.3/platform-capabilities/core-concepts/data/data-source-connectors/connect-to-postgres)
* [Redshift](/6.3/platform-capabilities/core-concepts/data/data-source-connectors/connect-to-redshift)
* [S3](/6.3/platform-capabilities/core-concepts/data/data-source-connectors/connect-to-generic-s3)
IAM-authenticated connections can be used only to execute Jobs and Workspaces. Other execution types, such as scheduled Jobs and Domino endpoints, require basic authentication.
To set up IAM authentication:
1. [Configure single sign-on (SSO)](/6.3/admin/identity-and-access/authentication/sso/sso-configuration).
2. [Configure AWS credential propagation](/6.3/admin/identity-and-access/authentication/sso/aws-credential-propagation).
3. Add IAM to the available authentication options for one or more of the supported Data Source types. In the Admin application, go to **Platform settings** > **Configuration records** and click **Add Record**.
4. Enter one of the following keys with the value `AWSIAMRole` or `Basic, AWSIAMRole`:
* `com.cerebro.domino.datasource.MySQLConfig.enabledAuthTypes`
* `com.cerebro.domino.datasource.PostgreSQLConfig.enabledAuthTypes`
* `com.cerebro.domino.datasource.RedshiftConfig.enabledAuthTypes`
* `com.cerebro.domino.datasource.S3Config.enabledAuthTypes`
See the [configuration reference](/6.3/admin/configuration/central-configuration#data-source-authentication) for details about the supported configurations.
5. Click **Create**.
## OAuth for Snowflake
OAuth-backed [Snowflake Data Sources](/6.3/platform-capabilities/core-concepts/data/data-source-connectors/connect-to-snowflake) authenticate and query Snowflake with a JSON Web Token (JWT). Once it is set up, users can select **OAuth** when they create a Snowflake Data Source.
When an OAuth-configured Snowflake Data Source is queried in a Domino execution, the `dominodatalab-data` library uses the Domino JWT token to authenticate against the underlying Snowflake database. No extra user action is needed.
Setup has two halves. First, an External OAuth Security Integration in your Snowflake account dictates what values Snowflake expects the token to carry. Second, Keycloak mappers propagate those values into the Domino JWT access token.
OAuth is not supported for Service Accounts. OAuth-authenticated connections can be used for any execution type *except* Domino endpoints.
Because authentication depends on the Domino JWT token propagating the correct values in the context of an execution, you cannot verify successful authentication when you create a Data Source. The green checkmark that appears when you create a Data Source does not guarantee that authentication succeeds. To confirm successful value propagation in a Domino JWT token, use the `dominodatalab-data` library to query data from an OAuth Data Source in a Domino execution, such as a Workspace.
### Configure the Snowflake security integration
To set up an External OAuth Security Integration in Snowflake, see the [Snowflake external OAuth documentation](https://docs.snowflake.com/en/user-guide/oauth-ext-custom.html#configure-custom-clients-for-external-oauth). Get familiar with the Snowflake terminology and the values that Snowflake needs in the token; correct spelling of the values is critical.
The following is a sample command to create a base security integration that allows all account roles to authenticate. Use it as a starting point for a configuration that fits your security needs.
```sql theme={null}
create security integration
type = external_oauth
enabled = true
external_oauth_type = custom
external_oauth_issuer = 'https:///auth/realms/DominoRealm'
external_oauth_rsa_public_key =
external_oauth_audience_list=('account')
external_oauth_scope_mapping_attribute = 'scp'
external_oauth_token_user_mapping_claim='preferred_username_snowflake'
external_oauth_snowflake_user_mapping_attribute='login_name';
alter security integration set EXTERNAL_OAUTH_ANY_ROLE_MODE = ENABLE;
```
Find the value for these keys:
* `external_oauth_issuer` in Keycloak's **Realm Settings** > **OpenID Endpoint Configuration** > **Issuer**
* `external_oauth_rsa_public_key` in Keycloak's **Realm Settings** > **Keys** > **Public Key from RS256**
The configured security integration (`desc security integration `) should look similar to the following:
### Configure Keycloak
Keycloak passes the values that the Snowflake security integration expects by way of mappers in the `domino-play` client. Each mapper is set to **add to access token**, which creates a key-value entry in the Domino JWT token keyed on **Token Claim Name**.
The value depends on the mapper type. A hardcoded-claim mapper takes a value that propagates equally to all users. A user-attribute mapper takes a **User Attribute** name, reads that attribute from each Keycloak user, and propagates its value, so you can vary the token per user.
Go to the Keycloak admin console at `https:///auth/`, then click **Domino Realm** > **Clients**.
From the lookup list, click **domino-play**, then click **Client scopes** > **domino-play-dedicated**.
Click **Mappers** > **Add mapper** > **By configuration**, choose a mapper from the listed mappings, and configure it.
The definitions in [Keycloak mapper reference](#keycloak-mapper-reference) match the base Snowflake security integration above. For the `scp` claim, use the prepopulated Scope Claim Aggregator mapper. You can write your mappers differently, as long as they conform to the requirements of the Snowflake security integration.
Keycloak propagates the values the mappers produce into the JWT token that authenticates against the External OAuth Security Integration you created in Snowflake.
If you added a mapper that requires a user attribute, add that attribute at **Users** > **View All Users** > `` > **Attributes**. You can automate this to create further roles and mappers.
Go to **Platform settings** > **Configuration records**, click **Advanced settings**, then click **Add Record**. Enter the `com.cerebro.domino.datasource.SnowflakeConfig.enabledAuthTypes` key with the comma-separated values `OAuth` and `KeyPair`. Order matters: the first value becomes the default authentication method for all users. Click **Create**.
#### Keycloak mapper reference
| Field | Value |
| :------------------------------ | :------------------------------------------------------------------ |
| Token Claim Name | `scp` |
| User Attribute | The name of the user attribute containing the Snowflake scope name. |
| Static Values (comma-separated) | `all` |
| Add to access token | `ON` |
| Field | Value |
| :------------------ | :------------------------------------------------------------------------------------------------------------------------------- |
| Token Claim Name | `external_oauth_issuer` |
| Claim JSON Type | `String` |
| Add to access token | `ON` |
| Claim value | The name of your OpenID Endpoint Configuration issuer, from **Realm Settings** > **OpenID Endpoint Configuration** > **Issuer**. |
| Field | Value |
| :------------------ | :------------------------------------------------------- |
| Token Claim Name | `external_oauth_audience_list` |
| Claim JSON Type | `String` |
| Add to access token | `ON` |
| Multivalued | `ON` |
| Claim value | The list of audiences the security integration supports. |
| Field | Value |
| :------------------ | :------------------------------------------------------------ |
| Token Claim Name | `external_oauth_user_mapping_claim` |
| Claim JSON Type | `String` |
| Add to access token | `ON` |
| Claim value | The mapping between the Keycloak user and the Snowflake user. |
| Field | Value |
| :------------------ | :------------------------------------------------- |
| Token Claim Name | `preferred_username_snowflake` |
| Claim JSON Type | `String` |
| Add to access token | `ON` |
| Claim value | The Snowflake username of the authenticating user. |
This mapper pairs with the security integration above, which sets `external_oauth_token_user_mapping_claim='preferred_username_snowflake'` and so instructs Snowflake to read the value of the `preferred_username_snowflake` key.
| Field | Value |
| :------------------ | :--------------------------------------------------------------------------------------------------------------------- |
| Token Claim Name | `external_oauth_rsa_public_key_value` |
| Claim JSON Type | `String` |
| Add to access token | `ON` |
| Claim value | The name of your OpenID Endpoint Configuration issuer, from **Realm Settings** > **Keys** > **Public Key from RS256**. |
| Field | Value |
| :------------------ | :------------------------------------------------------------------- |
| Token Claim Name | `external_oauth_snowflake_user_mapping_attribute` |
| Claim JSON Type | `String` |
| Add to access token | `ON` |
| Claim value | The attribute that maps the access token to a Snowflake user record. |
### Test the OAuth configuration
The user attributes `SCP`, `external_oauth_audience_list`, and `preferred_username_snowflake` must be present in the Domino user JWT token. There are two ways to inject them:
* **Locally in Keycloak**, for quick testing. Add the user attributes manually under the Keycloak user.
* **Through SAML attributes**, for enterprise-level implementations. Passing the user attributes from the identity provider as SAML attributes is the recommended approach. When SAML attributes include the user attributes, additional mappers extract them and make them available to the `domino-play` client.
Configure three identity-provider user attribute mappers:
1. Go to **Domino realm** > **Identity providers** > **Identity provider** > **Mappers**.
2. For **Mapper type**, select `Attribute importer`.
3. In **Attribute Name** and **User Attribute Name**, add the user attributes to pass to the `domino-play` client, for example `SCP`, `external_oauth_audience_list`, and `preferred_username_snowflake`.
To confirm that OAuth works with your Snowflake Data Source, register a Snowflake Data Source, select `OAuth` as the authentication type, open an execution, and query data from the registered Data Source.
To get the token itself, make a GET request to the [Domino API proxy](/6.3/reference/api/domino-api-authentication#use-the-api-proxy-for-domino-api-authentication) by running `curl $DOMINO_API_PROXY/access-token` in an execution, such as a Workspace terminal or a Job script. Paste the token into [jwt.io](https://jwt.io/) to see which values Keycloak propagated into the Domino JWT token.
## Starburst-powered Data Sources
Starburst-powered Data Sources cover MongoDB, Palantir Foundry, tabular S3 with AWS Glue, Starburst JDBC, and Teradata. Only an admin can create them, because each one needs a restart of the Starburst cluster in Domino.
For additional support in working with Starburst, contact your Domino representative.
### Enable Starburst
To enable Starburst-powered Data Sources in your Domino instance, add the following block to `domino.yaml` and `agent.yaml`.
```yaml theme={null}
release_overrides:
starburst-enterprise:
installed: true
```
### Create a Starburst-powered Data Source
Every type starts at **Manage Resources** > **Data Sources** > **Connect to External Data** and ends with a cluster restart. Expand the type you are configuring for the fields it asks for.
Create a Data Source for [MongoDB](https://www.mongodb.com/) that your Domino users can access.
Prerequisites:
* Starburst is enabled in the deployment. See [Enable Starburst](#enable-starburst).
* Network connectivity between MongoDB and your Domino deployment.
* Domino has enabled this feature. If you do not see this Data Source type in your deployment, contact your Customer Success Manager.
To create the Data Source:
1. Go to **Manage Resources** > **Data Sources**.
2. Click **Connect to External Data**.
3. In the New Data Source window, select **MongoDB**.
4. Enter your MongoDB hostname or IP address.
5. Optional: Enter the **Port**.
6. Enter a **Name** and **Description** for this Data Source.
7. Click **Next** twice.
8. Enter your MongoDB credentials.
9. Click **Next**.
10. Select whether **Everyone** can access this Data Source or just **Specific users or organizations**.
11. Click **Finish Setup**. Domino alerts you to restart the cluster.
12. Click **Click here to trigger a restart**, then click **Restart Cluster** to confirm. The restart takes approximately 5 to 10 minutes. When it completes, Domino users can use the MongoDB Data Source.
Create a Data Source for [Palantir Foundry](https://www.palantir.com/platforms/foundry/) that your Domino users can access.
Prerequisites:
* Starburst is enabled in the deployment. See [Enable Starburst](#enable-starburst).
* Network connectivity between Palantir and your Domino deployment.
* Domino has enabled this feature. If you do not see this Data Source type in your deployment, contact your Customer Success Manager.
To create the Data Source:
1. Go to **Manage Resources** > **Data Sources**.
2. Click **Connect to External Data**.
3. In the New Data Source window, select **Palantir**.
4. Enter a **Name**, **Description**, and the Palantir hostname or IP address. Optionally, enter the **Port**.
5. Select **Individual** or **Service Account** as the credential type.
6. Select **Client Credentials** or **OAuth Token** for authenticating to Palantir.
Individual credentials can only use **OAuth Tokens** to authenticate.
7. Enter your Palantir credentials.
8. Select whether **Everyone** can access this Data Source or just **Specific users or organizations**.
9. Domino alerts you to restart the cluster. Click **Click here to trigger a restart** > **Restart Cluster**. The restart takes approximately 5 to 10 minutes. When it completes, Domino users can [use the Palantir Data Source](/6.3/platform-capabilities/core-concepts/data/data-source-connectors/connect-to-palantir).
**Get Palantir client credentials.** To connect with a Service Account, retrieve your Palantir client ID and client secret:
1. Go to your Palantir instance.
2. Go to **Control Panel** > **Third-party applications**. An existing Domino integration appears here.
3. Next to the Domino integration, select **Actions** > **Manage application** to find your client ID.
If no Domino integration exists, [register Domino as a third-party application](https://www.palantir.com/docs/foundry/platform-security-third-party/register-3pa/index.html). Choose **Confidential client** as the client type and save the client secret, because Palantir does not show it again. If the integration already exists and you cannot see the client secret, contact your Palantir administrator.
**Generate a Palantir OAuth token.** To connect with an OAuth token, generate a new [Palantir token](https://www.palantir.com/docs/foundry/platform-security-third-party/user-generated-tokens/#generation).
Only Service Accounts are supported for [tabular S3 with AWS Glue](https://docs.aws.amazon.com/athena/latest/ug/data-sources-glue.html).
Prerequisites:
* Starburst is enabled in the deployment. See [Enable Starburst](#enable-starburst).
* Network connectivity between S3 and your Domino deployment.
* Domino has enabled this feature. If you do not see this Data Source type in your deployment, contact your Customer Success Manager.
To create the Data Source:
1. Go to **Manage Resources** > **Data Sources**.
2. Click **Connect to External Data**.
3. In the New Data Source window, select **Tabular S3 with AWS Glue**.
4. Enter your S3 account ID, database, and region.
5. Enter a name for this Data Source.
6. Click **Next**. In the Credential Type screen, **Service Account** is preselected. Individual accounts are not supported for this Data Source type.
7. Click **Next**.
8. Enter your S3 service account access key ID and secret access key.
9. Click **Next**.
10. Select whether **Everyone** can access this Data Source or just **Specific users or organizations**.
11. Click **Finish Setup**. Domino alerts you to restart the cluster.
12. Click **Click here to trigger a restart**, then click **Restart Cluster** to confirm. The restart takes approximately 5 to 10 minutes.
Starburst JDBC extends [Domino Data Sources](/6.3/platform-capabilities/core-concepts/data/data-source-connectors) to any Starburst-JDBC data entity. IBM DB2, IBM Netezza, and SAP HANA are enabled by default.
Prerequisites:
* Starburst is enabled in the deployment. See [Enable Starburst](#enable-starburst).
* Network connectivity between the data platform and your Domino deployment.
To create the Data Source:
1. Go to **Manage Resources** > **Data Sources**.
2. Click **Connect to External Data**.
3. Select the data entity you want to connect to under **Powered by Starburst JDBC**.
4. Enter the connection parameters.
5. Enter the **Data Source Name** and, optionally, a **Description** that explains the purpose of the Data Source to others.
6. Select the credential type.
7. Optionally, enter your credentials. You can enter these later.
8. Click **Next**.
9. Select whether **Everyone** can access this Data Source or just **Specific users or organizations**.
10. Click **Finish Setup**. Domino alerts you to restart the cluster and provides a link to trigger it.
To connect to a data entity that is not enabled by default, see [Add Starburst JDBC data entities](#add-starburst-jdbc-data-entities).
Create a Data Source for [Teradata](https://www.teradata.com/) that your Domino users can access.
Prerequisites:
* Starburst is enabled in the deployment. See [Enable Starburst](#enable-starburst).
* Network connectivity between Teradata and your Domino deployment.
* Domino has enabled this feature. If you do not see this Data Source type in your deployment, contact your Customer Success Manager.
To create the Data Source:
1. Go to **Manage Resources** > **Data Sources**.
2. Click **Connect to External Data**.
3. In the New Data Source window, select **Teradata**.
4. Enter your Teradata hostname or IP address.
5. Enter a name for this Data Source.
6. Click **Next**.
7. Select the credential type.
8. Optionally, enter your Teradata credentials. You can enter these later.
9. Click **Next**.
10. Select whether **Everyone** can access this Data Source or just **Specific users or organizations**.
11. Click **Finish Setup**. Domino alerts you to restart the cluster.
12. Click **Click here to trigger a restart**, then click **Restart Cluster** to confirm. The restart takes approximately 5 to 10 minutes.
Users must specify the database name in their queries, as in this example where `td_demo` is the database name and `yellow_cab` is the table name:
```
example res = ds.query("select * from td_demo.yellow_cab")
```
### Add Starburst JDBC data entities
Not every JDBC-powered data entity is available by default. You can enable ClickHouse, Druid, Greenplum, MariaDB, Ignite, SingleStore (MemSQL), Synapse, Vertica, and the generic JDBC capability that Starburst provides.
You may need to contact your Customer Success engineer for assistance to add some additional JDBC data entities.
1. Go to **Admin** > **Platform settings** > **Configuration records**.
2. Add or edit the `com.cerebro.domino.datasource.starburstJdbc.enabledConnectors` config option.
3. Add a comma-separated list of data-entity names as the value:
```console theme={null}
clickhouse,druid,db2,synapse,greenplum,ignite,mariadb,singlestore,vertica,generic_jdbc,sap_hana,netezza
```
4. Restart the Domino services to apply the new setting, following the prompt on the admin page.
The newly added data entity then appears in the drop-down list.
### Restart the Starburst cluster
Restart the Starburst cluster in Domino after actions including, but not limited to:
* Creating a Data Source
* Updating a Data Source configuration
* Updating credentials
Domino notifies you to restart your Starburst cluster and provides a link to trigger the restart. The restart takes approximately 5 to 10 minutes and disrupts executions that interact with Starburst-powered Data Sources, so plan it for a window when none are running.
### Starburst limitations
* Your Domino representative must enable this feature. If you do not see these Data Source types in your deployment, contact your Customer Success engineer.
* In a hybrid deployment, Starburst-powered Data Sources are only available from the local Data Plane.
## Data Plane accessibility
[Data Planes](/6.3/platform-capabilities/core-concepts/data-plane) in a Domino Nexus hybrid deployment have caveats around which of them can reach a Data Source.
| Data Source type | Remote Data Planes | Local Data Plane |
| :----------------------------------------------------------- | :--------------------------------------------------------------------------------- | :---------------------------------------------- |
| Domino native Data Sources using individual credentials | Automatically accessible and unconfigurable. | Automatically accessible and unconfigurable. |
| Domino native Data Sources using Service Account credentials | Accessible, with the ability to disable access from any or all remote Data Planes. | Accessible, with the ability to disable access. |
| Starburst-powered Data Sources | Unsupported. | Supported. |
You can disable adding a Data Source in remote Workspaces by setting the `EnableAddingDataSourcesInRemoteWorkspaces` [feature flag](/6.3/admin/configuration/feature-flags) to `false`.
## Related
* [External Data Volumes](/6.3/admin/data-administration/external-data/external-data-volumes) mount remote file systems into executions instead of querying them.
* [Data Source connectors](/6.3/platform-capabilities/core-concepts/data/data-source-connectors) covers what practitioners do with a Data Source once you create it.
* [Monitor Data Source logs](/6.3/admin/operations/audit-logs/monitor-data-source-logs) audits Data Source activity.
* [Domino Datasets](/6.3/platform-capabilities/core-concepts/data/datasets) are where practitioners land data pulled from a Data Source.
## Related topics
- [Manage data](/6.3/admin/data-administration/index.md)
- [Manage External Data Volumes](/6.3/admin/data-administration/external-data/external-data-volumes/index.md)
- [Data Source use cases](/6.3/reference/api/data_api/datasource_usecases/index.md)
- [Use Data Sources](/6.3/platform-capabilities/core-concepts/data/data-source-connectors/use-data-sources.md)