Manage Data Sources

A Data Source is a reusable connection to an external database, data service, or object store. You configure the connection once, and practitioners query it from a Workspace, Job, or app without handling hostnames, ports, or drivers, and write results back to a Domino Dataset. Admins can expose a Data Source to everyone in the deployment. When a practitioner creates a Data Source, they must name the users or organizations that can access it.

Create a Data Source

Open the Data Source wizard

Go to Manage Resources > Data Sources, then click Connect to External Data.

Configure the connection

Select your Data Source, then enter the configuration details for accessing it. The details vary depending on the Data Source type.

Select the credential type

Credential type	Who supplies the credentials	Use it when
Individual	Each user provides their own credentials before using the Data Source.	Access must be attributable to a person, or the underlying system enforces per-user permissions.
Service Account	You provide one set of static credentials that Domino applies on behalf of every user with permission to the Data Source.	The team shares one identity against the external system.

End users cannot access or extract Service Account credentials, nor can they manage user-level permissions on such Data Sources. Domino stores Data Source credentials in the Domino secret store, which is backed by HashiCorp Vault.

OAuth authentication is not supported for Service Accounts. The Data Source itself does not distinguish end users, because the same credentials authenticate every query.

Enter and test the credentials

Enter your credentials for Data Source authentication, then click Test Credentials to verify that authentication works, or click Skip for Now to continue.

Set access and finish

Select whether Everyone can use this Data Source or just Specific users or organizations, then click Finish Setup.

Authentication methods

Data Sources support multiple authentication methods to meet different security requirements. Which methods are available depends on the Data Source type, and you choose which of them your organization can use. Enable OAuth or IAM for more secure connections than credentials, enable several methods at once when different teams need different ones, and require certificate authentication where enterprise security standards demand it. Enabling a method takes one configuration record, and requires the SysAdmin role.

Open the configuration records

In the Admin panel, go to Platform settings > Configuration records, then click Add Record.

Add the authentication configuration

Add a key for your Data Source type. The Data Source authentication section of the configuration records reference lists the key for each type.Enter the value as comma-separated authentication types, for example certauth,basic,oauth for PostgreSQL with certificate support, or basic for IBM DB2 with basic authentication only.

The first value becomes the default authentication method for all users. Put your preferred method first.

Click Create.

Complete any additional setup

OAuth and IAM need configuration outside this page. See IAM authentication for AWS Data Sources and OAuth for Snowflake.

To remove a method, go to Platform settings > Configuration records, click the edit icon next to the relevant configuration record, remove the method from the list of values, and click Save.

IAM authentication for AWS Data Sources

You can enable users to create Data Sources that authenticate with AWS Identity and Access Management (IAM) for these Data Source types:

IAM-authenticated connections can be used only to execute Jobs and Workspaces. Other execution types, such as scheduled Jobs and Domino endpoints, require basic authentication.

To set up IAM authentication:

Configure single sign-on (SSO).
Configure AWS credential propagation.
Add IAM to the available authentication options for one or more of the supported Data Source types. In the Admin application, go to Platform settings > Configuration records and click Add Record.
Enter one of the following keys with the value AWSIAMRole or Basic, AWSIAMRole:
- com.cerebro.domino.datasource.MySQLConfig.enabledAuthTypes
- com.cerebro.domino.datasource.PostgreSQLConfig.enabledAuthTypes
- com.cerebro.domino.datasource.RedshiftConfig.enabledAuthTypes
- com.cerebro.domino.datasource.S3Config.enabledAuthTypes
See the configuration reference for details about the supported configurations.
Click Create.

OAuth for Snowflake

OAuth-backed Snowflake Data Sources authenticate and query Snowflake with a JSON Web Token (JWT). Once it is set up, users can select OAuth when they create a Snowflake Data Source. When an OAuth-configured Snowflake Data Source is queried in a Domino execution, the dominodatalab-data library uses the Domino JWT token to authenticate against the underlying Snowflake database. No extra user action is needed. Setup has two halves. First, an External OAuth Security Integration in your Snowflake account dictates what values Snowflake expects the token to carry. Second, Keycloak mappers propagate those values into the Domino JWT access token.

OAuth is not supported for Service Accounts. OAuth-authenticated connections can be used for any execution type except Domino endpoints.

Because authentication depends on the Domino JWT token propagating the correct values in the context of an execution, you cannot verify successful authentication when you create a Data Source. The green checkmark that appears when you create a Data Source does not guarantee that authentication succeeds. To confirm successful value propagation in a Domino JWT token, use the dominodatalab-data library to query data from an OAuth Data Source in a Domino execution, such as a Workspace.

Configure the Snowflake security integration

To set up an External OAuth Security Integration in Snowflake, see the Snowflake external OAuth documentation. Get familiar with the Snowflake terminology and the values that Snowflake needs in the token; correct spelling of the values is critical. The following is a sample command to create a base security integration that allows all account roles to authenticate. Use it as a starting point for a configuration that fits your security needs.

create security integration <external_oauth_name>
    type = external_oauth
    enabled = true
    external_oauth_type = custom
    external_oauth_issuer = 'https://<domino-domain>/auth/realms/DominoRealm'
    external_oauth_rsa_public_key = <keycloak-realm-settings-public-key>
    external_oauth_audience_list=('account')
    external_oauth_scope_mapping_attribute = 'scp'
    external_oauth_token_user_mapping_claim='preferred_username_snowflake'
    external_oauth_snowflake_user_mapping_attribute='login_name';

alter security integration <external_oauth_name> set EXTERNAL_OAUTH_ANY_ROLE_MODE = ENABLE;

Find the value for these keys:

external_oauth_issuer in Keycloak’s Realm Settings > OpenID Endpoint Configuration > Issuer
external_oauth_rsa_public_key in Keycloak’s Realm Settings > Keys > Public Key from RS256

The configured security integration (desc security integration <external_oauth_name>) should look similar to the following:

Snowflake security integration description output showing the configured external OAuth keys.

Configure Keycloak

Keycloak passes the values that the Snowflake security integration expects by way of mappers in the domino-play client. Each mapper is set to add to access token, which creates a key-value entry in the Domino JWT token keyed on Token Claim Name. The value depends on the mapper type. A hardcoded-claim mapper takes a value that propagates equally to all users. A user-attribute mapper takes a User Attribute name, reads that attribute from each Keycloak user, and propagates its value, so you can vary the token per user.

Open the Keycloak client scopes

Go to the Keycloak admin console at https://<domino-domain>/auth/, then click Domino Realm > Clients.

Keycloak client dashboard listing the realm clients.

From the lookup list, click domino-play, then click Client scopes > domino-play-dedicated.

Keycloak client scopes tab for the domino-play client.

Keycloak Mappers tab for the domino-play-dedicated client scope.

Add the mappers

Click Mappers > Add mapper > By configuration, choose a mapper from the listed mappings, and configure it.

Keycloak Add mapper by configuration dialog.

The definitions in Keycloak mapper reference match the base Snowflake security integration above. For the scp claim, use the prepopulated Scope Claim Aggregator mapper. You can write your mappers differently, as long as they conform to the requirements of the Snowflake security integration.Keycloak propagates the values the mappers produce into the JWT token that authenticates against the External OAuth Security Integration you created in Snowflake.

Add the user attributes

If you added a mapper that requires a user attribute, add that attribute at Users > View All Users > <user ID> > Attributes. You can automate this to create further roles and mappers.

Keycloak user attributes tab showing a custom attribute.

Enable OAuth for Snowflake in Domino

Go to Platform settings > Configuration records, click Advanced settings, then click Add Record. Enter the com.cerebro.domino.datasource.SnowflakeConfig.enabledAuthTypes key with the comma-separated values OAuth and KeyPair. Order matters: the first value becomes the default authentication method for all users. Click Create.

Keycloak mapper reference

SCP

Field	Value
Token Claim Name	`scp`
User Attribute	The name of the user attribute containing the Snowflake scope name.
Static Values (comma-separated)	`all`
Add to access token	`ON`

Keycloak mapper configuration for the scp claim.

Issuer

Field	Value
Token Claim Name	`external_oauth_issuer`
Claim JSON Type	`String`
Add to access token	`ON`
Claim value	The name of your OpenID Endpoint Configuration issuer, from Realm Settings > OpenID Endpoint Configuration > Issuer.

Audience

Field	Value
Token Claim Name	`external_oauth_audience_list`
Claim JSON Type	`String`
Add to access token	`ON`
Multivalued	`ON`
Claim value	The list of audiences the security integration supports.

User mapping

Field	Value
Token Claim Name	`external_oauth_user_mapping_claim`
Claim JSON Type	`String`
Add to access token	`ON`
Claim value	The mapping between the Keycloak user and the Snowflake user.

Preferred username

Field	Value
Token Claim Name	`preferred_username_snowflake`
Claim JSON Type	`String`
Add to access token	`ON`
Claim value	The Snowflake username of the authenticating user.

This mapper pairs with the security integration above, which sets external_oauth_token_user_mapping_claim='preferred_username_snowflake' and so instructs Snowflake to read the value of the preferred_username_snowflake key.

Keycloak mapper configuration for the preferred username claim.

RSA public key value

Field	Value
Token Claim Name	`external_oauth_rsa_public_key_value`
Claim JSON Type	`String`
Add to access token	`ON`
Claim value	The name of your OpenID Endpoint Configuration issuer, from Realm Settings > Keys > Public Key from RS256.

Keycloak mapper configuration for the RSA public key claim.

Snowflake user mapping attribute

Field	Value
Token Claim Name	`external_oauth_snowflake_user_mapping_attribute`
Claim JSON Type	`String`
Add to access token	`ON`
Claim value	The attribute that maps the access token to a Snowflake user record.

Test the OAuth configuration

The user attributes SCP, external_oauth_audience_list, and preferred_username_snowflake must be present in the Domino user JWT token. There are two ways to inject them:

Locally in Keycloak, for quick testing. Add the user attributes manually under the Keycloak user.
Through SAML attributes, for enterprise-level implementations. Passing the user attributes from the identity provider as SAML attributes is the recommended approach. When SAML attributes include the user attributes, additional mappers extract them and make them available to the domino-play client.

Configure three identity-provider user attribute mappers:

Go to Domino realm > Identity providers > Identity provider > Mappers.
For Mapper type, select Attribute importer.
In Attribute Name and User Attribute Name, add the user attributes to pass to the domino-play client, for example SCP, external_oauth_audience_list, and preferred_username_snowflake.

Keycloak identity provider mapper configured as an attribute importer.

To confirm that OAuth works with your Snowflake Data Source, register a Snowflake Data Source, select OAuth as the authentication type, open an execution, and query data from the registered Data Source.

To get the token itself, make a GET request to the Domino API proxy by running curl $DOMINO_API_PROXY/access-token in an execution, such as a Workspace terminal or a Job script. Paste the token into jwt.io to see which values Keycloak propagated into the Domino JWT token.

Starburst-powered Data Sources

Starburst-powered Data Sources cover MongoDB, Palantir Foundry, tabular S3 with AWS Glue, Starburst JDBC, and Teradata. Only an admin can create them, because each one needs a restart of the Starburst cluster in Domino. For additional support in working with Starburst, contact your Domino representative.

Enable Starburst

To enable Starburst-powered Data Sources in your Domino instance, add the following block to domino.yaml and agent.yaml.

release_overrides:
  starburst-enterprise:
    installed: true

Create a Starburst-powered Data Source

Every type starts at Manage Resources > Data Sources > Connect to External Data and ends with a cluster restart. Expand the type you are configuring for the fields it asks for.

MongoDB

Create a Data Source for MongoDB that your Domino users can access.Prerequisites:

Starburst is enabled in the deployment. See Enable Starburst.
Network connectivity between MongoDB and your Domino deployment.
Domino has enabled this feature. If you do not see this Data Source type in your deployment, contact your Customer Success Manager.

To create the Data Source:

Go to Manage Resources > Data Sources.
Click Connect to External Data.
In the New Data Source window, select MongoDB.
Enter your MongoDB hostname or IP address.
Optional: Enter the Port.
Enter a Name and Description for this Data Source.
Click Next twice.
Enter your MongoDB credentials.
Click Next.
Select whether Everyone can access this Data Source or just Specific users or organizations.
Click Finish Setup. Domino alerts you to restart the cluster.
Click Click here to trigger a restart, then click Restart Cluster to confirm. The restart takes approximately 5 to 10 minutes. When it completes, Domino users can use the MongoDB Data Source.

Palantir Foundry

Create a Data Source for Palantir Foundry that your Domino users can access.Prerequisites:

Starburst is enabled in the deployment. See Enable Starburst.
Network connectivity between Palantir and your Domino deployment.
Domino has enabled this feature. If you do not see this Data Source type in your deployment, contact your Customer Success Manager.

To create the Data Source:

Go to Manage Resources > Data Sources.
Click Connect to External Data.
In the New Data Source window, select Palantir.
Enter a Name, Description, and the Palantir hostname or IP address. Optionally, enter the Port.
Select Individual or Service Account as the credential type.
Select Client Credentials or OAuth Token for authenticating to Palantir.
Individual credentials can only use OAuth Tokens to authenticate.
Enter your Palantir credentials.
Select whether Everyone can access this Data Source or just Specific users or organizations.
Domino alerts you to restart the cluster. Click Click here to trigger a restart > Restart Cluster. The restart takes approximately 5 to 10 minutes. When it completes, Domino users can use the Palantir Data Source.

Get Palantir client credentials. To connect with a Service Account, retrieve your Palantir client ID and client secret:

Go to your Palantir instance.
Go to Control Panel > Third-party applications. An existing Domino integration appears here.
Next to the Domino integration, select Actions > Manage application to find your client ID.

If no Domino integration exists, register Domino as a third-party application. Choose Confidential client as the client type and save the client secret, because Palantir does not show it again. If the integration already exists and you cannot see the client secret, contact your Palantir administrator.Generate a Palantir OAuth token. To connect with an OAuth token, generate a new Palantir token.

Tabular S3 with AWS Glue

Only Service Accounts are supported for tabular S3 with AWS Glue.Prerequisites:

Starburst is enabled in the deployment. See Enable Starburst.
Network connectivity between S3 and your Domino deployment.
Domino has enabled this feature. If you do not see this Data Source type in your deployment, contact your Customer Success Manager.

To create the Data Source:

Go to Manage Resources > Data Sources.
Click Connect to External Data.
In the New Data Source window, select Tabular S3 with AWS Glue.
Enter your S3 account ID, database, and region.
Enter a name for this Data Source.
Click Next. In the Credential Type screen, Service Account is preselected. Individual accounts are not supported for this Data Source type.
Click Next.
Enter your S3 service account access key ID and secret access key.
Click Next.
Select whether Everyone can access this Data Source or just Specific users or organizations.
Click Finish Setup. Domino alerts you to restart the cluster.
Click Click here to trigger a restart, then click Restart Cluster to confirm. The restart takes approximately 5 to 10 minutes.

Starburst JDBC

Starburst JDBC extends Domino Data Sources to any Starburst-JDBC data entity. IBM DB2, IBM Netezza, and SAP HANA are enabled by default.Prerequisites:

Starburst is enabled in the deployment. See Enable Starburst.
Network connectivity between the data platform and your Domino deployment.

To create the Data Source:

Go to Manage Resources > Data Sources.
Click Connect to External Data.
Select the data entity you want to connect to under Powered by Starburst JDBC.
Enter the connection parameters.
Enter the Data Source Name and, optionally, a Description that explains the purpose of the Data Source to others.
Select the credential type.
Optionally, enter your credentials. You can enter these later.
Click Next.
Select whether Everyone can access this Data Source or just Specific users or organizations.
Click Finish Setup. Domino alerts you to restart the cluster and provides a link to trigger it.

To connect to a data entity that is not enabled by default, see Add Starburst JDBC data entities.

Teradata

Create a Data Source for Teradata that your Domino users can access.Prerequisites:

Starburst is enabled in the deployment. See Enable Starburst.
Network connectivity between Teradata and your Domino deployment.
Domino has enabled this feature. If you do not see this Data Source type in your deployment, contact your Customer Success Manager.

To create the Data Source:

Go to Manage Resources > Data Sources.
Click Connect to External Data.
In the New Data Source window, select Teradata.
Enter your Teradata hostname or IP address.
Enter a name for this Data Source.
Click Next.
Select the credential type.
Optionally, enter your Teradata credentials. You can enter these later.
Click Next.
Select whether Everyone can access this Data Source or just Specific users or organizations.
Click Finish Setup. Domino alerts you to restart the cluster.
Click Click here to trigger a restart, then click Restart Cluster to confirm. The restart takes approximately 5 to 10 minutes.

Users must specify the database name in their queries, as in this example where td_demo is the database name and yellow_cab is the table name:

example res = ds.query("select * from td_demo.yellow_cab")

Add Starburst JDBC data entities

Not every JDBC-powered data entity is available by default. You can enable ClickHouse, Druid, Greenplum, MariaDB, Ignite, SingleStore (MemSQL), Synapse, Vertica, and the generic JDBC capability that Starburst provides.

You may need to contact your Customer Success engineer for assistance to add some additional JDBC data entities.

Go to Admin > Platform settings > Configuration records.
Add or edit the com.cerebro.domino.datasource.starburstJdbc.enabledConnectors config option.

Add a comma-separated list of data-entity names as the value:

clickhouse,druid,db2,synapse,greenplum,ignite,mariadb,singlestore,vertica,generic_jdbc,sap_hana,netezza

Restart the Domino services to apply the new setting, following the prompt on the admin page.

The newly added data entity then appears in the drop-down list.

Restart the Starburst cluster

Restart the Starburst cluster in Domino after actions including, but not limited to:

Creating a Data Source
Updating a Data Source configuration
Updating credentials

Domino notifies you to restart your Starburst cluster and provides a link to trigger the restart. The restart takes approximately 5 to 10 minutes and disrupts executions that interact with Starburst-powered Data Sources, so plan it for a window when none are running.

Starburst limitations

Your Domino representative must enable this feature. If you do not see these Data Source types in your deployment, contact your Customer Success engineer.
In a hybrid deployment, Starburst-powered Data Sources are only available from the local Data Plane.

Data Plane accessibility

Data Planes in a Domino Nexus hybrid deployment have caveats around which of them can reach a Data Source.

Data Source type	Remote Data Planes	Local Data Plane
Domino native Data Sources using individual credentials	Automatically accessible and unconfigurable.	Automatically accessible and unconfigurable.
Domino native Data Sources using Service Account credentials	Accessible, with the ability to disable access from any or all remote Data Planes.	Accessible, with the ability to disable access.
Starburst-powered Data Sources	Unsupported.	Supported.

You can disable adding a Data Source in remote Workspaces by setting the EnableAddingDataSourcesInRemoteWorkspaces feature flag to false.

External Data Volumes mount remote file systems into executions instead of querying them.
Data Source connectors covers what practitioners do with a Data Source once you create it.
Monitor Data Source logs audits Data Source activity.
Domino Datasets are where practitioners land data pulled from a Data Source.

Architecture & deployment

Domino installation

Identity & access

Infrastructure & compute

Platform configuration

Data administration

Operations, monitoring & cost

Security & compliance

Troubleshooting

Manage Data Sources

Create a Data Source

Authentication methods

IAM authentication for AWS Data Sources

OAuth for Snowflake

Configure the Snowflake security integration

Configure Keycloak

Keycloak mapper reference

Test the OAuth configuration

Starburst-powered Data Sources

Enable Starburst

Create a Starburst-powered Data Source

Add Starburst JDBC data entities

Restart the Starburst cluster

Starburst limitations

Data Plane accessibility

​Create a Data Source

​Authentication methods

​IAM authentication for AWS Data Sources

​OAuth for Snowflake

​Configure the Snowflake security integration

​Configure Keycloak

​Keycloak mapper reference

​Test the OAuth configuration

​Starburst-powered Data Sources

​Enable Starburst

​Create a Starburst-powered Data Source

​Add Starburst JDBC data entities

​Restart the Starburst cluster

​Starburst limitations

​Data Plane accessibility

​Related

Create a Data Source

Authentication methods

IAM authentication for AWS Data Sources

OAuth for Snowflake

Configure the Snowflake security integration

Configure Keycloak

Keycloak mapper reference

Test the OAuth configuration

Starburst-powered Data Sources

Enable Starburst

Create a Starburst-powered Data Source

Add Starburst JDBC data entities

Restart the Starburst cluster

Starburst limitations

Data Plane accessibility

Related