Data Planes administration

A Data Plane is a compute location where Domino runs workloads. A Domino Nexus deployment can run several, in different regions or clouds. Manage all of them from the admin portal at Manage resources > Data Planes. Adding a Data Plane takes two halves: install it on its Kubernetes cluster, then register it here. Everything below covers the control plane side, from that registration through to archiving.

Register a Data Plane

Registration is the second half of bringing up a Data Plane. Install the Data Plane in its Kubernetes cluster first, so its data-plane-agent pod is running and logging Secret agent-bootstrap not found; waiting.... Registration exchanges configuration data and cryptographic keys between the two planes, which is what that pod is waiting for. Before you start, confirm you have admin access to the control plane and kubectl access to the Data Plane cluster.

Validate the configuration records

Check that the required Data Plane configuration records are set. They are auto-populated for hybrid-enabled control planes.

In the admin portal, go to Manage resources > Data Planes > Register Data Plane, then set these fields:

Field	What it sets
Name	An arbitrary, human-readable name for the Data Plane.
Namespace	The Kubernetes namespace to install the Data Plane into on the Data Plane cluster. This namespace is created if it does not already exist.
Storage Class	The Kubernetes StorageClass name to use for execution volumes in this Data Plane. If blank, it defaults to the control plane’s storage class specified by `com.cerebro.domino.computegrid.kubernetes.volume.storageClass` in the configuration records (default: `dominodisk`).
Data Plane Hostname	The subdomain component of the DNS name used for the Data Plane. If the control plane is `example.com` and the Data Plane is `dp.example.com`, enter `dp`. Append `:` and a port number if your Data Plane load balancer runs on a port other than 443. See Enable a Data Plane for Workspaces and Datasets.

Three more fields sit behind Advanced:

Advanced field	What it sets
Disable File Sync	Turns off Domino File System (DFS) file sync in this Data Plane. See Disable file sync.
Enable Istio	Indicates that Istio runs in this Data Plane’s cluster. Data Planes with Istio must have this set; setting the Helm value alone is not sufficient.
Override S3 Endpoint URL	Overrides the default S3 endpoint URL for this Data Plane.

Click Done.

Generate a Data Plane token

When registration is complete, create a bootstrap token:

In the admin portal, go to Manage Resources > Data Planes.
In the Actions column, click Setup for the Data Plane you are registering.
Click Load bootstrap token.
When the token loads, click Copy token.

The token is valid for 10 minutes. If you exceed the 10 minute window, load a new token through the same steps.

Run the bootstrap script

Make sure your default Kubernetes context is the cluster you want to deploy the Data Plane into.

If the Data Plane cluster’s API server is not publicly reachable, for example a private EKS cluster built with the Domino Terraform module, open that connectivity before you run the script and keep it open until the script finishes. See Reach the cluster through the bastion SSH tunnel.

Obtain the Kubernetes context of the Data Plane:

tsh login && tsh kube login <dp-deploy-id>

Download the bootstrap-data-plane.sh script from the Nexus repository on GitHub and save it as an executable file:

curl -Sso bootstrap-data-plane.sh \
  https://raw.githubusercontent.com/dominodatalab/nexus/refs/heads/main/bootstrap-data-plane.sh && \
  chmod +x bootstrap-data-plane.sh

Run the script, replacing <token> with the token you copied:
./bootstrap-data-plane.sh <token>

The script reports that it created a secret and exits. The Data Plane becomes healthy in the admin portal within a couple of minutes.

Verify the registration

In the admin portal, go to Manage resources > Data Planes and confirm the Data Plane reports Healthy. From the Data Plane cluster, confirm every node is Ready and every pod is Running or Completed:

kubectl get nodes
kubectl get pods -n domino-compute

Troubleshoot the registration

The token expired

The bootstrap token is valid for 10 minutes. If more time passed between copying it and running the script, load a new token and run the script again.

The Data Plane stays unhealthy

Read the agent logs, which name the reason the agent has not completed its handshake:

kubectl logs -n domino-compute -l app.kubernetes.io/name=data-plane-agent

The keys expired after a long outage

A Data Plane that stayed offline for more than 4 days can have expired security keys. Force a re-bootstrap:

./bootstrap-data-plane.sh --force <token>

Only use --force when the agent logs report key expiration. If they report waiting..., run the script without it.

Never delete or modify the data-plane-config or data-plane-config-extra ConfigMaps. Recovering a Data Plane after either one is lost is extremely difficult.

Enable a Data Plane for Workspaces and Datasets

Workspaces need a direct connection from the user’s browser to the Data Plane, so a Data Plane that serves interactive workloads needs its own hostname, load balancing, and TLS.

Configure the hostname

Serve the Data Plane from a subdomain of the domain used for the control plane. If users connect to Domino at example.com, then Data Planes must be served from data-plane.example.com.

Configure load balancing

Resolve that hostname to a load balancer that routes traffic to port 8080 on pods with this label selector:

app.kubernetes.io/component: auth-proxy
app.kubernetes.io/instance: auth-proxy
app.kubernetes.io/name: auth-proxy

Use a NodePort service plus a load balancer on-premises, or a LoadBalancer service type in the major cloud providers.

Configure TLS

Users must connect to Data Planes over TLS (HTTPS). If a load balancer routes traffic to the Data Plane, the simplest option is to configure that load balancer to serve valid TLS certificates for the domain. To serve certificates that you provide from the Data Plane itself:

Create a Kubernetes secret containing the certificates:

kubectl create secret tls custom-certs -n <data plane namespace> --cert=<cert file> --key=<key file>

Set this value when you deploy the Data Plane Helm chart:

--set auth-proxy.config.nginx.tlsSecretName=custom-certs

Configure Datasets

Datasets on a remote Data Plane need the volume operator and the filetask operator, set under release_overrides in the Data Plane’s configuration file before you apply it:

release_overrides:
  data-plane:
    chart_values:
      enable_volume_operator: true
      enable_filetask_operator: true

Edit a registered Data Plane

Go to Manage resources > Data Planes, click the three-dot menu next to the Data Plane, and select Edit. What you can change varies by field:

Field	What changing it means
Name	Change it freely. Tell your users, since they select Data Planes by name.
Namespace	Migrating a Data Plane to a new namespace is technically possible but not generally supported. Contact Domino for support.
Storage Class	Does not affect running executions, but applies to new ones. This can disrupt existing Workspaces.
Hostname	Affects running executions. Re-run the registration setup for the Data Plane, then restart the `auth-proxy` pod in the Data Plane namespace.
Disable File Sync	Change it at any time. It applies to newly launched workloads.
Enable Istio	Enabling or disabling Istio support on an existing Data Plane is possible but not trivial. Contact Domino for support.

Disable file sync

When data must not leave a specific Data Plane, disable file sync on it to prevent anyone pushing data from that Data Plane to the control plane or to another Data Plane. On a Data Plane with file sync disabled:

Users can pull Project files.
Users can pull and push any mounted Git repositories.
Users cannot push changes to the Domino File System (DFS). Domino warns before starting an execution that cannot push to DFS, and the job history records that no artifacts or output files were captured because the execution ran in a restricted Data Plane.

To disable file sync:

In the admin portal, go to Manage resources > Data Planes.
Click the three-dot menu next to the Data Plane and click Edit.
Click Advanced.
Select Disable File Sync, then click Save.

Hardware Tiers for Data Planes

Hardware tiers are how users reach a Data Plane. Every hardware tier is assigned to one Data Plane, so granting a user access to a hardware tier grants them access to run workloads on that Data Plane with the resources the tier defines. Create and restrict those tiers from Manage a hardware tier, which covers the Data Plane field on the tier form, the node labels a Data Plane’s nodes need, and how to limit a tier to specific organizations.

Upgrade a Data Plane

Keep your remote Data Planes in sync with the control plane to maintain compatibility and access new features. A Data Plane upgrade replaces the fleetcommand-agent and Domino services running in the Data Plane cluster; the same Platform Operator that installed the Data Plane also reconciles the upgrade.

Before you start

The control plane must already be running the target version. Running a Data Plane newer than its control plane is not supported.
Check version compatibility below.
The Platform Operator and the ddlctl command line must be installed against the Data Plane cluster’s Kubernetes context. The UI Upgrade button uses the operator that already runs in the Data Plane cluster; the manual path uses ddlctl from a workstation with that same cluster context.
Read Before you upgrade for cluster-health, backup, and maintenance-mode guidance. It is written for the control plane, but the same checks apply to a Data Plane cluster.

Version compatibility

Starting with Domino 6.1, control planes and Data Planes are compatible across two regular releases. A control plane is compatible with the current and one past regular release of the Data Plane. For instance, control plane 6.2.x is compatible with Data Planes 6.2.x and 6.1.x.

Upgrade the control plane before any Data Plane. Running a Data Plane version newer than the control plane is not supported.

When a Data Plane’s version falls outside that range, the admin portal reports it:

Status	What it means
Degraded	The Data Plane runs an older version. New executions are not permitted until you upgrade it.
Error	Communication between the Data Plane and the control plane is failing.
Disconnected	The Data Plane is not reachable.

Check the deployment type

Which upgrade methods are available depends on whether the Platform Operator or Helm manages the Data Plane. Run this against the Data Plane cluster:

kubectl get domino -A

A domino custom resource means the Platform Operator manages it, which is the default from Domino 6.1. No custom resource means it is a legacy Helm deployment, and the methods below do not apply.

Choose an upgrade method

A remote Data Plane managed by the Platform Operator has two self-service upgrade paths. Both drive the same underlying reconciliation: they update the Domino custom resource in the Data Plane cluster, which the Platform Operator picks up and applies through a fleetcommand-agent job. Either method preserves the existing custom resource, including its .spec.config.release_overrides.

Method	When to use	What runs
UI Upgrade button	The Data Plane was registered against a control plane that supports remote upgrades and its Upgrade action is available.	The control plane instructs the Data Plane’s operator to reconcile against the target version.
Manual upgrade with `ddlctl`	The UI Upgrade action is greyed out, or you need to review or change the configuration file (`release_overrides`, image registry, offline settings) before applying.	You run `ddlctl upgrade domino` against the Data Plane cluster’s Kubernetes context.

A third path exists for Data Planes deployed through Fleetcommand, but only inside a Domino-operated deployment. Contact Domino Support if a Data Plane you administer is managed that way.

Upgrade from the UI

Use the Domino UI when the Upgrade action is available and you do not need to change the Data Plane’s configuration file as part of the upgrade. The action requires both planes to be running Domino 6.1.2 or later under the Platform Operator, and the Data Plane to have online access to the deployment artifacts.

In the admin portal, go to Manage resources > Data Planes.
Identify the Data Plane that needs an upgrade by looking for a status message indicating “Data Plane is running an older version” or “Data Plane is running a different version”.
Click the three vertical dots next to the Data Plane.
Select Upgrade from the dropdown menu.

The Upgrade option is greyed out when the Data Plane already matches the control plane’s version, when Helm rather than the Platform Operator manages it, when it runs 6.1.0 or 6.1.1, or when the upgrade path is not supported from the UI. Upgrade manually with ddlctl instead.

Upgrade manually with ddlctl

Use the manual path when you need to review or change the Data Plane’s configuration before applying, when the UI button is unavailable, or when the Data Plane is air-gapped. Run these commands from a workstation whose default Kubernetes context targets the Data Plane cluster.

Get the $FLEETCOMMAND_AGENT_TAG for your target release from the releases page. Export it so the commands below can reuse it.
export FLEETCOMMAND_AGENT_TAG=<tag from the releases page>
Confirm you are pointed at the Data Plane cluster, then check for drifted Helm releases before starting:
ddlctl get domino ddlctl get helmrelease -A --status stalled=true
Any stalled releases signal out-of-band changes the upgrade does not natively preserve. Reconcile them into the configuration file or expect to reapply them after the upgrade. See Drift detection for the full mechanics.
Run the upgrade:
ddlctl upgrade domino --agent-version $FLEETCOMMAND_AGENT_TAG
This fetches the existing .spec.config from the Data Plane’s Domino custom resource, upgrades it to the latest schema, and opens a text editor for you to review or modify the configuration. When you save and close the editor, ddlctl patches the Domino custom resource with the new configuration, agent tag, and version, then tails the fleetcommand-agent logs while the operator reconciles. Alternatively, run ddlctl create config --from-domino --agent-version $FLEETCOMMAND_AGENT_TAG first if you want to migrate the configuration file as a standalone step and inspect the diff before applying.
To dry-run the upgrade before it hits the cluster, follow Dry run an upgrade of Domino. The dry-run mode applies to Data Plane clusters the same way it applies to control-plane clusters.

Handle release overrides

release_overrides live under .spec.config.release_overrides on the Data Plane’s Domino custom resource. They override Helm chart values for the services that the fleetcommand-agent deploys into the Data Plane. A typical example, adding CPU requests and limits to a service that ships without them:

spec:
  config:
    release_overrides:
      domino-connect:
        chart_values:
          resources:
            requests:
              cpu: "2"
            limits:
              cpu: "4"

Both the UI Upgrade button and ddlctl upgrade domino read the existing .spec.config, so overrides set on the current Data Plane remain in place across the upgrade. There is no need to re-apply them from scratch. For reference, examples of release_overrides shapes appear in the per-cloud install guides (AWS, Azure, GCP) and in deployment identification.

Starting with the Helm 4 upgrade, chart values are applied to Kubernetes with server-side apply, so any mistyped or invalid field in release_overrides now fails the upgrade instead of being silently ignored. This is a change from earlier releases and applies whether you upgrade from the UI or with ddlctl.

Before running the upgrade, validate any changes to release_overrides:

Diff your intended configuration against what is applied today:
ddlctl create config --from-domino --agent-version $FLEETCOMMAND_AGENT_TAG
Edit the exported file, then compare it against the live .spec.config.
Dry-run the upgrade (see Dry run an upgrade of Domino) so the operator reports what it would change without applying it.
Check fleetcommand-agent logs during the reconciliation for release_overrides validation errors; the job fails fast on a rejected value.

Air-gapped upgrades

An air-gapped Data Plane pulls images from a private registry rather than quay.io, so the upgrade has to refresh that registry with the target release’s images before the operator can pull them.

Generate the target release’s image list against the Data Plane’s $FLEETCOMMAND_AGENT_TAG:
ddlctl images --agent-version $FLEETCOMMAND_AGENT_TAG -o json > images.json
Copy the images into the Data Plane’s private registry with either the Skopeo one-shot or the two-step tarball copy described in the offline install guide. The mechanics are identical to the initial install.

Migrate the existing configuration file, keeping the offline registry settings:

ddlctl create config --from-domino --offline --agent-version $FLEETCOMMAND_AGENT_TAG \
  --agent-repository "your-registry-url.domain:port/custom-repository" \
  --image-registry "your-registry-url.domain:port" \
  --username "your-username" --password "your-password"

Confirm the generated configuration still contains the correct helm.image_registries block for your private registry, then apply it:

ddlctl create domino --config config-$FLEETCOMMAND_AGENT_TAG.{timestamp}.yaml --agent-version $FLEETCOMMAND_AGENT_TAG

Helm-managed Data Planes (legacy)

A Data Plane deployed with Helm, which is to say one from before Domino 6.1 with no domino custom resource, cannot use any of the upgrade methods on this page. Migrate it to the Platform Operator deployment model, which enables UI-initiated upgrades and simplifies ongoing maintenance. Until you do, the stopgaps are a standalone upgrade script, which Domino Support provides, or redeploying the Data Plane with the Platform Operator.

If you have Helm-based Data Planes, migrate them to the Platform Operator before you upgrade the control plane. Migration is a complete redeployment of the Data Plane, and all running executions must be stopped. Contact your Domino representative or Domino Support for help managing this upgrade.

Verify the upgrade

On the workstation with the Data Plane’s Kubernetes context, confirm the operator reconciled to the new version:
ddlctl get domino
VERSION and AGENT VERSION should match the target release, and READY should be True.
In the control plane’s admin portal, go to Manage resources > Data Planes and confirm the Data Plane is marked Healthy and its reported version matches the target.
Start a Workspace or run a job pinned to this Data Plane to confirm Workspace connectivity, ingress, and any hardware tiers still behave as expected.
For a Data Plane you upgraded manually, sanity-check HelmRelease status once more so nothing landed in a stalled state during the reconciliation:
ddlctl get helmrelease -A --status stalled=true

If the Data Plane is unhealthy after the upgrade, read the agent logs first:

kubectl logs -n domino-compute -l app.kubernetes.io/name=data-plane-agent

Logs reporting expired keys mean the Data Plane needs re-bootstrapping. Generate a fresh token from Manage Resources > Data Planes > Setup, then run ./bootstrap-data-plane.sh --force <token>. See Troubleshoot the registration.

Monitor a Data Plane

Check a Data Plane’s health, and see which Kubernetes nodes, user executions, and Workspaces are running in it, from the admin portal:

To see	Go to	Where the Data Plane appears
Data Plane health	Manage resources > Data Planes	The status marks a Data Plane Healthy when it is available for use.
Kubernetes nodes across all your clusters	Reports > Infrastructure	The column on the right indicates the Data Plane that each node is in.
All running user executions	Reports > Executions	The rightmost column indicates the Data Plane for the execution.
All durable Workspaces, running and stopped	Reports > Workspaces	The rightmost column indicates the Data Plane for the Workspace.

Data Plane health is a simple check that the Data Plane is contactable. If a Data Plane indicates Error, Disconnected, or Degraded, communication between the Data Plane services and the control plane may have a problem. Contact Domino for help troubleshooting. For more advanced monitoring and alerting, see remote Data Plane monitoring.

Some of the node detail views are not yet implemented for nodes in remote Data Planes.

Archive a Data Plane

Archiving a Data Plane effectively deletes it. An archived Data Plane cannot be un-archived.

Stop all executions in the Data Plane.
Delete all Workspaces in the Data Plane.
Archive all hardware tiers in the Data Plane.
In the admin portal, go to Manage resources > Data Planes.
Click the three-dot menu next to the Data Plane and select Archive.
Click Archive Data Plane.
Use Helm to delete the data-plane release in the Data Plane namespace on the Data Plane Kubernetes cluster.

Architecture & deployment

Domino installation

Identity & access

Infrastructure & compute

Platform configuration

Data administration

Operations, monitoring & cost

Security & compliance

Troubleshooting

Data Planes administration

Register a Data Plane

Verify the registration

Troubleshoot the registration

Enable a Data Plane for Workspaces and Datasets

Configure the hostname

Configure load balancing

Configure TLS

Configure Datasets

Edit a registered Data Plane

Disable file sync

Hardware Tiers for Data Planes

Upgrade a Data Plane

Before you start

Version compatibility

Check the deployment type

Choose an upgrade method

Upgrade from the UI

Upgrade manually with ddlctl

Handle release overrides

Air-gapped upgrades

Helm-managed Data Planes (legacy)

Verify the upgrade

Monitor a Data Plane

Archive a Data Plane

​Register a Data Plane

​Verify the registration

​Troubleshoot the registration

​Enable a Data Plane for Workspaces and Datasets

​Configure the hostname

​Configure load balancing

​Configure TLS

​Configure Datasets

​Edit a registered Data Plane

​Disable file sync

​Hardware Tiers for Data Planes

​Upgrade a Data Plane

​Before you start

​Version compatibility

​Check the deployment type

​Choose an upgrade method

​Upgrade from the UI

​Upgrade manually with ddlctl

​Handle release overrides

​Air-gapped upgrades

​Helm-managed Data Planes (legacy)

​Verify the upgrade

​Monitor a Data Plane

​Archive a Data Plane

​Related

Register a Data Plane

Verify the registration

Troubleshoot the registration

Enable a Data Plane for Workspaces and Datasets

Configure the hostname

Configure load balancing

Configure TLS

Configure Datasets

Edit a registered Data Plane

Disable file sync

Hardware Tiers for Data Planes

Upgrade a Data Plane

Before you start

Version compatibility

Check the deployment type

Choose an upgrade method

Upgrade from the UI

Upgrade manually with ddlctl

Handle release overrides

Air-gapped upgrades

Helm-managed Data Planes (legacy)

Verify the upgrade

Monitor a Data Plane

Archive a Data Plane

Related