Overview
Persistent volumes (PVs) let Domino give each Job and Workspace fast local storage for project files, instead of reading and writing directly to the Domino blob store on every operation. As an administrator, you manage how these volumes are provisioned, reused, backed up, and cleaned up, so that your cluster balances storage cost against the performance benefit of reusing volumes.
How it works
When not in use, Domino project files are stored and versioned in the Domino blob store. When a Domino execution is started from a project, the project files are copied to a Kubernetes persistent volume (PV) that is attached to the compute node and mounted in the Run.
You can run the following command to see all current PVs, sorted by last-used:
Storage workflow for Jobs
When a user starts a new Job, Domino assigns a new execution pod to the cluster. This pod has an associated Persistent Volume Claim (PVC) which defines what type of storage it requires from Kubernetes.
If an idle PV exists that matches the PVC, Kubernetes mounts that PV on the node it assigns to host the pod, and the Job starts. If an appropriate idle PV does not exist, Kubernetes creates a new PV according to the storage class.
When the user completes their Job, the PV data is written to the Domino File System, and the PV unmounts and sits idle until it is either reused for the user’s next Job or until garbage collection. By reusing PVs, users who are actively working in a project do not have to repeatedly copy data from the blob store to a PV.
A Job only matches with a fresh PV or one previously used by that project. PVs are not reused between projects.
Garbage collection
This feature applies to Idle PVs used for jobs and scheduled jobs, not durable workspaces.
-
Limit the total number of idle PVs to
32.
-
Terminate any idle PV that has not been used in
7 days.
For more information, please reach out to Domino Support.
Storage workflow for Workspaces
Workspace volumes are handled differently than volumes for Jobs. Workspaces are potentially long-lived development environments that users stop and resume repeatedly without writing data back to the Domino File System each time. As a result, the PV for the Workspace is a similarly long-lived resource that stores the user’s working data.
These workspace PVs are durably associated with the durable Workspace they are initially created for. Each time that Workspace is stopped, the PV is detached and preserved so that it’s available the next time the user starts the Workspace. When the Workspace starts again, it reattaches its PV and the user sees all their working data saved during the last session.
The contents of the project files in the workspace PV are written back to the Domino File System only when a user chooses to initiate a sync. A durable workspace PV is deleted only if the user deletes the associated Workspace.
Durable Workspace volume backups on AWS
Since the data in durable workspace volumes is not automatically written back to the Domino File System, users can lose work if the volume is lost or deleted. When Domino is running on AWS, it safeguards against this by backing up the EBS volume that backs up the workspace PV with EBS snapshotting to S3.
EBS snapshots can be expensive, especially if you have a lot of users. To manage snapshots for your organization, please reach out to Domino Support.
If you accidentally deleted or lost a durable workspace volume that contains data you want to recover, contact Domino Support for assistance in restoring from the snapshot.
Enable automatic Workspace deletion
Domino can automatically delete unused workspaces to make more storage available.
When a Workspace has been stopped for 30 days, Domino marks it for deletion and notifies the workspace user.
The user is notified several times during the 14-day grace period. Domino notifies the user once more when the Workspace is deleted.
To enable this feature or extend the periods, please reach out to Domino Support.
Enable volume provisioning recommendations
When a user starts a Workspace or a Job, Domino can provide volume provisioning recommendations based on project size and previous usage to reduce storage costs and maximize performance. This feature is enabled by default.
The volume size can be configured on a per-project basis on the Project Settings page. This includes setting the default volume size for new projects, as well as the minimum and maximum volume size allowed.
For more information, please reach out to Domino Support.
