Issue
An execution fails with the following error in the logs:Root Cause
Each execution runs inside a compute instance with a finite amount of disk space. Total disk usage combines project files, external Git repository contents, the compute environment image, and any data generated during the run. When combined usage exceeds the available disk size, the execution fails. Crash or dump files left behind by integrated development environments (IDEs) or libraries can also consume large amounts of disk space without being immediately visible.Resolution
For users
- Switch to a different hardware tier. This moves the execution to a different compute instance with available disk space. It’s the quickest fix if you don’t have admin privileges.
- Reduce project data. Remove unnecessary files from the project. Git repository contents count toward disk usage.
- Move large files to Datasets. Move large project files to a Domino Dataset rather than keeping them in the project directory.
- Check for crash or dump files. Jupyter and other IDEs can crash and leave large files in the workspace that aren’t immediately visible.
- Increase the disk size (v4.4 and later). In your project, go to Settings → Hardware & Environment → Workspace and Jobs Volume Size and increase the allocated disk size. Changes apply to new workspaces only and don’t affect currently running sessions.
For admins
- Place the affected executor into Maintenance Mode to take it out of service.
- In the Domino Dispatcher UI, locate the executor and verify that no executions are currently running on it. Then terminate the executor to release the storage. Because executors are ephemeral, terminating one is a safe and fast resolution.
Note: Garbage collection may also recover disk space on the executor without requiring termination, but termination is the fastest option. If this issue recurs, consider increasing the default disk size using the Central Config setting: com.cerebro.domino.computegrid.kubernetes.volume.volumesSizeInGB