Skip to main content

​
Overview

Domino File System (DFS) Projects store your files in the DFS, Domino’s traditional storage for Project assets. Domino versions your files automatically with a Git server it runs in the background. Every time you sync changes, Domino records a new Project version. Choose a DFS Project if you or your collaborators aren’t comfortable with Git, or if your organization’s policies require it; otherwise Domino recommends Git-based Projects.

​
How it works

A DFS Project organizes assets into two sections in the navigation pane:
  • Code lists your Project files, files imported from other Projects, and files in attached Git repositories. Unlike Git-based Projects, DFS Projects keep artifacts here too.
  • Data lists the data sources used in the Project: Domino Datasets, external data volumes, and Dataset scratch spaces.

​
The working directory

When you start a run, Domino loads the Project’s files into the working directory, /mnt by default. Domino sets the DOMINO_WORKING_DIR environment variable for all runs; use it instead of hardcoding paths, because importing another Project changes the path to /mnt/<project-owner-username>/<project-name>. Dataset volumes for your home Project mount at /domino/datasets/local/<name>; see Dataset paths in DFS Projects. Alongside your own files, the working directory holds files that Domino generates or reads: 
DOMINO_WORKING_DIR/
├── ipynb_checkpoints   # Contains the auto-saved Jupyter states.
├── results             # Contains your generated results.
    └── stdout.txt      # Tail of the console output from your Run.
├── requirements.txt    # Optional: Specifies Python package dependencies.
├── .dominoresults      # Determines which files are rendered as results.
├── .dominoignore       # Contains file patterns for Domino to ignore.
├── .dominokeep         # Prevents directory removal of empty directory.
├── dominostats.json    # Values shown in the Jobs dashboard.
├── email.html          # Controls the formatting of notification emails.
├── .noLock             # Solves "too many open files" messages.
├── app.sh              # Contains app-launching code for Domino Apps.
├── domino.log          # Contains CLI logs for local CLI projects only.
└── .domino.vmoptions   # Contains proxy settings for local CLI projects only.
For details on individual files, see .dominoresults, requirements.txt, .noLock, email.html and dominostats.json, and domino.vmoptions.

​
Upload files

You can drag and drop files up to 550 MB into the Domino web application (on-premises and VPC administrators can change this limit). For larger files, use one of these routes:
  • Domino CLI: install the CLI, then in the local folder holding your files run domino restore to associate the folder with a Project (this downloads the Project’s current files), and domino sync to upload your local files.
  • Download from a URL: if the file is reachable over the web, save a shell script in your Code tab that fetches it with wget, and run the script once in Domino.
With large data (tens of GB), runs spend longer in the Preparing stage while files copy to the executor. For large data Projects, Domino natively integrates with Hadoop, Spark, and other big data platforms; contact support@dominodatalab.com for more information.

​
Compare file revisions

To see what changed between versions of a file, open the file from the Code section and click Compare Revisions. Set the starting version in Base and the version to compare in Target. On the Project’s Code page you can select among all Project revisions; when viewing an individual file, the menu lists only revisions that changed that file.

​
Revert Projects and files

You can return a whole Project or a single file to an earlier version. Reverting creates a new version at the top of the history, so the version you reverted from stays accessible; this makes it cheap to recover from failed experiments.
  • Revert a Project: In the Code section, open the Commit menu, which lists every version with the current one first. Click an older version to view it, then click Revert Project.
  • Revert a file: In the Code section, open the file, pick an older version from the Commit menu, and click Revert File. Other files in the Project keep their current versions.

​
Exclude files from sync

To keep files out of the domino sync operation, list them in the .dominoignore file that Domino creates at the root of every new Project. This applies whether you sync from your local machine (CLI or R package), at the end of a run, or during a Workspace session. Add a file name or a folder name (which excludes its contents); use * as a wildcard, with all paths relative to the Project root. A .git/ directory is always ignored, even if not listed. Two other patterns help when .dominoignore isn’t enough:
  • Isolated results branches: configure the Project to save results to a separate branch. The results stay accessible from the application and CLI, but domino sync doesn’t download them and future runs don’t use them as input unless you say so. See Files, artifacts, and results.
  • Project importing: put your code in one Project and import it into a second Project where the large-output runs happen. You can sync the code Project without downloading the outputs, and even export the code Project as a package so it installs automatically at runtime.

​
Export files as a Python or R package

If a Project’s files are organized as an installable package, you can export them as one: any Project that imports it gets the package automatically installed at runtime. Enable exports for the Project, then select the language under Code Package.
See Packaging Python Projects. In summary, a Python package requires:
  • A setup.py file containing a setup() function imported from setuptools.
  • A folder containing your Python modules and packages, usually named after the overall package.
  • Domino recommends also including a README file.

​
Import Git repositories

You can attach external Git repositories to a DFS Project. Repositories added to the Project are cloned into /repos in every run and Workspace you start, so your code can use their contents just like your Domino files. Domino supports HTTPS and SSH connections for both public and private repositories. For private repositories, SSH, or writing commits back, first add Git credentials to your Domino account.
If you want to work primarily from a Git provider of your choice, use a Git-based Project instead of importing repositories into a DFS Project.

​
Add a repository to a Project

  1. Open the Project, click Code, then open the Git Repositories tab and click Add a New Repository.
  2. Enter a directory name (the directory in /repos the repository clones into; defaults to the repository name) and the HTTPS or SSH URI.
  3. Select what Domino checks out when it clones the repository into a run or Workspace: Use default branch checks out your Git service’s default branch (typically master), or specify a branch name, tag name, commit ID, or custom Git ref.
  4. Click Add Repository.
Your Domino working directory /mnt and /repos are siblings at the filesystem root, so scripts stored as Domino files reach repository contents through absolute paths like /repos/<repo-name>/<file>.

​
Pin a repository to a commit ID

Specifying a commit ID checks out that fixed point in the repository’s history on every run, in a detached HEAD state; unlike a branch, it never evolves. Git tracks changes you make and you can commit them as usual, but a push from a detached HEAD fails because Git needs a branch to push to. Domino also can’t push automatically to a repository pinned this way. To publish changes, create a local branch from the detached HEAD in the Workspace command line, check it out, and run git push -u origin.

​
Commit back to a repository

In a Workspace session, repositories added to the Project are listed in the Session Overview under Git repos, with your changes itemized file by file. To commit them, select the checkbox next to the repository and click Full Sync, then enter a commit message. The message attaches to the Git commits and, if Domino files also changed, to a new revision of the Project. Commits push to the default branch you chose when adding the repository. Stopping a Workspace with uncommitted repository changes prompts you the same way: select the repositories, enter a message, and click Stop and Commit. If your changes conflict with the remote default branch, Domino creates a new branch from the local state and pushes that branch to the remote. Resolve the conflicts outside Domino or on the Workspace command line; the next session checks out the remote default branch again, not the pushed branch. You can also work with the repositories directly: Jupyter and RStudio Workspaces both have command lines (Jupyter: New > Terminal from the Files tab; RStudio: Tools > Shell…) where standard Git commands work in /repos. To audit what a run did to its repositories, expand the Repositories panel at the bottom of the run’s Details tab; Domino records the checked-out commit at the start and end of the run.

​
Troubleshoot repository access

  • Credentials are required for your repository: the Git credential in Domino may have the wrong domain. Check that the credential’s domain field matches your repository URL exactly (github.com, bitbucket.com, or your internal Git host).
  • The repository provided requires credentials but none were found: confirm your SSH key or PAT is in Git Credentials in your account settings, and if your organization requires SSO, authorize the key for SSO.
  • Invalid username or password / Authentication failed: usually the same SSO authorization issue; authorize the credential for your organization.