numpy, include the following.
On-demand Spark
Manage Spark dependencies
Package Python libraries, JARs, and pinned versions in both Spark cluster and execution Compute Environments through Docker Instructions.
Listen to this pageabout 1 min
In a shared Spark cluster, it can be challenging for teams to manage their dependencies (for example, Python packages or JARs). Installing every dependency that a Spark application may need before it runs and dealing with version conflicts can be complex and time-consuming.
Domino allows you to easily package and manage dependencies as part of your Spark-enabled compute environments. This approach creates the flexibility to manage dependencies for individual projects or workloads without having to deal with the complexity of a shared cluster.
To add a new dependency, add the appropriate statements in the Docker Instructions section of the relevant Spark and execution compute environments.
For example to add
Last modified on September 3, 2026
Was this page helpful?
⌘I