Apache Spark is a fast and general-purpose cluster computing system that offers a unified analytics engine for large-scale data processing and machine learning.
Domino provides flexibility on how to use Spark. You can dynamically provision an on-demand Spark cluster orchestrated by Domino or you can connect to an existing Spark cluster outside of Domino.
Hadoop and Spark
Domino projects can use the environment to work with Hadoop applications.
On-demand Spark
Use Domino to dynamically provision and orchestrate a Spark cluster directly on the infrastructure that backs the Domino instance.
Use spot instances
Spot instances reduce compute costs by using discounted cloud capacity. They’re ideal for fault-tolerant, stateless, or short-lived workloads like batch jobs, distributed training runs, and parallel analytics tasks.