> ## Documentation Index
> Fetch the complete documentation index at: https://docs.domino.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Run batch jobs

> Enable batch in LLM Gateway 2.0 and submit OpenAI and Anthropic batch jobs through gateway aliases, with vendor batch discounts where the vendor supports them.

LLM Gateway 2.0 supports both the OpenAI and Anthropic batch APIs, so the OpenAI and Anthropic software development kits (SDKs) work unchanged. Batch is off by default because it opens a file upload and spend surface to callers.

## Enable batch

Enable **Batch** on the Settings page. It takes effect without a restart.

## Batch endpoints

* OpenAI: `/v1/files`, `/v1/batches`, and `/v1/batches/{id}`, including cancel and long-poll with `?wait=true`.
* Anthropic: `/anthropic/v1/messages/batches` and its retrieve, results, cancel, and delete routes.

The `model` on every line is a gateway alias, and every line in a batch must use the same alias.

## How the gateway runs a batch

Batches to OpenAI, Azure OpenAI, or Anthropic aliases go to the vendor's own batch API and get the vendor's batch discount. Batches to every other provider, including Domino-hosted models, Bedrock, and Vertex, run line by line through the gateway with no discount.

Input and output guardrails apply to every line in both modes. The gateway excludes a blocked line from the batch and reports it in the error file by its `custom_id`.

## Submit a batch

```python theme={null}
from openai import OpenAI

client = OpenAI(base_url="https://<your-domino-host>/apps/<gateway-app-id>/v1", api_key=token)
f = client.files.create(file=open("prompts.jsonl", "rb"), purpose="batch")
batch = client.batches.create(input_file_id=f.id, endpoint="/v1/chat/completions",
                              completion_window="24h")

b = client.batches.retrieve(batch.id)
if b.output_file_id:
    print(client.files.content(b.output_file_id).text)  # successful lines
if b.error_file_id:
    print(client.files.content(b.error_file_id).text)   # failed lines, by custom_id
```

Read both files. A batch can finish as `completed` with no output file if every line failed.

With the Anthropic SDK, authenticate with `auth_token=`, not `api_key=`, or the gateway returns `401`.

## Limits

* Streaming and fallback chains don't apply inside a batch.
* One bring your own token (BYOT) key applies to the whole batch.
* Bedrock and Vertex batches don't get the AWS or Google discount.
* The gateway keeps input files for 7 days and output files for 30 days by default.

## Related

* [LLM Gateway 2.0](/cloud/platform-capabilities/features/llms/llm-gateway#make-your-first-call): get a token and point the SDK at the gateway.
* [Configure guardrails and fallback routing](/cloud/admin/platform-configuration/llm-gateway/guardrails-and-fallback#guardrails): the rules applied to each batch line.


## Related topics

- [Configure guardrails and fallback routing](/cloud/admin/platform-configuration/llm-gateway/guardrails-and-fallback.md)
- [Configure batch scoring Job](/cloud/platform-capabilities/features/models/batch-scoring.md)
- [Preventing Jupyter timeout errors when running notebooks as a batch/scheduled job](/knowledge-base/preventing-jupyter-timeout-errors-when-running-notebooks-as-a-batch-scheduled-job.md)
- [Work with Jobs](/cloud/platform-capabilities/core-concepts/jobs/index.md)