pip install at startup.
Before you start
- The App owner must be a Domino admin. The gateway reads organization memberships and role assignments with the owner’s credentials. With a non-admin owner, organization grants never match and admin detection fails.
- Use a dedicated Project, and restrict its Dataset to the admin owner. The Dataset holds the gateway database, the key that encrypts provider credentials, and any backups. Anyone with Dataset access can read them.
- Use a hardware tier with at least 2 CPU cores and 8 GB of RAM.
- Use a Compute Environment based on Ubuntu 22.04 or later. The binary needs glibc 2.35.
Deploy the App
1
Get and verify the release
Get the
llm-gateway-X.Y.Z-linux-x86_64.tar.gz release from your Domino field representative. Verify it against its .sha256 file with sha256sum -c.2
Unpack the release in the Project
Upload the tarball to the Project’s Files, then unpack it from a Workspace terminal:This creates
./llm-gateway-X.Y.Z/. Don’t rename the folder: app.sh finds it by name.3
Configure the App
In the Project’s App tab, configure the App:
4
Publish the App
Click Publish. The first line in the App log reads
[app.sh] launching binary: ./llm-gateway-X.Y.Z/llm-gateway.5
Complete first boot
Open the App URL as the owner. On first boot, the gateway creates its database and encryption key in the Project Dataset and adds the Domino Platform provider.
6
Add providers and aliases
Add providers and aliases from the UI, then grant access to them.
Upgrade
Unpack the new release next to the old folder, then restart the App.app.sh starts the newest version it finds. After you confirm the upgrade, delete the old folder. The database, encryption key, and scheduled Jobs live in the Dataset, so they carry over.
Troubleshoot
Operate the gateway
The gateway Settings page holds 3 opt-in features that keep the gateway database small and recoverable: backups, log purge, and warehouse mirroring. Each runs as a Domino scheduled Job that calls back into the App.Back up the database
Schedule daily, weekly, monthly, or quarterly backups, or click Run now. Backups go to<dataset>/llm_gateway/backups/ by default and are standard SQLite files. The gateway prunes older backups according to the retention setting.
To restore, stop the App, replace llm_gateway.db in the Dataset with a backup, and restart. Provider credentials in the backup stay encrypted, and the .gateway_key file next to the database decrypts them.
Purge old logs
Delete usage and audit logs older than a set horizon. 6 months is a good default. The gateway won’t purge inside your longest active budget period, and can refuse to purge without a backup from the last 7 days. Use Dry run to see row counts first.Mirror logs to a warehouse
Mirror usage logs, audit logs, or both to a Domino Data Source: Snowflake, Postgres, Redshift, BigQuery, MySQL, ClickHouse, or MariaDB.1
Pick the Data Source and tables
In Settings, pick the Data Source and table names, then click Test connection.
2
Create the table
Click Create table.
3
Turn on mirroring
Set each log type to
mirror. Rows arrive 5 to 15 seconds after the request.4
Copy history
Click Backfill now to copy existing logs. It’s safe to rerun.
Monitor health
GET /health returns {"status": "ok"}. GET /health?verbose=1 adds database, backup age, and mirror status. When the mirror queue backs up or the last backup is older than twice its interval, the gateway writes a warning to the audit log.
Environment variables
Set these under App Configuration > Environment. A standard deployment needs none of them.Related
- LLM Gateway 2.0: what the gateway routes, its endpoints, and how callers make their first request.
- Control access, spend, and usage: grant callers access to aliases, set budgets, and track cost.
- Configure guardrails and fallback routing: screen requests and route around failures.