Publish a Streamlit LLM App in Domino

This guide walks you through building and deploying a chatbot powered by large language models (LLMs) using the Streamlit framework in Domino. The app combines a couple of models hosted on Hugging Face with a model served via a Domino endpoint through a single, interactive UI.

What you’ll do

Set up API access and compute environment prerequisites
Use Domino’s example code to build a Streamlit chatbot with selectable model backends
Integrate Hugging Face and Domino endpoints
Configure app startup and publish it in Domino
Share the app with others in your organization

Prerequisites

Before you begin, make sure you have the following in place:

Hugging Face API key
Get a Hugging Face API key to access models hosted on Hugging Face:
1. Create your account on Hugging Face.
2. Get your Access token from Hugging Face.
3. In Domino, go to Account Settings > Environment Variables and add a user-level variable:
  1. Name: HUGGING_FACE_API_TOKEN
  2. Value: your key
Domino compute environment
If you’re using a Domino Standard Environment (DSE), these are already included. Your environment must include:
1. Jupyter or JupyterLab
2. The jupyter-server-proxy package
3. Streamlit and necessary Python libraries
Domino endpoint
To demonstrate multi-model access, this guide uses:
1. Hugging Face-hosted models, such as Llama 3.2 3B and Mistral 7B.
2. A model deployed as a Domino-hosted prediction endpoint. It is assumed you already have a Domino endpoint configured and available.

Step 1: Set up the environment

To run and publish your Streamlit chatbot, first configure a Domino environment with the required packages. Streamlit apps are launched through a streamlit run and are hosted at port 8501 by default. You can configure this to use a specific port later.

Go to Environments > Create Environment.
Enter a name and description.
Select a base image, such as the Domino Standard Environment.

In the Dockerfile, add the following if not already present:

USER root
RUN pip install --no-cache-dir \
    streamlit \
    jupyter-server-proxy \
    requests \
    pandas
USER ubuntu
RUN pip install transformers torch streamlit

Click Build and wait for the environment to finish building.

You now have a compute environment with Streamlit and proxy support, ready to run your chatbot app.

Step 2: Create the Project and build the App UI

Next, you’ll create a new Domino project and begin building your chatbot app using Streamlit. You’ll start with a basic layout: a title, a sidebar with model selection, and a password field for your Hugging Face API key. You’ll develop the app interactively inside a Jupyter or JupyterLab workspace and view it live in a browser tab using a URL generated by Domino.

Create the Domino Project

First, create the Domino Project you’ll use for your app:

Go to your Domino home page, or your Projects page, and choose Develop > Projects > Create Project.
Name your project something like streamlit-llm-chatbot and click Create.
In the project sidebar, go to Settings > Compute environment and select the environment you just created.

Next, create the app title and sidebar for the app:

Launch a new Jupyter or JupyterLab workspace.
In the /mnt directory, create a new file named chatbot.py.
Add the starter code found here in Domino’s example streamlit LLM app repo.
Update the following values in your file:
1. DOMINO_ENDPOINT_URL: set to your Domino-hosted model endpoint
2. DOMINO_MODEL_ACCESS_TOKEN: your model’s access token

More information about Domino endpoints and tokens can be found in the Select Domino endpoint authorization mode documentation.

Get your App URL

To preview your app, construct the URL based on your Domino run context.

Replace //your-domino-url/ with your actual Domino domain and run:

echo -e "import os\nprint('https://your-domino-url/{}/{}/notebookSession/{}/proxy/8501/'.format(os.environ['DOMINO_PROJECT_OWNER'], os.environ['DOMINO_PROJECT_NAME'], os.environ['DOMINO_RUN_ID']))" | python3

Port 8501 is Streamlit’s default in a Workspace. Published Apps must serve traffic on port 8888; Step 4 configures that port.

Run the app in your workspace:
streamlit run chatbot.py
Open the app in a browser tab using the generated URL.

You now have a live app preview with a sidebar and model selector ready for user interaction and backend logic.

Step 3: Add model logic and user interaction

Now that the UI is in place, you’ll add logic to handle chat input, store conversation history, and route model calls based on user selection. The app supports both a Hugging Face-hosted model and a Domino-hosted endpoint. This step uses Streamlit session state to maintain chat history across user inputs.

Store chat messages in session state

This logic is already included in the chatbot.py code. It initializes and displays the conversation history.

Define model query functions

This app already includes logic to:

Call Hugging Face-hosted models via InferenceClient.
Fallback to direct HTTP calls if needed.
Call your Domino-hosted model endpoint.

All function definitions are in the file, including:

chat_with_hf(…)
chat_with_api_direct(…)
query_domino_endpoint(…)

Capture user input and generate a response

User prompts are captured using st.chat_input, and responses are routed depending on the selected model. The logic makes sure that:

Messages are appended to the session state
Responses are streamed with a spinner
Fallbacks are handled gracefully

The chatbot.py file has the full implementation details.

Sync all changes

Once your file is complete and tested interactively in your workspace, you’ll need to sync all changes.

Save all files, including chatbot.py and any updates to tokens or environment values.
If you’re ready to move on from development, stop your workspace. This makes sure Domino syncs all file changes back to your project.
Confirm that both files - chatbot.py and app.sh - exist in your project root.

Your app is now fully interactive, capturing input, maintaining history, and calling different LLMs based on user selection.

Step 4: Configure the app file and prepare for publishing

To publish your Streamlit app in Domino, you need a startup script that tells Domino how to launch the app and configure it to use a specific host and port.

Problem: Streamlit defaults to port 8501, but Apps published in Domino must serve traffic on port 8888.
Solution: Override the default settings using a config.toml file inside your app.sh startup script.

Create the configuration file

In the root of your project, create a file named app.sh with the following content:

#!/bin/bash

mkdir -p ~/.streamlit

cat << EOF > ~/.streamlit/config.toml
[browser]
gatherUsageStats = true

[server]
port = 8888
enableCORS = false
enableXsrfProtection = false
address = "0.0.0.0"
EOF

streamlit run chatbot.py

Verify: Make sure the file is executable by running chmod +x app.sh in the terminal from inside your workspace, if needed.

Why this works

The config.toml file tells Streamlit to use the required port 8888 and host 0.0.0.0.
Domino runs app.sh automatically when launching your app with no manual intervention needed.

You’re now ready to publish the chatbot as a live Domino app.

Step 5: Publish and deploy the app

Domino apps run inside containers based on your project’s environment and are launched by executing the app.sh script you configured in the previous step. Creating an App no longer deploys it: preview the App in a workspace, publish a version, then deploy that version.

Create the App

On your project’s Deploy > Apps & Agents page, select Create, then App in Workspace. Any workspace you already have open works too. See Create an App.

Preview it

In the workspace, open the App Preview tab and set the App file, Compute Environment, and Hardware Tier. Commit your code first, because preview runs the latest committed code. See Develop and test an App.

Publish a version

Select Publish App, then Publish as a new App. This records an immutable version and doesn’t start it.

Deploy the version

On the App details page, select the version and select Deploy, then set access under Access and sharing. See Publish and deploy App versions and Share an App.

Give the App an informative title and description, for example LLM Chatbot with Hugging Face and Domino Endpoints. Set the permissions to Anyone in Domino so other users on your Domino deployment can view it, and leave Make globally discoverable selected so they can find it from the Deploy > Apps view. You can change both settings later. Once the app status changes to Running, select View App to open your fully interactive chatbot in a new browser tab, with model selection, live input, and responses from either the Hugging Face-hosted or Domino-hosted model. Once your chatbot is running, you can share it with colleagues or verify that another Domino user can open it. Access depends on the permissions you set during publishing. Anyone in Domino allows any user with a Domino account to view the App through its URL.

Copy the App link

From the Deployments > App screen, click Copy App Link.
Share the link with others who have access to your Domino instance.

Test access

Open the App URL while signed in as a Domino user who isn’t a Project collaborator to confirm broad access. If you’re prompted for a Hugging Face token, enter it in the sidebar to test API access as a new user.

Browse or discover more apps

From the top nav, go to Deploy > Apps to explore other published tools within your organization.

You’ve now successfully published and shared an LLM-powered Streamlit chatbot with interactive inputs and backend model integration.

Next steps

Apps in Domino gives an overview of how apps work within the Domino ecosystem.
Create an App covers the two entry points for a new App, and why creating one no longer deploys it.
Learn more about how apps run in Domino and what identity and permissions are used.

Core Concepts

Features

Publish a Streamlit LLM App in Domino

What you’ll do

Prerequisites

Step 1: Set up the environment

Step 2: Create the Project and build the App UI

Create the Domino Project

Create the app title and sidebar

Get your App URL

Step 3: Add model logic and user interaction

Store chat messages in session state

Define model query functions

Capture user input and generate a response

Sync all changes

Step 4: Configure the app file and prepare for publishing

Create the configuration file

Why this works

Step 5: Publish and deploy the app

Copy the App link

Test access

Browse or discover more apps

Next steps

​What you’ll do

​Prerequisites

​Step 1: Set up the environment

​Step 2: Create the Project and build the App UI

​Create the Domino Project

​Create the app title and sidebar

​Get your App URL

​Step 3: Add model logic and user interaction

​Store chat messages in session state

​Define model query functions

​Capture user input and generate a response

​Sync all changes

​Step 4: Configure the app file and prepare for publishing

​Create the configuration file

​Why this works

​Step 5: Publish and deploy the app

​Step 6: Share the app

​Copy the App link

​Test access

​Browse or discover more apps

​Next steps

What you’ll do

Prerequisites

Step 1: Set up the environment

Step 2: Create the Project and build the App UI

Create the Domino Project

Create the app title and sidebar

Get your App URL

Step 3: Add model logic and user interaction

Store chat messages in session state

Define model query functions

Capture user input and generate a response

Sync all changes

Step 4: Configure the app file and prepare for publishing

Create the configuration file

Why this works

Step 5: Publish and deploy the app

Step 6: Share the app

Copy the App link

Test access

Browse or discover more apps

Next steps