Skip to main content
GET
/
api
/
gen-ai
/
beta
/
endpoints
/
{endpointId}
/
versions
/
{versionNumber}
/
vllm-token-metrics
Get vLLM token metrics for a model endpoint version
curl --request GET \
  --url https://api.example.com/api/gen-ai/beta/endpoints/{endpointId}/versions/{versionNumber}/vllm-token-metrics \
  --header 'X-Domino-Api-Key: <api-key>'
{
  "generationTokensTotal": {
    "observations": [
      {
        "timestamp": "2024-01-15T10:00:00Z",
        "value": 0.123
      }
    ]
  },
  "promptTokensTotal": {
    "observations": [
      {
        "timestamp": "2024-01-15T10:00:00Z",
        "value": 0.123
      }
    ]
  },
  "totalGenerationTokens": 8500,
  "totalPromptTokens": 15000
}

Authorizations

​
X-Domino-Api-Key
string
header
required

Path Parameters

​
endpointId
string
required

The ID of the model endpoint to get vLLM token metrics for

Example:

"62313ce67a0af0281c01a6a5"

​
versionNumber
integer
required

The version number of the model endpoint to get vLLM token metrics for

Example:

1

Query Parameters

​
startTime
string<date-time>

Start time for the metrics query in ISO 8601 format. Defaults to 24 hours ago.

Example:

"2024-01-15T10:00:00Z"

​
endTime
string<date-time>

End time for the metrics query in ISO 8601 format. Defaults to current time.

Example:

"2024-01-16T10:00:00Z"

​
aggregationWindow
enum<string>

Optional aggregation window for metrics data points. When provided, uses the specified window (5min, hourly, or daily) for step size. When omitted, uses adaptive step sizing based on the time range to optimize data point count.

Available options:
hourly,
daily
Example:

"hourly"

​
maxHistoricalExecutions
integer
default:3

Maximum number of historical executions to include in metrics aggregation. Defaults to 3 if not specified.

Required range: 1 <= x <= 10
Example:

3

Response

Success

​
generationTokensTotal
object
required

Number of generation tokens produced per interval

​
promptTokensTotal
object
required

Number of prompt tokens processed per interval

​
totalGenerationTokens
number
required

Total number of generation tokens produced across the entire queried time period

Example:

8500

​
totalPromptTokens
number
required

Total number of prompt tokens processed across the entire queried time period

Example:

15000