Skip to main content
GET
/
api
/
gen-ai
/
beta
/
endpoints
/
{endpointId}
/
versions
/
{versionNumber}
/
vllm-latency-metrics
Get vLLM latency metrics for a model endpoint version
curl --request GET \
  --url https://api.example.com/api/gen-ai/beta/endpoints/{endpointId}/versions/{versionNumber}/vllm-latency-metrics \
  --header 'X-Domino-Api-Key: <api-key>'
{
  "averageEndToEndLatency": 1.234,
  "averageTimePerOutputToken": 0.056,
  "averageTimeToFirstToken": 0.245,
  "endToEndLatency": {
    "observations": [
      {
        "timestamp": "2024-01-15T10:00:00Z",
        "value": 0.123
      }
    ]
  },
  "timePerOutputToken": {
    "observations": [
      {
        "timestamp": "2024-01-15T10:00:00Z",
        "value": 0.123
      }
    ]
  },
  "timeToFirstToken": {
    "observations": [
      {
        "timestamp": "2024-01-15T10:00:00Z",
        "value": 0.123
      }
    ]
  }
}

Authorizations

​
X-Domino-Api-Key
string
header
required

Path Parameters

​
endpointId
string
required

The ID of the model endpoint to get vLLM latency metrics for

Example:

"62313ce67a0af0281c01a6a5"

​
versionNumber
integer
required

The version number of the model endpoint to get vLLM latency metrics for

Example:

1

Query Parameters

​
startTime
string<date-time>

Start time for the metrics query in ISO 8601 format. Defaults to 24 hours ago.

Example:

"2024-01-15T10:00:00Z"

​
endTime
string<date-time>

End time for the metrics query in ISO 8601 format. Defaults to current time.

Example:

"2024-01-16T10:00:00Z"

​
aggregationWindow
enum<string>

Optional aggregation window for metrics data points. When provided, uses the specified window (5min, hourly, or daily) for step size. When omitted, uses adaptive step sizing based on the time range to optimize data point count.

Available options:
hourly,
daily
Example:

"hourly"

​
maxHistoricalExecutions
integer
default:3

Maximum number of historical executions to include in metrics aggregation. Defaults to 3 if not specified.

Required range: 1 <= x <= 10
Example:

3

Response

Success

​
averageEndToEndLatency
number<double>
required

Average end-to-end latency across the entire queried time period in seconds

Example:

1.234

​
averageTimePerOutputToken
number<double>
required

Average time per output token across the entire queried time period in seconds

Example:

0.056

​
averageTimeToFirstToken
number<double>
required

Average time to first token across the entire queried time period in seconds

Example:

0.245

​
endToEndLatency
object
required

Smoothed average end-to-end latency in seconds

​
timePerOutputToken
object
required

Smoothed average time per output token in seconds

​
timeToFirstToken
object
required

Smoothed average time to first token in seconds