Evaluate Dataset

Evaluate Dataset

POST 
/api/2/auto_eval/evaluation/evaluate_dataset

Evaluate a metric on a dataset, returning scores for each example. Specify metric.id or metric.name to identify the metric.

Request

application/json

Body

metric

object

required

id string

name string

description string

deploymentStatus ModelDeploymentStatus (string)

Possible values: [MODEL_DEPLOYMENT_STATUS_UNSPECIFIED, MODEL_DEPLOYMENT_STATUS_PENDING, MODEL_DEPLOYMENT_STATUS_ONLINE, MODEL_DEPLOYMENT_STATUS_OFFLINE, MODEL_DEPLOYMENT_STATUS_PAUSED]

datasetId stringrequired

Responses

200

Successful operation

application/json

Schema

Schema

metric

object

required

id string

name string

description string

deploymentStatus ModelDeploymentStatus (string)

Possible values: [MODEL_DEPLOYMENT_STATUS_UNSPECIFIED, MODEL_DEPLOYMENT_STATUS_PENDING, MODEL_DEPLOYMENT_STATUS_ONLINE, MODEL_DEPLOYMENT_STATUS_OFFLINE, MODEL_DEPLOYMENT_STATUS_PAUSED]

scores number[]required

Example (from schema)

{
  "metric": {
    "id": "string",
    "name": "string",
    "description": "string",
    "deploymentStatus": "MODEL_DEPLOYMENT_STATUS_UNSPECIFIED"
  },
  "scores": [
    0
  ]
}

Loading...