Get LLM Judge labeling Job Status

POST 
/api/2/auto_eval/pseudo_label_job/get_status

Get the status of an LLM Judge labeling job.

Request

application/json

Body

jobId

object

required

value stringrequired

Responses

200

Successful operation

application/json

Schema

Schema

status stringrequired

pseudoLabelJobResult

object

required

Subset of columns to be used in pseudo-labeling. Expected columns: input, output, ground_truth For example, a summarization task might not need an input column. TODO: Should this be repeated EvaluationMetricParameter enum?

name string

Optional name for the job.

description string

Optional description for the job.

datasetId

object

required

value stringrequired

fewShotDatasetId

object

value stringrequired

activeLabeledDatasetId

object

value stringrequired

selectedColumns string[]required

Subset of columns to be used in pseudo-labeling. Expected columns: input, output, ground_truth For example, a summarization task might not need an input column. TODO: Should this be repeated EvaluationMetricParameter enum?

chatCompletionConfig

object

required

The list of messages in the conversation so far.

vendor stringrequired

model stringrequired

The ID of the model to use for the completion.

messages

object[]

required

The list of messages in the conversation so far.

Array [

role stringrequired

Role can be 'system', 'user', or 'assistant'.

content stringrequired

The content of the message.

]

maxTokens integerrequired

The maximum number of tokens to generate.

Possible values: >= -2147483648 and <= 2147483647

temperature numberrequired

The temperature to use for the completion.

topP numberrequired

The top_p value to use for the completion.

promptTemplate

object

required

id stringrequired

template stringrequired

The template string that defines the prompt

baseEvaluationMetric stringrequired

TODO: @Ankush flesh out default prompt templates or "Base Metric" representation of prompt lmai.proto.model_fine_tuning.v1.templates.

skipActiveLabeling booleanrequired

If true, skip active labeling.

Example (from schema)

{
  "status": "string",
  "pseudoLabelJobResult": {
    "name": "string",
    "description": "string",
    "datasetId": {
      "value": "string"
    },
    "fewShotDatasetId": {
      "value": "string"
    },
    "activeLabeledDatasetId": {
      "value": "string"
    },
    "selectedColumns": [
      "string"
    ],
    "chatCompletionConfig": {
      "vendor": "string",
      "model": "string",
      "messages": [
        {
          "role": "string",
          "content": "string"
        }
      ],
      "maxTokens": 0,
      "temperature": 0,
      "topP": 0
    },
    "promptTemplate": {
      "id": "string",
      "template": "string"
    },
    "baseEvaluationMetric": "string",
    "skipActiveLabeling": true
  }
}

Loading...