LastMile x Helicone

Helicone is a popular open-source AI observability platform that helps you monitor AI applications, track experiments and run evaluations.

LastMile has partnered with Helicone to provide AutoEval's metrics natively within the Helicone platform.

Demo

Getting Started

tip

Check out the Helicone docs for detailed instructions:

• Getting Started
• Evaluation Scores
• Experiments

1. Go to https://helicone.ai and create an account

2. Go to Evaluators and Create New Evaluator

3. Select LastMile AutoEval tab, and select the desired metric as the Evaluator Type.

   Currently, only predefined evaluator metrics are supported. Fine-tuned metrics cannot be used at this time.

   

4. Select the columns of the data to evaluate.

   For example, you can measure the Faithfulness of the output to the context provided with respect to the input:

   

5. (optional) You can test the metric right there with the Test option

6. After the evaluator has been saved, you can open it and Create Online Evaluator.

   Configure the sampling rate and it will be run on sampled live traffic. For more information, check out the Helicone Evaluation docs.

   

7. You can also use the saved evaluators in Helicone Experiments. For more information, check out the Helicone Experiments docs.