Skip to main content

al-BERTa-LC-8k

A 400M parameter state-of-the-art, encoder based model for calcuating the factual consistency of the generated answer against the provided context.

Model Description

Performance

SQuAD 2.096.7%
Latency< 2ms for 1000 tokens

Example Use

import json
import oså
from typing import Any, Generator
import pandas as pd
from lastmile_auto_eval import (
EvaluationMetric,
EvaluationResult,
evaluate as auto_evaluate,
stream_evaluate,
)
from IPython.display import display

result: EvaluationResult = auto_evaluate(
dataframe=evaluation_data,
metrics=[
EvaluationMetric.P_FAITHFUL,
],
lastmile_api_token=os.getenv("LASTMILE_API_TOKEN"),
)