Example Datasets
Synthetic datasets representing typical AI applications
We have generated a few datasets that are representative of typical AI applications so you have test data to try out the platform.
All datasets are available in the lastmile-docs repository.
The datasets were generated using gen_autoeval_datasets.ipynb notebook. You can use it to generate other kinds of synthetic datasets.
Dataset Index
SuperCard Dining Concierge
An AI dining recommendation bot that takes in a user query in natural language, and returns a list of restaurants (in natural language). Analogous to a RAG application with the restaurant DB being the source of truth.
The dataset contains example interactions, as well as a Relevance label -- 1 if the interaction is related to dining, and 0 if it isn't.
MediaCo Data Warehouse Agent
A multi-agent AI application, where each agent is able to query a data warehouse, and together answer questions spanning all parts of a media company's various properties (book publisher, music label, TV network, etc.)
The dataset contains example interactions, as well as a Faithfulness label -- 1 if the interaction invoked the correct agent and returned a relevant response, and 0 if it didn't.
StrikeAir Customer Support
A conversational AI system that answers customers' questions on behalf of an airline ("Strike Air").
The dataset contains example interactions, as well as a Clarity & Politeness label -- 1 if the interaction was accurate and cordial, and 0 if it wasn't.
WMB Financial Wealth Advisor
An AI retrieval system for financial advisors, helping them more efficiently provide financial advice to their clients.
The dataset contains example interactions, as well as a Correctness label -- 1 if the interaction was accurate and factual given some ground truth data, and 0 if it wasn't.