language:

• en

Open LLM Leaderboard Results

This repository contains the outcomes of your submitted models that have been evaluated through the Open LLM Leaderboard. Our goal is to shed light on the cutting-edge Large Language Models (LLMs) and chatbots, enabling you to make well-informed decisions regarding your chosen application.

Evaluation Methodology

The evaluation process involves running your models against several benchmarks from the Eleuther AI Harness, a unified framework for measuring the effectiveness of generative language models. Below is a brief overview of each benchmark:

1. AI2 Reasoning Challenge (ARC) - Grade-School Science Questions (25-shot)
2. HellaSwag - Commonsense Inference (10-shot)
3. MMLU - Massive Multi-Task Language Understanding, knowledge on 57 domains (5-shot)
4. TruthfulQA - Propensity to Produce Falsehoods (0-shot)
5. Winogrande - Adversarial Winograd Schema Challenge (5-shot)
6. GSM8k - Grade School Math Word Problems Solving Complex Mathematical Reasoning (5-shot)

Together, these benchmarks provide an assessment of a model's capabilities in terms of knowledge, reasoning, and some math, in various scenarios.

Exploring Model Details

For further insights into the inputs and outputs of specific models, locate the "📄" emoji associated with the desired model in the leaderboard. Clicking on this icon will direct you to the respective GitHub page containing detailed information about the model's behavior during the evaluation process.