Eval Harness Documentation
Welcome to the docs for the LM Evaluation Harness!
Table of Contents
- To learn about the public interface of the library, as well as how to evaluate via the command line or as integrated into an external library, see the Interface.
- To learn how to add a new library, API, or model type to the library, as well as a quick explainer on the types of ways to evaluate an LM, see the Model Guide.
- For an extended description of how to extend the library to new model classes served over an API, see the API Guide.
- For a crash course on adding new tasks to the library, see our New Task Guide.
- To learn more about pushing the limits of task configuration that the Eval Harness supports, see the Task Configuration Guide.