Examples

Detailed, runnable examples are available as Jupyter Notebooks in the examples folder:

• quickstart.ipynb:

  Using GAICo's Experiment module to provide a simple, quickstart workflow.

• example-1.ipynb:

  Evaluating multiple models (LLMs, Google, and Custom) using a single metric.

• example-2.ipynb:

  Evaluating a single model on multiple metrics.

• DeepSeek-example.ipynb

  The aim for this notebook was to aid with evaluating DeepSeek R1 for AI4Society's Point of View (POV).

Advanced Examples

The advanced-examples directory contains advances notebooks showcasing more complex use cases and metrics. These examples are intended for users who are already familiar with the basics of GAICo. Please refer to the README.md file in that directory for details. A quick description:

• llm_faq-example.ipynb

  Comparison of various LLM responses (Phi, Mixtral, etc.) on FAQ dataset from USC.

• threshold-example.ipynb

  Exploration of default and custom thresholding techniques for LLM responses.

• viz-example.ipynb

  Hands-on visualizations for LLM results.