Aarush Sah - Head of Evals @ Groq

I lead evaluation systems at Groq. I'm interested in how we can better understand AI capabilities through systematic evaluation. In my spare time, I build benchmarks that test the capabilities of LLMs.

Before Groq, I was at Nous Research developing synthetic data pipelines for training language models.

AIME Problem #1 - 2025 AIME I

Problem: Find the sum of all integer bases $b>9$ for which $17_b$ is a divisor of $97_b$.

Projects

Eris

July 22, 2024

An evaluation framework using debate simulations to assess AI models' reasoning and communication skills.

AI Debate Evaluation

Topic: The potential of quantum computing in breaking current encryption standards

Affirmative Position

Negative Position

Judge's Analysis

Final Verdict

✓ Negative position wins the debate

Set-Eval

March 22, 2024

A multimodal benchmark for testing vision capabilities and reasoning in AI models.

Set-Eval

Blog

The Internship Game

and why you shouldn't play it • September 03, 2024• 6 min read

There's a curious disconnect in how we talk about tech internships. The conventional wisdom—polish your resume, practice interview questions, network aggressively—isn't wrong, exactly. But it misses something essential.