Honc Out #37 – LLM-as-Judge: How Models Score Evals скачать в хорошем качестве

Honc Out #37 – LLM-as-Judge: How Models Score Evals

In this episode, we’re talking about LLM-as-judge — how language models can be used to score evals, where this approach works well, and where it gets tricky. We’ll start with how we handle evals and scoring in code, then shift to the role of LLMs as evaluators: how they compare to rule-based scoring, what kinds of bias or variance they introduce, and how to make their judgments more consistent and reliable. Topics we’ll cover: How evals and scoring work in code The benefits and pitfalls of LLM-as-judge Techniques to make model judgments more stable As always, join us live to ask questions in chat.