eval-harness
Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles
Best for AI development teamsWorks with GitHubLow risk
⌘source
- author
@affaan-m- language
- JavaScript
★159kstars
25kforks
Apr 17, 2026
✦overview.md
Key Features
- ·Formal evaluation framework for Claude Code sessions
- ·Implements eval-driven development (EDD) principles
- ·Defines pass/fail criteria for task completion
- ·Measures agent reliability with pass@k metrics
- ·Creates regression test suites for prompt or agent changes
- ·Benchmarks agent performance across model versions
Use Cases
- →Setting up eval-driven development for AI-assisted workflows
- →Creating regression test suites after prompt or agent modifications
- →Benchmarking Claude's performance across different model versions
Best for
- ✓AI development teams
- ✓Prompt engineering workflows
name
eval-harness
description
Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles
origin:ECC
tools:Read, Write, Edit, Bash, Grep, Glob
Eval Harness Skill
A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.
When to Activate
- Setting up eval-driven development (EDD) for AI-assisted workflows
- Defining pass/fail criteria for Claude Code task completion
- Measuring agent reliability with pass@k metrics
- Creating regression test suites for prompt or agent changes
- Benchmarking agent performance across model versions
Philosophy
Eval-Driven Development treats evals as the "unit tests of AI development":
- Define expected behavior BEFORE implementation
- Run evals continuously during development
- Track regressions with each change
- Use pass@k metrics for reliability measurement
Eval Types
Capability Evals
Test if Claude can do something it couldn't before:
[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] Criterion 3
Expected Output: Description of expected result
Regression Evals
Ensure changes don't break existing functionality:
[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
$install
1-click copynpx skills add affaan-m/everything-claude-code --skill eval-harnessSafety assessment
★
Clarity score
How clear and easy to understand the SKILL.md instructions are, rated from 1 to 5.
4/ 5
very goodClear and well structured, with only minor parts that might need a second read.
◎
Actionability score
How directly an agent can act on the SKILL.md instructions, rated from 1 to 5.
3/ 5
mediumPartially actionable with several concrete steps, but still missing important details.