agent-evaluation
Testing and benchmarking LLM agents including behavioral testing,
Best for User mentions or implies: a…Works with GitHubLow risk
⌘source
- author
@sickn33- language
- Python
★34kstars
5.6kforks
Apr 18, 2026
✦overview.md
Use Cases
- →User mentions or implies: agent testing
- →User mentions or implies: agent evaluation
- →User mentions or implies: benchmark agents
- →User mentions or implies: agent reliability
- →User mentions or implies: test agent
Not ideal for
- !Use this skill only when the task clearly matches the scope described above.
- !Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- !Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.
name
agent-evaluation
description
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks
risk:safe
source:vibeship-spawner-skills (Apache 2.0)
date_added:"2026-02-27T00:00:00.000Z"
Agent Evaluation
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks
Capabilities
- agent-testing
- benchmark-design
- capability-assessment
- reliability-metrics
- regression-testing
Prerequisites
- Knowledge: Testing methodologies, Statistical analysis basics, LLM behavior patterns
- Skills_recommended: autonomous-agents, multi-agent-orchestration
- Required skills: testing-fundamentals, llm-fundamentals
Scope
- Does_not_cover: Model training evaluation (loss, perplexity), Fairness and bias testing, User experience testing
- Boundaries: Focus is agent capability and reliability, Covers functional and behavioral testing
Ecosystem
Primary_tools
- AgentBench - Multi-environment benchmark for LLM agents (ICLR 2024)
- τ-bench (Tau-bench) - Sierra's real-world agent benchmark
- ToolEmu - Risky behavior detection for agent tool use
- Langsmith - LLM tracing and evaluation platform
Alternatives
- Braintrust - When: Need production monitoring integration LLM evaluation and monitoring
$install
1-click copynpx skills add sickn33/antigravity-awesome-skills --skill agent-evaluationSafety assessment
★
Clarity score
How clear and easy to understand the SKILL.md instructions are, rated from 1 to 5.
1/ 5
poorThe SKILL.md content is hard to understand and quite ambiguous.
◎
Actionability score
How directly an agent can act on the SKILL.md instructions, rated from 1 to 5.
1/ 5
not actionableThe SKILL.md is hard to act on; an agent would not know what to do.