# Design AI Benchmarking

> Design and validity review for studies that benchmark one or more AI systems against a human-expert panel as the reference. Covers the evaluation question and arm definition, decoupled multi-dimensional rubrics with anchors, planted calibration probes, reviewer-panel construction, inter-rater reliability targets, LLM-as-judge versus human-as-judge adjudication, construct-independence guards, and a structured rating-export schema. Use before data collection on an AI-vs-expert evaluation.

## Facts
- Page: https://tashan.sh/capability/skill-aperivue-design-ai-benchmarking
- tashan id: skill:Aperivue/design-ai-benchmarking
- Source: https://github.com/Aperivue/medsci-skills
- Type: skill
- Category: design
- tashan score: not scored (catalogued only — too little public evidence)
- Adoption: 9.0
- Upkeep: 94.0
- Freshness: 87.0
- Evidence coverage: 84% of the inputs this score can use
- Health: active
- Instruction depth: not yet graded
- License: MIT
- Official: no

## Install

```sh
cp -r design-ai-benchmarking ~/.claude/skills/
```

## Security audit
Not scanned. We audit npm-published capabilities; this one has no npm package we can resolve, or has not reached the queue. This is not a clean bill of health.

---
Measured 2026-09-13 by tashan (https://tashan.sh) from public evidence. Scorer s5.