How tashan measures

tashan exists because the AI-capability ecosystem has massive supply and almost no honest signal. So the first rule is:

we score capabilities on public evidence — and we show every component. Here is exactly what we count, how tashan score is computed, and what we deliberately refuse to do.

No user tracking. tashan installs nothing, instruments no one, and reads no private data. Every number here is derived from public sources — the MCP registry, npm, and public GitHub configs — and every signal links to the source it came from.

Coverage

We ingest the official MCP registry as the spine of coverage: thousands of capabilities with their canonical name, description, source repository, package, and active/deprecated status.

On top of that we mine public GitHub configs (mcp.json, claude_desktop_config.json, .cursor) for real-world usage the registry can't see.

The goal is simple: track everything, then rank it on evidence — and tracking runs well ahead of measuring, which is stated as a number further down under how much of this is measured rather than left for you to discover.

What we measure

The tashan score

The single Index number is a transparent composite — and every input is shown on the capability's page, never hidden:

raw = mean(Upkeep, Freshness) × gate(Adoption) × coverage
tashan score = calibrate(raw)
gate = 0.30 + 0.70 × Adoption⁄100  ·  coverage discounts a score built on thin evidence
calibrate = piecewise-linear, raw 50 → 68; strictly order-preserving

Why it is called the tashan score, not a trust score. We measure whether people use a capability and whether it is kept alive. That is all the number is.

The risk scan is a separate reading, it does not feed this score, and it does not read the capability’s code or test it for prompt injection.

Calling that "trust" claimed something the arithmetic does not support, and this page had to walk it back a paragraph later. The number is our measurement, on public evidence, and it is named accordingly.

Upkeep and freshness set the ceiling; adoption decides most of what a capability actually scores. That balance is deliberate and it was wrong before: upkeep and recency both saturate near 100 for any repo that was pushed recently, so when the gate floor sat at 0.6 a well-kept capability nobody used still kept 60% of the score, and a plugin with 30 stars outranked packages with thousands of weekly downloads.

The floor is now 0.30, which means real use — not a recent commit — is what separates the top of the Index from the middle. Adoption itself blends npm downloads (log-scaled) and config reach. Capabilities we can't yet measure quality on are shown unrated — plainly marked, never invented.

When retention and evals land, they'll be separate, labelled scores, never quietly folded into one number.

Missing evidence is not a bad score. Upkeep is scored only across the axes we can actually read for that capability — maintainer count, release cadence, freshness — and rescaled over those, so a capability that simply isn't on npm is no longer marked down for metadata that does not exist.

It used to be: anything without npm was capped at 35/100 however well kept it was, which ranked a 51,000-star plugin pushed yesterday below a package with 422 weekly downloads.

Instead, thin evidence discounts tashan score in proportion to how much of the picture we have, and "we found no adoption signal" is treated no better than a measured-but-low one — absence of evidence must never outscore evidence of absence.

As we fill in contributor and release data for non-npm repositories, that discount shrinks on its own, because the coverage becomes real.

Why there is a calibration step. The raw formula multiplies three factors that are each at most 1, so it decides the order well and lands in a useless range.

A capability with no adoption signal has gate = 0.30, which caps it at about 27⁄100 however well kept it is — and 38% of what we measure has neither a download count nor a star count.

The result was half the index inside a 13-point band, 98% below 50, and "well kept but unproven" published as a number that reads as bad.

The calibration is a fixed, strictly order-preserving stretch of that raw number onto the published scale: it cannot move any capability past another, it only decides how far apart they are printed.

It is deliberately not a percentile — a percentile would re-baseline every night as the index grows, so a badge minted today would quietly mean something different next week.

The middle of the index sits near 39 rather than 50, because a median of 50 would claim we think half of these are proven when for most of them we have no usage evidence either way.

When we change the ruler, we say so. Every stored score records which version of the scorer produced it. That matters because the scorer was rewritten four times in one week: one capability reads 43, 43, 43, 42, 40 across those days and every step down was us, not the capability. Comparing across that line would report our own recalibration as something declining, so we do not do it — a trend is only ever computed within a single scorer version, and a version change starts the clock again. The current version is s5, measured 2026-09-13.

And now we say when we won't. Honest labelling was only half the problem. A version boundary restarts every trend, so a scorer rewritten four times in a week produces a history that is never more than two days long — accurately labelled and worth nothing.

So the ruler is now held still on purpose: s5 is frozen through 2026-11-01, and no change to the scoring ships before that date. A correctness fix — a severity that was wrong, a signal that was misread — can still ship inside the window, because leaving a known error in place to protect a graph would be the worse trade; it lands as a new version, with the reason recorded, and the old series is kept rather than restated.

The commitment is enforced in the build, not in this paragraph: the suite refuses a scoring change inside the window unless it is explicitly broken open (tests/test_scorer_version.py).

A capability the registry removes stays visible to you. The MCP registry marks servers deleted, and its moderation policy names spam, malware and illegal content as the usual reasons.

We keep that row rather than dropping it: it is never scored, never ranked and never recommended, but it is still there so doctor can tell you if it is in your config. Deleting the record would have turned the most serious thing we can tell you into silence.

How much of this is measured

We track 93,380 capabilities and have measured far fewer. The obvious number to publish is the ratio between them, and we did — until it fell from 47% to 41% in two days without anything getting worse.

Discovery had grown the denominator faster than measurement grew the numerator, which it will do every time discovery works. A metric that improves when you find less is not a coverage metric.

So the denominator is demand. What costs you is not a gap in the tail; it is an absence on the thing you looked up. Ranked by the adoption evidence we hold, on the top 1,000:

tashan score 98%  ·  security scan 100%  ·  expertise grade 30%  ·  task mapping 52%
all four, on the same capability: 25%  ·  across all 93,380 tracked: 1%

Two of those are ours to fix by running more of what already works, and they will move nightly. Two — expertise and task mapping — are graded reads rather than fetched facts, and they are the honest weak spot in this product today: on the capabilities people ask about most, they are mostly absent.

That is the roadmap, stated as a number rather than as a plan, and it is re-measured on every run into coverage.json so you can watch it move or catch us if it doesn't.

Absent still means unmeasured, never bad. Nothing here is scored down for a gap on our side — a capability we have not graded shows as unrated and is ranked on what we did read.

If we got it wrong

Every graded capability carries a “This grade is wrong” link on its own page, pre-filled with the id and the verdict so you do not have to explain which page you mean. It opens a public issue, because a correction argued in the open is worth more than one settled privately — and because the same evidence that changes our mind should be visible to anyone who read the original.

What we commit to. A reply within five working days. If the grade was read off the wrong document, or off documentation that has since improved, it is withdrawn or re-graded on the next pipeline run and the page says so.

If we think the grade was right, we say why, in the issue, where you can argue back. We do not remove a grade because someone asks. A rating nobody can contest is an opinion with better typography; a rating that folds on request is not a measurement at all.

This is not hypothetical. 87 published judgments were withdrawn in one pass after we found they had been read off a README the capability shares with other packages, or one that never names it — including Anthropic’s own filesystem server, graded thin for a monorepo index it shares with three siblings.

Those pages now state the fact instead: shares its documentation with 3 other capabilities.

What a low score does NOT mean

It does not mean safe. It usually means nothing is happening — and nothing happening includes nothing being observed.

We tested that against our own series. Capabilities scoring 70–84 turned out 224× more likely to have a security-relevant change than those under 40.

Read naively, that says high scores are dangerous. It says nothing of the kind.

A dormant package cannot add an install script, because it never ships. Only 0.35% of capabilities under 40 published a release in the window, against 24% of those scoring 55–69.

Among the ones that actually shipped, the effect largely collapses.

So the score separates alive from dormant by roughly eightyfold, which is what it claims to measure. It is not a safety verdict.

Whether it predicts risk once activity is held constant, we cannot yet say: ten events across 573 shippers is too few to read.

We would rather publish that than round a null into a headline. pipeline/validate_score.py re-runs as the series grows.

What we deliberately don't do

The security audit

Separate from the score, and deliberately so: well maintained and nothing known is wrong with it are different claims, and a popular, actively maintained package can still ship a vulnerability.

Folding one into the other would hide exactly the case you most need to see. Four layers, all from public evidence:

What the audit does not do, just as plainly: no source-code review, no execution or sandboxing, no testing of outputs for prompt injection, and no coverage of capabilities with no npm package to resolve. A clean audit means nothing known is wrong.

Every other number here is a proxy: downloads, publish cadence, stars, config appearances. Each can exist without one person finding the thing useful.

A settled payment cannot. Where a capability publishes an x402 price, a calling agent signs a USDC transfer and the receipt is on Base permanently.

We read those receipts and join them to the catalog — who gets paid.

It is not an input to the tashan score, for the same reason the security audit is not: "someone pays for this" and "this is well made" are different claims.

Almost every MCP server is free by design. Being unpaid is the normal case, not a defect.

What this is not (yet) — stated plainly

The roadmap

Each is a separate, defensible measurement. Stacked, they become something no directory can copy: an evidence base for what actually works.

‹ Back to the Index