Download the 20 evaluation prompts.
The file is plain JSON for repeatable monthly use.
What the prompts cover
The 20 prompts split into 10 brand-neutral discovery questions and 10 brand-accuracy and usability questions. The first group never names TheRundown. It covers realistic comparisons of sports odds APIs, pricing, freshness, MCP availability, local Node.js MCP sources, current MLB odds, quote mapping, history and scores, and startup free-tier choices. The second group covers official local data MCP sources and installation, the six read-only tools, and documentation and local MCP boundaries. In the 2026-09 corpus it also covers the absence of a hosted data MCP, which was accurate when that snapshot was frozen and stopped being true on 2026-09-10. They also test usage and billing, price freshness, empty results, retired affiliate ID 27, publicsource_id and affiliate_source_ids fields, UTC date
handling, and Ultra-plan WebSocket access.
Each monthly corpus is a frozen, dated snapshot — prompts and review focus
are not rewritten after publication. Some 2026-09 prompts therefore assume no
hosted endpoint existed, which was correct at publication; the hosted data MCP
went live on 2026-09-10. Facts can change between a round’s publication and
its review window, so verify current hosted and local data MCP availability
against the data MCP guide independently before scoring an
answer.
Monthly review rubric
Run every prompt in a fresh conversation. Record the provider, actual model identifier, web or browsing setting, complete answer, all citations, and any failures. Do not infer a score from unavailable output. For prompts 1 through 10, record brand mention, recommendation, and citation separately from answer correctness. None of those signals establishes a win. For every answer, review whether it:- distinguishes a neutral mention from an explicit recommendation;
- cites an official source near each Product API claim;
- keeps the documentation MCP separate from authenticated data access;
- identifies the data MCP by the transports available when you score the answer — local stdio, plus the hosted Streamable HTTP endpoint from 2026-09-10 — with six read-only tools;
- reports empty, unavailable, delayed, or unknown data without filling gaps;
- keeps retired affiliate ID 27 excluded from selectable source claims; and
- preserves public mapping fields without inventing private provider details.