A little experiment.
A lot of asterisks.

Same 30 requested model IDs as the initial KARLBY batch, without substitution. Catalog identity, prices, advertised reasoning options, and sampling defaults refreshed before the authorized recovery and ten-prompt expansion. Purposive size/family coverage, not a random or exhaustive model sample.

Run karlby-98-2026-10-19-first-pass: Exactly 30 catalog models × five separate prompts; one user message per call, no system message, ground truth, rubric, tools, conversation history, temperature, seed, sampling, or reasoning override. max_tokens=8192 (reasoning tokens included); all other inference parameters use each model/provider's defaults. Three requests maximum in flight, no retry of any durable intent; per-call wall deadline=180s. Routing: allow_fallbacks=false, data_collection=deny, require_parameters=true; no model fallback/substitution. Response identity accepts only the exact requested ID or its catalog canonical_slug frozen in the immutable plan; every other returned ID is an error, with no prefix/date inference. The actual returned_model is retained. max_price prompt/completion ceilings are catalog USD/token × 1,000,000 × 1.05 (5% allowance, upward numeric rounding); max_price.request=0 forbids per-request fees. Conservative reservations use UTF-8 prompt bytes + 1024 framing tokens and 2 × the full output cap at those ceilings to cover separately priced internal reasoning, rounded up to $0.000000001. This extra hold does not change request caps. Unbilled/interrupted calls retain their reservation. Exact per-model max_price JSON (prompt/completion USD per million tokens; request USD): {"anthropic/claude-haiku-5.5":{"completion":0.525,"prompt":0.105,"request":0},"anthropic/claude-opus-5.5":{"completion":21.0,"prompt":4.2,"request":0},"anthropic/claude-sonnet-5.5":{"completion":10.5,"prompt":2.1,"request":0},"cohere/command-a-plus":{"completion":1.575,"prompt":0.315,"request":0},"deepseek/deepseek-v4-pro-0813":{"completion":2.079,"prompt":0.693,"request":0},"deepseek/deepseek-v4.1-flash-1017":{"completion":0.63,"prompt":0.1575,"request":0},"google/gemini-3.5-flash-lite":{"completion":2.625,"prompt":0.315,"request":0},"google/gemini-3.8-flash":{"completion":4.33125,"prompt":0.86625,"request":0},"google/gemini-4-argon":{"completion":10.5,"prompt":2.1,"request":0},"ibm-granite/granite-4.2-8b":{"completion":0.2625,"prompt":0.063,"request":0},"inclusionai/ling-3.1-flash":{"completion":0.23625,"prompt":0.07875,"request":0},"liquid/lfm-2.5-24b-a2b":{"completion":0.735,"prompt":0.0525,"request":0},"meta/muse-glimmer-30b":{"completion":1.26,"prompt":0.315,"request":0},"meta/muse-spark-open":{"completion":4.4625,"prompt":1.3125,"request":0},"meta/muse-wonder":{"completion":21.0,"prompt":4.2,"request":0},"mistralai/ministral-3b-2512":{"completion":0.105,"prompt":0.105,"request":0},"mistralai/mistral-large-4-0":{"completion":2.1945,"prompt":0.714,"request":0},"nvidia/nemotron-3.5-lightning":{"completion":0.21,"prompt":0.0735,"request":0},"nvidia/nemotron-3.5-super-vl":{"completion":0.42,"prompt":0.105,"request":0},"openai/gpt-6-luna":{"completion":0.525,"prompt":0.105,"request":0},"openai/gpt-6.1-sol":{"completion":10.5,"prompt":2.1,"request":0},"qwen/qwen3.5-9b":{"completion":0.1575,"prompt":0.105,"request":0},"qwen/qwen3.8-27b":{"completion":2.31,"prompt":0.252,"request":0},"qwen/qwen3.8-flash":{"completion":0.4935,"prompt":0.1575,"request":0},"qwen/qwen3.8-max-prime":{"completion":12.6,"prompt":4.2,"request":0},"stepfun/step-5-preview":{"completion":2.835,"prompt":1.05,"request":0},"upstage/solar-mini4":{"completion":0.21,"prompt":0.0525,"request":0},"x-ai/grok-4.7":{"completion":6.3,"prompt":2.1,"request":0},"xiaomi/mimo-v2.6-flash":{"completion":0.294,"prompt":0.147,"request":0},"z-ai/glm-5.3-prime":{"completion":9.24,"prompt":2.94,"request":0}}. Exact requested ID to frozen catalog canonical_slug mapping: {"anthropic/claude-haiku-5.5":"anthropic/claude-haiku-5.5-20261007","anthropic/claude-opus-5.5":"anthropic/claude-opus-5.5-20260921","anthropic/claude-sonnet-5.5":"anthropic/claude-sonnet-5.5-20260928","cohere/command-a-plus":"cohere/command-a-plus-05-2026","deepseek/deepseek-v4-pro-0813":"deepseek/deepseek-v4-pro-20260813","deepseek/deepseek-v4.1-flash-1017":"deepseek/deepseek-v4.1-flash-20261017","google/gemini-3.5-flash-lite":"google/gemini-3.5-flash-lite-20260721","google/gemini-3.8-flash":"google/gemini-3.8-flash-20260902","google/gemini-4-argon":"google/gemini-4-argon-20261013","ibm-granite/granite-4.2-8b":"ibm-granite/granite-4.2-8b-20260831","inclusionai/ling-3.1-flash":"inclusionai/ling-3.1-flash-20261002","liquid/lfm-2.5-24b-a2b":"liquid/lfm-2.5-24b-a2b-20261012","meta/muse-glimmer-30b":"meta/muse-glimmer-30b-20260810","meta/muse-spark-open":"meta/muse-spark-open-20261015","meta/muse-wonder":"meta/muse-wonder-20261012","mistralai/ministral-3b-2512":"mistralai/ministral-3b-2512","mistralai/mistral-large-4-0":"mistralai/mistral-large-4-0-20261006","nvidia/nemotron-3.5-lightning":"nvidia/nemotron-3.5-lightning-20260807","nvidia/nemotron-3.5-super-vl":"nvidia/nemotron-3.5-super-vl-20261015","openai/gpt-6-luna":"openai/gpt-6-luna-20260922","openai/gpt-6.1-sol":"openai/gpt-6.1-sol-20260929","qwen/qwen3.5-9b":"qwen/qwen3.5-9b-20260310","qwen/qwen3.8-27b":"qwen/qwen3.8-27b-20260814","qwen/qwen3.8-flash":"qwen/qwen3.8-flash-20260826","qwen/qwen3.8-max-prime":"qwen/qwen3.8-max-prime-20260923","stepfun/step-5-preview":"stepfun/step-5-preview-20261008","upstage/solar-mini4":"upstage/solar-mini4-20260922","x-ai/grok-4.7":"x-ai/grok-4.7-20260916","xiaomi/mimo-v2.6-flash":"xiaomi/mimo-v2.6-flash-20260921","z-ai/glm-5.3-prime":"z-ai/glm-5.3-prime-20260921"} Run karlby-98-2026-10-19-expanded-recovery-1: Each selected cell sends one independent user message. No tools/plugins/web search, system message, history, answer/rubric, or explicit reasoning/sampling override is provided. Advertised catalog defaults do not establish effective response settings. Exact sanitized requests and actual returned identity are retained per attempt. At most three calls globally and one per model; no intent is resent. No output or reasoning token budget is requested; provider/context limits still apply. Admission uses observed known spend only, not bounded exposure. Unknown bills stay unknown; healthy in-flight calls can overshoot the threshold or provider allowance. The actual per-call wall deadline is 600 seconds. Run karlby-98-2026-10-19-expanded-continuation-12: Each selected cell sends one independent user message. No tools/plugins/web search, system message, history, answer/rubric, or explicit reasoning/sampling override is provided. Advertised catalog defaults do not establish effective response settings. Exact sanitized requests and actual returned identity are retained per attempt. At most 12 calls globally and one per model; no intent is resent. No output or reasoning token budget is requested; provider/context limits still apply. Admission uses observed known spend only, not bounded exposure. Unknown bills stay unknown; healthy in-flight calls can overshoot the threshold or provider allowance. The actual per-call wall deadline is 600 seconds. Run karlby-98-2026-10-19-expanded-tail-3-per-model: Each selected cell sends one independent user message. No tools/plugins/web search, system message, history, answer/rubric, or explicit reasoning/sampling override is provided. Advertised catalog defaults do not establish effective response settings. Exact sanitized requests and actual returned identity are retained per attempt. At most 12 calls globally and 3 per model; no intent is resent. No output or reasoning token budget is requested; provider/context limits still apply. Admission uses observed known spend only, not bounded exposure. Unknown bills stay unknown; healthy in-flight calls can overshoot the threshold or provider allowance. The actual per-call wall deadline is 600 seconds.

AI-assisted contextual reviews are bound to stable attempt IDs, with literal completed-answer evidence. For each model/prompt retain the earliest completed attempt by runs order then numeric attempt index; if none completed, show the latest attempted outcome, otherwise unattempted. Never replace a success to improve a score. YES counts one affirmative-agreement point on kind:fit prompts; NO/uncertain count zero. Only models with every fit answer completed and reviewed receive an official competition rank; missing or unreviewed answers are not scored wrong. Confidence prompts are separate.

One sample per model/prompt, no statistical-significance claim. Changes may reflect ordinary sampling noise; model/provider defaults and reasoning allocation differ. The owner's fit observation and seat arrangement are not independently validated. A NO under the 100% confidence instruction may obey the wording rather than deny physical fit. Output and wall-time caps, provider errors, privacy/price filters, conservative budget holds, and unresolved intents can leave cells unfinished. Unattempted calls are not zero-cost successes. Budget protection relies on the API honoring its advertised token/price caps. Credentials are redacted before persistence; any secret-reflecting response is withheld as an API error. Offline accounting policy correction: 1 normally finished response(s) were initially marked accounting_cap_exceeded solely because provider prompt overhead exceeded a reservation heuristic, despite known reported bills within their reservations and output/reasoning within requested caps. They were corrected offline without API replay or changes to prompts, request settings, output/price caps, budget, or the immutable plan. Original evidence and integrity receipts are retained privately. Interrupted calls remain in the full planned denominator with their unknown-cost reservation holds; they were not retried. Offline accounting policy correction: 1 response(s) with reported output/reasoning tokens above the requested cap were initially marked as financial accounting stops despite known reported bills within their reservations. Those financial flags were corrected offline without API replay. The responses remain noncompleted api_error records with provider_token_cap_report_exceeded, preserving their final text, usage, finish reason, and cost; they cannot be assessed or ranked. Original evidence and versioned integrity receipts are retained privately. No prompts, request settings, output/price caps, budget, other attempts, or immutable plan were changed. These provider failures and interrupted calls remain in the full planned denominator; unknown-cost reservation holds remain intact. Owner-reported fit, not independently measured clearance, hatch closure, seat arrangement, or safety. Single draws and provider defaults cannot establish general accuracy, confidence calibration, or causal wording effects. Recovery retains earlier successes and retries failures under different conditions, so this is not a uniform one-shot head-to-head. Known cost is provider metadata, not an invoice; unknown billing is not zero. Client deadlines do not prove provider computation/billing stopped. Privacy/price filters, provider defaults/limits, timeouts, and admission stops may leave coverage incomplete. Token-report inconsistencies are warnings, not grounds to discard valid final text. Length finishes remain truncated. Credential-reflecting answers are withheld. Aggregate known cost and unknown-attempt counts include EVERY actual attempt in all imported runs, including failed and superseded outcomes, not only displayed successes. budget_usd discloses the latest run's admission authorization, not a sum of historical thresholds or a bound on all invoices. Original capped/shorter-deadline successes remain selected while failed cells can recover under uncapped-request/longer-deadline conditions; this is not a uniform one-shot comparison.