Pipeline health
Per-model rates are broken out by panel version and cover each panel's full lifetime. Panels are separate instruments — they run different model slots under different extraction rules — so pooling them would average two regimes into one misleading number. Today's progress shows how much of the day's work remains. The collect + rate pipeline runs unattended; this page exists so anyone can check whether it is, in fact, running.
Today
panel_v5_free · UTCLatest run per model
Freshness & status| Model | Panel | Latest run | Status |
|---|---|---|---|
| DeepSeek V3.1 (SambaNova) | panel_v4_freeretired | 2026-08-18 01:53 UTC | completed |
| Gemini 3.5 Flash Lite (Google) | panel_v5_free | 2026-11-14 01:39 UTC | running |
| GLM 4.5 Air (OpenRouter) | panel_v4_freeretired | 2026-08-18 01:53 UTC | completed |
| GPT-OSS 120B (Groq) | panel_v5_free | 2026-11-14 01:39 UTC | running |
| Llama 3.3 70B (Groq) | panel_v4_freeretired | 2026-08-18 01:53 UTC | completed |
| Llama 3.3 70B (HuggingFace) | panel_v5_free | 2026-11-14 01:39 UTC | failed |
| Llama 4 Scout 17B (Groq) | panel_v4_freeretired | 2026-08-18 01:53 UTC | completed |
| Magistral Small (Mistral) | panel_v5_free | 2026-11-14 01:39 UTC | running |
| Mistral Medium Latest (Mistral) | panel_v5_free | 2026-11-14 01:39 UTC | running |
| Mistral Small Latest (Mistral) | panel_v5_free | 2026-11-14 01:39 UTC | running |
| Nemotron 3 Super 120B (OpenRouter) | panel_v5_free | 2026-11-14 01:39 UTC | failed |
| Qwen 3 32B (Groq) | panel_v4_freeretired | 2026-08-18 01:53 UTC | completed |
panel_v5_free current
7 slots · 2026-08-18 → 2026-11-14 · 3440/13041 analysable| Model | Runs | Rows | Data | Analysable | Partial | Incoherent | Avg latency | Last data | Verdict |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash Lite (Google) | 89 | 1869 | 947 (50.7%) | 947 (50.7%) | 0 (0%) | 5 (0.3%) | 2049 ms | 2026-11-14 | ok |
| GPT-OSS 120B (Groq) | 88 | 1848 | 797 (43.1%) | 797 (43.1%) | 0 (0%) | 2 (0.1%) | 12397 ms | 2026-11-14 | ok |
| Llama 3.3 70B (HuggingFace) | 89 | 1869 | 86 (4.6%) | 86 (4.6%) | 0 (0%) | 619 (33.1%) | 5398 ms | 2026-10-07 | dark 38d billing |
| Magistral Small (Mistral) | 89 | 1869 | 340 (18.2%) | 340 (18.2%) | 0 (0%) | 440 (23.5%) | 2400 ms | 2026-09-03 | dark 72d |
| Mistral Medium Latest (Mistral) | 89 | 1869 | 332 (17.8%) | 332 (17.8%) | 0 (0%) | 428 (22.9%) | 2570 ms | 2026-09-04 | dark 71d |
| Mistral Small Latest (Mistral) | 88 | 1848 | 332 (18%) | 331 (17.9%) | 0 (0%) | 430 (23.3%) | 2286 ms | 2026-09-03 | dark 72d |
| Nemotron 3 Super 120B (OpenRouter) | 89 | 1869 | 608 (32.5%) | 607 (32.5%) | 1 (0.1%) | 350 (18.7%) | 19763 ms | 2026-10-20 | dark 25d model_unavailable |
panel_v4_free retired
7 slots · 2026-04-26 → 2026-08-18 · 9473/16905 analysableThis panel predates the per-turn score contract, so “Partial” is structurally zero here — well-formed envelopes with null scores were counted as successes rather than flagged. Its “Analysable” column is therefore the honest number, and its historical “Parsed OK” was not.
| Model | Runs | Rows | Data | Analysable | Partial | Incoherent | Avg latency | Last data | Verdict |
|---|---|---|---|---|---|---|---|---|---|
| DeepSeek V3.1 (SambaNova) | 115 | 2415 | 535 (22.2%) | 516 (21.4%) | n/a | 1880 (77.8%) | 2754 ms | 2026-05-24 | dark 86d |
| GLM 4.5 Air (OpenRouter) | 115 | 2415 | 656 (27.2%) | 564 (23.4%) | n/a | 1759 (72.8%) | 32540 ms | 2026-06-09 | dark 70d |
| GPT-OSS 120B (Groq) | 115 | 2415 | 2217 (91.8%) | 2184 (90.4%) | n/a | 198 (8.2%) | 12998 ms | 2026-08-18 | ok |
| Llama 3.3 70B (Groq) | 115 | 2415 | 2363 (97.8%) | 2363 (97.8%) | n/a | 52 (2.2%) | 6762 ms | 2026-08-17 | ok |
| Llama 4 Scout 17B (Groq) | 115 | 2415 | 1583 (65.5%) | 1194 (49.4%) | n/a | 832 (34.5%) | 1996 ms | 2026-07-17 | dark 32d |
| Mistral Small Latest (Mistral) | 115 | 2415 | 2198 (91%) | 1552 (64.3%) | n/a | 219 (9.1%) | 2134 ms | 2026-08-18 | ok |
| Qwen 3 32B (Groq) | 115 | 2415 | 1167 (48.3%) | 1100 (45.5%) | n/a | 1249 (51.7%) | 18996 ms | 2026-07-17 | dark 32d |
panel_v3_free retired
5 slots · 2026-04-21 → 2026-04-25 · 238/250 analysableThis panel predates the per-turn score contract, so “Partial” is structurally zero here — well-formed envelopes with null scores were counted as successes rather than flagged. Its “Analysable” column is therefore the honest number, and its historical “Parsed OK” was not.
| Model | Runs | Rows | Data | Analysable | Partial | Incoherent | Avg latency | Last data | Verdict |
|---|---|---|---|---|---|---|---|---|---|
| GLM 4.5 Air (OpenRouter) | 6 | 50 | 38 (76%) | 38 (76%) | n/a | 12 (24%) | 50584 ms | 2026-04-24 | ok |
| GPT-OSS 120B (Groq) | 6 | 50 | 50 (100%) | 50 (100%) | n/a | 0 (0%) | 8104 ms | 2026-04-25 | ok |
| Llama 3.3 70B (Groq) | 6 | 50 | 50 (100%) | 50 (100%) | n/a | 0 (0%) | 1460 ms | 2026-04-25 | ok |
| Llama 4 Scout 17B (Groq) | 6 | 50 | 50 (100%) | 50 (100%) | n/a | 0 (0%) | 1222 ms | 2026-04-25 | ok |
| Qwen 3 32B (Groq) | 6 | 50 | 50 (100%) | 50 (100%) | n/a | 0 (0%) | 11585 ms | 2026-04-25 | ok |
panel_v2_free retired
5 slots · 2026-04-20 → 2026-04-20 · 44/50 analysableThis panel predates the per-turn score contract, so “Partial” is structurally zero here — well-formed envelopes with null scores were counted as successes rather than flagged. Its “Analysable” column is therefore the honest number, and its historical “Parsed OK” was not.
| Model | Runs | Rows | Data | Analysable | Partial | Incoherent | Avg latency | Last data | Verdict |
|---|---|---|---|---|---|---|---|---|---|
| GLM 4.5 Air (OpenRouter) | 1 | 10 | 7 (70%) | 7 (70%) | n/a | 3 (30%) | 43475 ms | 2026-04-20 | ok |
| GPT-OSS 120B (Groq) | 1 | 10 | 10 (100%) | 10 (100%) | n/a | 0 (0%) | 7701 ms | 2026-04-20 | ok |
| Llama 3.3 70B (Groq) | 1 | 10 | 8 (80%) | 8 (80%) | n/a | 2 (20%) | 1109 ms | 2026-04-20 | ok |
| Llama 4 Scout 17B (Groq) | 1 | 10 | 10 (100%) | 10 (100%) | n/a | 0 (0%) | 1713 ms | 2026-04-20 | ok |
| Qwen 3 32B (Groq) | 1 | 10 | 9 (90%) | 9 (90%) | n/a | 1 (10%) | 8398 ms | 2026-04-20 | ok |