How an Open-Weights Sovereign Stack Stacks Up Across 335,000+ Benchmark Queries

1. Condensed 7-Pillar External Scorecard (14 Suites)

Rather than inventing custom micro-benchmarks or grading models with themselves, I ran each subsystem of krusch-stack (v1.1.0) and krusch-grounded-harness (v0.3.0) against fourteen established academic and industry benchmarks inside an isolated 96 GB NVIDIA RTX PRO 6000 Blackwell (sm_120) clean-room VM and local 1024d snowflake-arctic-embed2 loopback (zero outbound cloud API calls on every local open-weights inference run; 28-run, 16,800-trace, 6-family reader [17 $0.00 open-weights + 11 frontier/model-matched runs], Wave-4/5/6 QLoRA+DPO, model-matched GPT 5.6 Luna, Thinking × Search, INV-STACK-11 Grounded v0.2/v0.3 & INV-STACK-10 2PC harness ablations on byte-identical 0o444 SQLite stores & official upstream openai/gpt-5.6-sol rubric judge on DolphinBench; 1,540/1,540 Category 1–4 QA probes across 10 multi-month conversations / 5,882 turns on Snap LoCoMo [5/5 zero-leakage gates]; plus 120 clean-room adyen/DABstep episodes with 42/42 upstream scorer.py parity).

2. How the Stack Compares Against Three Architecture Archetypes

A. vs. Standard Open-Source "Wrapper" Stacks

Typical open-source agent stacks glue LangChain or LlamaIndex splitters (fixed 500-token windows) to a single vector store, stuff every MCP tool schema into the system prompt, pass raw untrusted tool outputs straight back into the model, hand-roll ad-hoc HTTP/auth/SQLite boilerplate on every repo, and allow coding, SQL, or data-science agents to execute unverified 1-shot generations. My controlled ablations isolate why each of those defaults breaks at scale:

• Out-of-Distribution Routing Collapse & Preflight Latency: Supervised routers fit in-distribution benchmarks well, then degrade sharply when domains shift. On ACL 2026 LLMRouterBench, RouteLLM (8.75%) and MLP (9.25%) collapse on unseen ARC-AGI reasoning tasks while in-process pure-CPU Stage-0 structural complexity gating (<15µs, <25µs ceiling prior to model inference) + isotropic 8K snowflake-arctic-embed2 centroids reach 25.25% and the best overall Pareto distance in this self-run evaluation (0.4269). Independently on Martian RouterBench (arXiv:2403.12031 Sec. 5.2, 36,497 0-shot + 36,483 5-shot rows), KNN (0.6648), SVM (0.6363), and MLP (0.5572) all collapse below non-predictive ZeroRouter (0.6941) under Leave-One-Suite-Out OOD, whereas KruschZeroShotRouter (0.7268 0-shot / 0.7455 5-shot) and KruschCascadeRouter (0.7083 / 0.7389) rank #1 and #2 above ZeroRouter while also taking top In-Distribution Micro AIQ (0.7488 / 0.7695). On multi-step Aider Polyglot (n=60), replacing a 569.51ms 7B LLM preflight classifier with deterministic compiler/test failure routing (0.395ms, 1,442x faster) lifts Pass@2 from 21.67% to 51.67% (2.38x).
• Structure-Preserving Indexing vs. Fixed Token Windows: Holding snowflake-arctic-embed2 (1024d) constant on LegalBench-RAG, replacing fixed 160-token windows (11.70% F1) with KruschNexus hierarchical Section → Clause parsing + contextual section-path headers + local mxbai-rerank-large-v2 cross-encoder reranking lifts span-exact Macro F1 by +128.5% (26.74%) while deterministic [CITE_ERROR] citation gates maintain a 0.00% false-statement rate (FSR) across labeled statutory/contract golden sets. Across all 10/10 ACL 2025 CoIR tasks (2,389,123 docs & 162,216 queries), krusch-git AST-spine + identifier subtoken hybrid RRF reaches 56.40% Macro NDCG@10 (+7.42 pp over BM25 48.98%, lifting codefeedback-mt +8.00 pp to 60.23% and synthetic-text2sql +4.22 pp to 48.71%) while beating pure dense vectors across all 8/8 RepoBench-R splits.
• Turn-0 Grounded Harness, 2PC State Grounding, Deferred MCP Discovery & Taint-Tracked Tool Guards: Staging code patches behind TruncationGuard + Verify-on-Stop (with <32ms local 2PC commit finalization across the PostgreSQL write-ahead apply journal, POSIX atomic renameSync/fsync, and pre-commit SHA-256 drift check) heals 72 failed Aider Polyglot suites (+32.00 pp), while read-only node:sqlite FK/cell-value introspection + 2PC SQL self-healing lifts BIRD-SQL Mini-Dev from 26.00% to 40.20% EX (100.00% valid SQL across 500/500 queries) at just 0.09 extra calls/query. Extending Tier 3A to arbitrary MCP tool workflows via krusch-stack (v1.1.0, 68/68 tests) and its INV-STACK-10 Generalized 2PC Tool-Action Harness (action-harness-engine.js) eliminates -95.2% of duplicate MCP calls (42 → 2), cuts premature zero-action stops -50.0%, shrinks archive bloat -17.7%, lifts LBF>=4 synthesis +6.67 pp (22.67% → 29.33%), and reaches 50.33% (302/600) Oracle@2 across all 600 Official DolphinBench tasks on local Qwen3-Coder-30B-A3B (Wave-4 QLoRA+DPO) at $0.00 (and 10 → 0 duplicate MCP calls / -66.7% [3 → 1] zero-action stops / +20.0 pp [0/10 → 2/10] recovery on the 10-task failure slice). Complementing 2PC, Tier 3B krusch-grounded-harness (v0.3.0 · INV-GH-01..10 / INV-STACK-11) replaces unconditional stop-nudging with Turn-0 workspace grounding, passive BLOCK_SPIN_LOOP memoization, scale-aware tiering, and 0-turn defect-only verification—achieving 90.0% (9/10: 100% easy, 85.7% hard) on Gemini 4 Argon @ $0.81 (-74.5% cost vs. $2.64/$3.19 ungrounded & 2.54x faster [75.5s vs. 191.5s], 5.1 turns, 0 extra stop nudges) and 2x accuracy (40.0% vs. 20.0% @ $0.00) on Gemma 4 31B (30.0% local Qwen3-30B-MoE @ $0.00) across 120 clean-room episodes of Adyen DABstep, while across 7 × 600 = 4,200 grounded DolphinBench traces driving Gemini 4 Argon (Low) + Grounded Harness v0.3 to 90.67% (544/600 @ $30.54, 90.50% [181/200] highest Riley slice in self-run 28-run evaluation, +2.00 pp over Std., highest invoked judge accuracy in self-run evaluation 553/590 = 93.73%, +1.50 pp / +9 net tasks over Grounded v0.2 89.17% [535/600] @ $23.92, Argon Low Oracle@2 = 94.67% [568/600], Oracle@3 = 95.83% [575/600]), lifting krush420/gemma4-31b-sre-lora (Wave-6 QLoRA+DPO) to 57.33% (344/600 @ $0.00, highest open-weights score in self-run 28-run evaluation; 54.17% [325/600] uploaded stock submission UUID 288671ad queued on dolphinbench.ai; +3.17 pp / +19 tasks over Stock), lifting Stock Gemma 4 31B by +2.50 pp (54.17% → 56.67% [340/600], +5.50 pp on Riley, 15 spin loops blocked), lifting local 30B Wave-4 to 43.00% (258/600, highest Qwen/MoE in self-run evaluation @ $0.00), and accelerating GPT 5.6 Luna (High) 1.57x (47.02s → 30.03s, 81.50% @ $1.35). Mounting a 64-tool Enterprise MCP hub on all 2,251 UC Berkeley BFCL v3 Live queries, createDeferredToolBridge + schema-repair guard cuts prompt schema tokens by -80.67% (7,894 → 1,526 tok/turn; -68.5% static) while lifting Micro accuracy from 59.88% to 66.28% (+144 queries) and irrelevance abstention from 67.23% to 84.69% (+17.46 pp). And on ETH SpyLab AgentDojo v1.2.2 (1,046 episodes), taint-tracked untrusted boundary delimiting + deterministic sink-provenance policy floors drive indirect prompt-injection ASR from 19.28% (183/949) down to 0.00% (0/949 across 1,046 episodes) with zero benign utility degradation.
• Certified Dual-Runtime Lego Architecture vs. AI-Generated Boilerplate Slop: Rather than regenerating fragile plumbing per service, @krusch/toolkit v1.8.1 & krusch_toolkit v1.6.0 ship 39 dual-runtime JS/TS & Python Lego bricks (64 .d.ts subpaths, 141/141 Node & 201/201 pytest suites—including atomic Idempotency-Key locks, immutable SHA-256/SQLite audit-log chains, zero-node:* rbac/IDOR guards, and HMAC keyset pagination) plus 7 zero-LLM blueprints and 16 <1ms Slop-Breaker detectors (krusch-lego, adding unclamped Prisma/Mongoose pagination, outbound SSRF taint, and hardcoded auth-secret fallback detectors), enforcing 104/104 (100%, 0/72 scripts/) zero-boilerplate compliance across all 104 live workspace targets monitored by Jean SRE.

B. vs. Commercial Cloud Router, Memory, Code & Legal Stacks

• Cloud Routing Economics: Across a clean-room 8,400 / 8,400 live OpenRouter rebuild on RouteWorks/RouterArena (PR #169 open awaiting maintainer merge, origin/main 44cc1ff, github-actions[bot] CI-verified upstream PR Run #780, 8,400 unique gen-... receipts, 0 peer copies, 0 template rules), krusch-cascade-router achieves 0.7263 (72.63, #13 / 34) with 74.44% accuracy and 91.43% robustness at $0.3802 / 1K queries ($3.1939 total, $0.00 router overhead via <15µs pure-CPU Stage-0 pre-inference gating)—surpassing Microsoft Azure-Model-Router (0.7037, 72.93% Acc @ $0.75/1K, 2.0x cheaper), OpenAI GPT-5 (0.6432, 73.96% Acc @ $10.02/1K, 26.4x cheaper), OrcaRouter-Adaptive (0.7248 @ $0.83/1K, 22.62% rob.), Martian (0.5756 @ $4.83/1K, 12.7x cheaper), NotDiamond (0.5729 @ $4.10/1K, 10.8x cheaper), and RouteLLM (0.4807). On Martian RouterBench (0-shot ID, n=10,953 eval), KruschCascadeRouter retains 98% of GPT-4 accuracy at $9.89 vs. $36.15 (-72.63% cost, 504.7 µs/q—2.83x faster and half the cost of KNN $18.71) and 95% of GPT-4 at $6.08 (-83.18%). On live multi-GPU generation, speculative SSE logprob/loop stream abort (Path B) cuts 120B TTFT by -48.0% (1,694.5ms → 880.7ms) and compute cost by -37.4% even with all 325 aborted streams (58,743 wasted tokens) strictly billed.
• Snap LoCoMo, Contradiction, 10M Multi-Session & 28-Run Official DolphinBench Memory: On Snap LoCoMo (ACL 2024, 1,540/1,540 Category 1–4 QA probes across 10 multi-month conversations / 5,882 turns, unmodified 0o444 SovereignMemoryStore, $0.00 zero-LLM ingestion via local snowflake-arctic-embed2 1024d + Porter-FTS5 4-Way RRF, 5/5 zero-leakage gates), argon_matched achieves 90.26% (1,390/1,540) judged accuracy and 95.77% dia_id evidence recall at 181.8 ms median retrieval (+19.86 pp above LivingBrain zero-LLM 70.4%), while argon_tight holds 89.68% (1,381/1,540) at 4,465 mean tokens (6.73x compression) & 91.05% dia_id recall (LoCoMo Oracle@2 = 92.66% [1,427/1,540]). Packing a fixed ~8K-token slice (0.66% of a 10M-token chat) on zero-Docker node:sqlite for ICLR 2026 BEAM, krusch-context-mcp v2.0.0 shows zero E2E degradation from 100K → 500K → 1M (70.60% → 71.82% → 72.86% on hermes3) and beats cloud past.dev (openai/gpt-5.6-luna) on Contradiction Resolution across all 4/4 splits (92.50% Stock / 98.13% SRE-QLoRA vs. 78.13% at 100K; 89.64% vs. 78.57% at 500K; 77.68% vs. 75.36% at 1M; 78.75% vs. 73.13% at 10M) and on 10M Multi-Session Reasoning (29.54% Qwen / 38.33% Hermes vs. 28.17%). Across 28 Official 600-Task DolphinBench runs (16,800 openai/gpt-5.6-sol traces across 6 model families: 17 $0.00 open-weights + 11 frontier/model-matched runs across Google Gemini, OpenAI GPT-5.6 Luna, Google Gemma 4, Alibaba Qwen, OpenAI GPT-OSS, and Nous Hermes) reading the byte-identical 0o444 frozen $0.00 SovereignMemoryStore SQLite checkpoints under the official openai/gpt-5.6-sol rubric judge, open-weights krush420/gemma4-31b-sre-lora (Wave-6 assistant-masked QLoRA + BF16 ref-anchored DPO) + Grounded reaches 57.33% (344/600 @ $0.00, highest open-weights score in self-run 28-run evaluation; 54.17% [325/600] uploaded stock submission UUID 288671ad queued on dolphinbench.ai; Morgan 149, Alex 103, Riley 92; Mech 61.22%, Sem 63.67%, 50.81s median; +3.17 pp / +19 tasks over Stock [+5 Morgan, +8 Alex, +6 Riley], +0.67 pp / +4 tasks over 56.67% [340/600] Stock Google Gemma 4 31B + Grounded [+2.50 pp overall / +5.50 pp on Riley, 15 spin loops blocked], +2.50 pp over 54.83% [329/600] Wave-5, +3.67 pp over 53.67% [322/600] Wave-4, 52.00% No-Think, Gemma 4 Oracle@6 = 71.83% [431/600, +1.16 pp above published 70.67% Mem0], +22.00 pp over matched-scale Qwen2.5-Coder-32B 35.33%) and local krush420/qwen3-coder-30b-sre-lora (Wave-4 + Grounded) hits 43.00% (258/600, highest Qwen/MoE in self-run evaluation; 42.67% [256/600, 85.73% mechanical accuracy] Wave-4 Std.; 42.00% + 2PC, 50.33% Oracle@2; 38.50% [231/600] Wave-6 + Grounded MoE control; Wave-4 Oracle@3 = 58.50% [351/600], 30B Oracle@5 = 61.17% [367/600]) at $0.00 (alongside 39.50% [237/600] gpt-oss-120b-sre-lora, 34.50% Hermes-4-14B Stock, and a 15.7x Wave-4 lift on Hermes-3-8B from 1.17% → 18.33% [110/600] across 4/4 @krush420 adapters)—shattering the 50% open-weights barrier to beat all 8 non-GPT-5.6-Luna commercial cloud baselines (MiniMax M3 + Mem0 47.83% @ $148.02 through Claude Sonnet 5 + Mem0 32.33% @ $1,830.57, exceeding all 8 of those published cloud entries at $0.00; local open-weights study →). Crucially, running model-matched GPT 5.6 Luna (High) + Grounded—the exact same frontier backbone powering the current 70.67% published Hermes + GPT 5.6 Luna + Mem0 entry ($96.21)—against my $0.00 SovereignMemoryStore reaches 81.50% (489/600, 30.03s = 1.57x faster than 81.33% [488/600] Base @ $1.35; 80.50% + 2PC @ $1.38; Luna Oracle@5 = 91.83% [551/600]), beating published 70.67% Mem0 by +10.83 pp (+65 tasks) at 71.1x lower total cost (while Luna Low hits 76.67% [460/600] @ $1.10, +6.00 pp at 87.4x lower cost, giving Sovereign Raw-RAG 100% of the 4-vertex global Cost–Accuracy Pareto frontier: $0.00 → $1.10 → $1.35 → $24.54). Pairing those exact same 0o444 frozen $0.00 SQLite stores with Gemini 4 Argon (Low) scales to 91.17% (547/600 @ $24.54, uploaded dolphinbench.ai submission UUID c047b063, queued awaiting leaderboard acceptance; exceeds current 70.67% published entry; 35.38s median latency) and 90.67% (544/600 @ $30.54, 90.50% [181/200] highest Riley slice in self-run 28-run evaluation) on Argon Low + Grounded Harness v0.3 (+2.00 pp over Std., highest invoked judge accuracy in self-run evaluation 553/590 = 93.73%, +1.50 pp / +9 net tasks over Grounded v0.2 89.17% [535/600] @ $23.92; Argon Low Oracle@2 = 94.67% [568/600], Argon Low Oracle@3 = 95.83% [575/600], Argon Oracle@4 = 96.83% [581/600]; 89.17% Argon Medium @ $24.57; and 78.00% Turn-0 Briefing-Only @ $15.76)—decomposing my +20.50 pp (+123-task) margin over published 70.67% Mem0 at 3.9x lower cost into 52.0% (+10.67 pp) Zero-LLM Raw-RAG memory and 48.0% (+9.83 pp, p = 5.96×10⁻⁹) Argon Low backbone, while establishing 98.33% (590/600) Global Oracle@28 retrieval sufficiency (<= 1.67% [10/600] zero-LLM Raw-RAG misses), 83.17% (499/600) Oracle@17 across $0.00 open-weights alone (+12.50 pp above published 70.67% Mem0), and 71.83% (431/600) Gemma 4 Oracle@6 (+1.16 pp above published 70.67% Mem0), a 4.19x LBF=4+ multiplier (89.33% vs. 21.33%), a significant 2×2 Thinking × Search interaction (ΔΔ = +3.17 pp, p = 0.0320), and flatter corpus-entropy decay (-7.00 pp Argon Low / -3.50 pp Argon Medium vs. -10.00 pp on Mem0 [156 → 136] to -32.50 pp on MiniMax M3 + Hindsight; Return on Memory [queued submission] scaling study →).
• Offline-Capable Code & Legal Retrieval Precision: On MAUD merger agreements (1,676 queries), KruschNexus hits 33.95% character-exact Recall@2 locally—exceeding published OpenAI text-embedding-3-large + Cohere Rerank Recall@64 (28.28%) while retrieving 32x fewer spans and enforcing a 0.00% false-statement rate across labeled statutory/contract golden sets. On ACL 2025 CoIR (10/10 tasks, 2.39M docs), krusch-git's local 1024d hybrid RRF (56.40% NDCG@10) tops proprietary Voyage-Code-002 (56.26%) and OpenAI-ada-002 (53.66%; see §3 context-window disclosure).

C. Clean-Room Auditability vs. Unverified Leaderboard Claims

Every run in my benchmark archive passes 95 / 95 automated integrity gates: (1) loopback air-gap verification (127.0.0.1:11434, cloud API keys unset during sovereign model inference), (2) runtime JavaScript Proxy traps, opaque RPC session tokens, frozen 0o444 pre-populated SQLite memory stores & data partitions on DolphinBench, Snap LoCoMo (5/5 zero-leakage gates), and DABstep, and Python pre-eval column stripping on ground-truth answer/SQL/qrel/injection fields that block oracle access before inference, (3) RestrictedUnpickler + SafeZipExtractor guards, read-only DatabaseSync SQL enforcement, full-corpus 2,389,123-doc anti-shrinkage checks, and zero hardcoded attacker literals on AgentDojo, (4) mandatory Exercism test un-skipping + pristine test SHA-256 checks, and (5) zero LLM self-judging (deterministic compiler, SQL set-equivalence, AST bijection, upstream scorer.py [42/42 parity], and math verifiers on 10 benchmarks; dual cross-family open-weights referees on BEAM; official openai/gpt-5.6-sol rubric judge on DolphinBench; official Category 1–4 QA judge + exact dia_id turn verification on Snap LoCoMo).

3. Where Frontier Cloud Stacks Still Beat My Stack (Honest Gaps & Open PRs)

• 1. Open-Weights Multi-Fact Synthesis Ceiling vs. Frontier Reasoning on BEAM & DolphinBench (70.6%–73.5% BEAM & 34.50%–57.33% Open-Weights DolphinBench vs. 81.50% Luna / 91.17% Argon / 92.1% Cloud): My 28-run (16,800-trace, 6-family) Official openai/gpt-5.6-sol DolphinBench ablation isolated the open-weights gap strictly to multi-fact reader synthesis and post-training specialization rather than retrieval: holding the $0.00 0o444 SovereignMemoryStore byte-for-byte constant (98.33% [590/600, <= 1.67% / 10/600 misses] Global Oracle@28 overall, 83.17% [499/600] Oracle@17 across $0.00 open-weights runs alone—already +12.50 pp above published 70.67% Mem0!—and 71.83% [431/600] Gemma 4 Oracle@6), krush420/gemma4-31b-sre-lora (Wave-6 QLoRA+DPO) + Grounded reaches 57.33% (344/600 @ $0.00, highest open-weights score in self-run 28-run evaluation; 54.17% [325/600] uploaded stock submission UUID 288671ad queued on dolphinbench.ai; 56.67% [340/600] Stock + Grounded; 54.83% Wave-5; 53.67% Wave-4)—beating matched-size code-specialized Qwen2.5-Coder-32B (35.33%) by +22.00 pp (+21.33 pp Stock + Grounded, +18.83 pp Stock, p = 1.35×10⁻¹⁶) and all 8 non-GPT-5.6-Luna cloud stacks—while model-matched GPT 5.6 Luna (High) + Grounded reaches 81.50% (489/600, 30.03s = 1.57x faster than 81.33% Base @ $1.35, +10.83 pp over published 70.67% GPT 5.6 Luna + Mem0 @ $96.21 at 71.1x lower cost; Luna Low 76.67% @ $1.10, 87.4x cheaper; Luna Oracle@5 = 91.83%) and Gemini 4 Argon (Low) reaches 91.17% (547/600 @ $24.54, uploaded dolphinbench.ai submission UUID c047b063, queued awaiting leaderboard acceptance; exceeds current 70.67% published entry) alongside 90.67% (544/600 @ $30.54, 90.50% [181/200] highest Riley slice in self-run 28-run evaluation) on Grounded Harness v0.3 (89.17% Grounded v0.2 & Medium; Oracle@2 = 94.67% [568/600], Oracle@3 = 95.83% [575/600], Argon Oracle@4 = 96.83% [581/600]), decomposing my +20.50 pp margin into 52.0% (+10.67 pp) Zero-LLM Raw-RAG memory + 48.0% (+9.83 pp, p = 5.96×10⁻⁹) Argon Low backbone. Across purely open-weights readers, four distinct factors explain the remaining gap:
  • Cross-family referees vs. cloud self-judging on BEAM: I grade open-weights gpt-oss:120b (5.1B active MoE) with two independent cross-family open-weights judges (qwen2.5:32b and hermes3:latest, 81%–88% binary agreement). By contrast, past.dev's 92.1% uses openai/gpt-5.6-luna as both reader and judge.
  • Exact-position ordering rubrics: On Event Ordering (100K), krusch-context-mcp v2.0.0 retrieves 81.19% of gold turns, yet qwen2.5:32b scores 9.68% while hermes3:latest scores 43.17% (vs. 100% past.dev) because qwen2.5:32b awards 0.0 whenever a 10-event timeline shifts by one slot.
  • Passive supersedes_id = NULL ingestion on Knowledge Update: Despite 97.50% retrieval recall at 100K, E2E QA lands at 37.5%–52.5% (vs. 86%–92.5% past.dev) because my zero-heuristic benchmark loader ingests raw chat turns with supersedes_id = NULL, leaving outdated historical turns active in the 8K window. In production IDE sessions, coding agents explicitly invoke krusch_context_revise({ action: "supersede" }) to mark superseded facts inactive at the SQLite/Postgres layer.
  • 4+-fact exhaustive compositional synthesis in open-weights decoders: While $0.00 open-weights readers (57.33% gemma4-31b-sre-lora [Wave-6 + Grounded], 56.67% Gemma 4 31B [Stock, Think ON + Grounded], 54.83% Gemma 4 31B [Wave-5 Std.], 54.17% Gemma 4 31B [Stock, Think ON], 53.67% Gemma 4 31B [Wave-4 Std.], 52.00% Gemma 4 31B [No-Think] [Gemma 4 Oracle@6 = 71.83%], 43.00% 30B Wave-4 [+ Grounded, highest Qwen/MoE in self-run evaluation] [30B Oracle@5 = 61.17%], 42.67% 30B Wave-4, 42.00% 30B Wave-4 [+ 2PC] lifting LBF>=4 +6.67 pp to 29.33% with 50.33% Oracle@2, 42.00% 30B Stock, 39.50% 120B Wave-4, 39.17% 120B Stock, 38.50% 30B Wave-6 [+ Grounded] & 80B Stock, 35.33% 32B Dense Stock, and 34.50% Hermes-4-14B Stock) beat commercial cloud stacks up to $1,830.57, open-weights decoders still trail frontier reasoning models on 4+-fact exhaustive synthesis (21.33% [16/75] on Gemma 4 31B Stock & 30B Stock vs. 68.00% / 72.00% on Luna High [Base / + 2PC] and 89.33% [67/75] on Argon Low at LBF=4+, a 4.19x multiplier and 16.75x over 8B Wave-4 5.33%) where open-weights models omit secondary constraint clauses despite retrieved facts sitting in context.
• 2. Multi-Turn Frontier Reasoning Upper Bound on Coding, Text-to-SQL & Data Science (51.56% Polyglot, 40.20% BIRD-SQL & 40.0% Open-Weights DABstep vs. 65%–90%+ Cloud): Lifting open-weights gpt-oss:120b at $0 API cost from 19.56% → 51.56% (116/225) on Aider Polyglot matches Claude 3.5 Sonnet (51.6%) and beats GPT-4o (23.1%), lifting BIRD-SQL Mini-Dev from 26.00% → 40.20% EX (100% valid SQL at 1.09 calls/query) recovers +14.20 pp without fine-tuning, and krusch-grounded-harness doubles open-weights Gemma 4 31B accuracy on Adyen DABstep from 20.0% → 40.0% ($0.00)—yet hard algorithmic C++/Rust/Go/Java suites, multi-join SQL aggregates, and multi-file financial reconciliations still separate open-weights decoders from frontier reasoning (90.0% on Gemini 4 Argon + Grounded @ $0.81 on DABstep, and 75%–88%+ coding / 65%–75%+ multi-candidate SQL). That gap is why my Cascade Router retains an L3 frontier escalation tier for the hard tail.
• 3. Monolingual English Feature Variance on Held-Out CJK OOD (0.0811 leave-chinese-out 0-Shot): On Martian RouterBench 0-Shot Leave-One-Suite-Out OOD, learned KruschCascadeRouter ranks first among learned routers on 7-suite Macro AIQ (0.6873) and #2 overall Micro AIQ (0.7083, behind zero-shot KruschZeroShotRouter at 0.7268), yet drops to AIQ = 0.0811 on held-out chinese (n=236, pulling Macro-9 to 0.6156) because training exclusively on English 0-shot suites produces std(cjk_ratio) ≈ 0 against the 1e-4 z-score variance floor. In-distribution 0-shot chinese ranks #1 (0.4880) and 5-shot OOD chinese recovers to 0.3059; zero post-hoc test tuning was applied.
• 4. ACL 2025 CoIR Context-Length Protocol Difference (8,192-Token Production Window vs. Table 3 512-Token Truncation): In the ACL 2025 CoIR paper's Table 3, all 10 baseline retrievers (including Voyage-Code-002 56.26% and E5-Mistral 7B 55.18%) were evaluated under a uniform 512-token truncation cap so legacy 512-token encoders could be compared side by side. My krusch-git run (56.40% 10-task Macro NDCG@10 across all 2,389,123 unshrunk docs) evaluates snowflake-arctic-embed2:latest at its native 8,192-token production context window + AST spine headers—reflecting real repository indexing while accounting for part of the margin on long-document tasks such as apps and stackoverflow-qa.
• 5. Open Upstream Leaderboard Submissions (Explicit Status):
  • DolphinBench (dolphinbench.ai): All 8/8 validated official .zip submission archives—sovereign_gemini_4_argon_low.zip (91.17%, 547/600 @ $24.54, uploaded dolphinbench.ai submission UUID c047b063-822c-49fb-8b36-27c3763d6ca7, queued awaiting leaderboard acceptance; exceeds current 70.67% published entry; SHA-256 80a199a60aef65f0d963d0402a2e3b0af99b82ed63e2f422001234b6ed770f0b), sovereign_gemini_4_argon.zip (89.17%, 535/600 @ $24.57, UUID 29389157-b01b-42b5-8193-17d9a496f580, SHA-256 9a7fa41fb3d1cf5f71f0c3cf74a0c97f8f39d50e26a7dd484c323f7719ecb7a0), sovereign_gemma4_31b_stock.zip (54.17%, 325/600 uploaded stock open-weights submission @ $0.00, UUID 288671ad-afe0-40a3-93af-b834c9bbd306, queued on dolphinbench.ai, SHA-256 2a23c2a023473a6642f5a61f1d99ff569f3d50ac53e60576f8195eef659c254b), plus local $0.00 30B Wave-4 42.67% (fd45f16c-fd3c-471c-b93d-413f61f1dbcf), 30B Stock 42.00% (9cac5ab1-ecba-40a6-ad84-55ec2d67bfce), 120B Wave-4 39.50% (83a4c085-0e44-42af-a625-b496e63c8949), 120B Stock 39.17% (7b5073fd-e1d9-458b-a607-7c55f76b6df1), and 8B Wave-4 18.33% (19552654-be39-4fe0-8043-a49158582356)—are verified (submission.validate / export_submission --check, error_code: null) and queued on dolphinbench.ai with status: "queued" awaiting upstream maintainer queue processing, alongside 20 additional clean-room Wave-4/5/6, Grounded (v0.2/v0.3), model-matched, 2PC harness, and scaling controls (Gemini 4 Argon Low + Grounded Harness v0.3 90.67% [544/600 @ $30.54, 90.50% (181/200) highest Riley slice in self-run 28-run evaluation, 93.73% (553/590) invoked judge], Argon Low + Grounded v0.2 89.17% [535/600 @ $23.92], krush420/gemma4-31b-sre-lora Wave-6 + Grounded 57.33% [344/600 @ $0.00, highest open-weights score in self-run 28-run evaluation, SHA-256 b7a26ef9…], Stock Gemma 4 31B + Grounded 56.67% [340/600], Gemma 4 31B Wave-5 Std. 54.83% [329/600, SHA-256 5b1a49b1…], Gemma 4 31B Wave-4 Std. 53.67% [322/600, SHA-256 05e2d106…], Gemma 4 31B 52.00% No-Think, GPT 5.6 Luna High + Grounded 81.50%, GPT 5.6 Luna High 81.33% Base & 80.50% + 2PC, Luna Low 76.67% Base & 76.50% + 2PC, krush420/qwen3-coder-30b-sre-lora Wave-4 + Grounded 43.00% [258/600, highest Qwen/MoE in self-run evaluation, SHA-256 efc93d02…], 30B Wave-4 + 2PC 42.00%, 30B Wave-6 + Grounded 38.50% [231/600, SHA-256 f00271bb…], Argon Turn-0 Briefing-Only 79.17% & 78.00%, 80B Stock 38.50%, Qwen2.5-Coder-32B 35.33%, and Hermes-4-14B 34.50%; 28/28 verified 600-task runs = 16,800 official openai/gpt-5.6-sol traces).
  • RouteWorks/RouterArena (PR #169): Clean-room 8,400 / 8,400 live OpenRouter rebuild (origin/main 44cc1ff, 2026-10-31 pricing, github-actions[bot] CI-verified upstream PR Run #780, 8,400 unique gen-... IDs, receipts_sha256: bede6659ebc8513d7f13fb7588b4ca2ec4df91c71d1accad8b9795b60d3db45e, 0 peer copies, 0 template rules) scoring 0.7263 (72.63, #13 / 34) with 74.44% accuracy, $0.3802 / 1K cost ($3.1939 total, $0.00 router overhead), and 91.43% robustness across all 8,400 queries + 420 robustness perturbations (0 abnormal entries); PR #169 is open and awaiting maintainer review and merge onto the upstream leaderboard.
  • withmartian/routerbench (kruschdev/routerbench:feat/krusch-cascade-router): Drop-in AbstractRouter implementation (KruschCascadeRouter + KruschZeroShotRouter) and 4-regime ID/OOD harness published on my public fork branch and ready for upstream PR submission.
  • ETH SpyLab AgentDojo, Snap LoCoMo, Adyen DABstep, UC Berkeley BFCL & Aider Polyglot: My full 1,046-episode AgentDojo v1.2.2 (0.00% ASR), 1,540/1,540-probe Snap LoCoMo (90.26% / 95.77% dia_id recall, 5/5 zero-leakage gates), 120-episode adyen/DABstep (42/42 scorer.py parity), 2,251-query BFCL v3 Live, and 225/225 Aider Polyglot runs execute the official upstream suites inside clean-room harnesses; upstream Python adapter PR playbooks for ethz-spylab/agentdojo and Aider-AI/polyglot-benchmark remain unsubmitted next steps.
• 6. Upstream Dataset Hygiene Notes: Two upstream rows in RepoBench-R ([5522, 7591] in python_cff/test_easy) carry out-of-bounds gold_snippet_index pointers and are deterministically quarantined (47,998 / 48,000 evaluated). In Martian RouterBench routerbench_5shot.pkl, 28 upstream rows have missing model cells across the 11-model matrix and are deterministically filtered so all 6 routers evaluate the identical 36,483 / 36,511 complete rows. In UC Berkeley BFCL v3 BFCL_v3_live_relevance.json, upstream ships a duplicate row ID (live_relevance_3-3-0 at lines 3 and 4 with distinct prompts), deterministically disambiguated as live_relevance_3-3-0__dup2 so all 18 / 18 rows (2,251 / 2,251 Live queries) execute without overwrite. In ICLR 2026 BEAM 100K, upstream Conversation 14 Knowledge Update Q1 asks about a nonexistent "running shoe" topic; my verifier asserts score == 0.00 (canary_conv14_ku_q1_zero: true).

4. Raw Artifacts & 60-Second Cryptographic Verification

Every raw per-query audit artifact (49 SHA-256-pinned Hugging Face audit logs = 48 raw JSON + 1 live PR #169 receipt bundle across 64 files across all 14 external suites, 40 dolphinbench_600 rows; 50 ecosystem total w/ DABstep, alongside all 28 SHA-256-pinned Official openai/gpt-5.6-sol archives / 16,800 traces extending Paper #27) and 14 Hugging Face Dataset Viewer splits are published under CC-BY-4.0 at krush420/sovereign-g4-benchmarks, alongside the interactive Pareto & trajectory explorer at krush420/sovereign-router-arena and repository-level reproduction docs in kruschdev/krusch-stack and kruschdev/grounded-harness.

# Clone the raw clean-room audit logs & verify all SHA-256 digests in <60s
git clone https://huggingface.co/datasets/krush420/sovereign-g4-benchmarks
cd sovereign-g4-benchmarks

python3 -c '
import hashlib, json, pathlib
rows = [json.loads(l) for l in pathlib.Path("data/scorecard_summary.jsonl").read_text().splitlines()]
for r in rows:
    if r["audit_log_file"].startswith("raw_audit_logs/"):
        digest = hashlib.sha256(pathlib.Path(r["audit_log_file"]).read_bytes()).hexdigest()
        assert digest == r["audit_log_sha256"], f"Mismatch: {r[\"audit_log_file\"]}"
        print("VERIFIED:", r["audit_log_file"], digest[:16])
'