METHOD / V1
Receipts before rankings.
The rules are deliberately boring: deterministic grading, visible configurations, and no winner when the inputs don’t match.
Grading
Benchmark tasks use exact matching, substring checks, JSON parsing, or another declared deterministic rule. An LLM does not judge the answer.
Provenance levels
- Documented local
- Transcribed from dated local notes. Useful, but not independently reproducible without the raw artifact.
- Artifact-backed local
- Includes a sanitized JSON artifact, SHA-256 digest, and harness commit.
- Published external
- Links to an HTTPS publisher source. Never determines a local winner recommendation.
Comparison policy
Hardware, benchmark version, task-set hash, and concurrency must match. Runtime, quant, context, and reasoning differences remain visible warnings.
Memory-fit rule
Total weights must fit within available memory with room for runtime overhead, KV cache, the operating system, and co-resident services. “The model file fits” is not the same as “the workload fits safely.”
Privacy claim
Artifacts and the final build pass an automated scanner plus manual review. This lowers risk; it is not an absolute guarantee that no sensitive detail can ever escape.