官方公开
Manus 的上下文工程经验
状态感知动作、文件化记忆、保留失败记录。
产品团队的工程说明，不等于内部系统复原。 Agent architecture mechanism study 两组正式实验
同一模型，为什么会跑出完全不同的 Agent？
模型只负责提出下一步，真正决定执行质量的，往往是它外面的状态、动作边界、恢复规则和完成检查。我们固定同一个 DeepSeek V4 Flash，把这些控制方式拆成三种工程架构，再用两组配对实验观察它们在哪些任务里有效、又会在哪里失灵。
30 秒结论
在这组规则明确的任务里，Pilot A 的工作流受控执行通过了全部代码验收；Pilot B 没有出现统一赢家。目标图混合执行缩短了并行关键路径，却因重复选择已被拒绝的动作，消耗了约 3.3 倍 tokens 和 3.1 倍时间。架构不是越复杂越好，关键是控制逻辑是否与任务结构相配。
- 正式运行
- 54Pilot A 30 + Pilot B 24
- Pilot A
- 0/15 · 15/15开放式 / 工作流受控
- Pilot B
- 8/12 · 8/12开放式 / 目标图混合
- 冻结模型
- V4 Flashthinking off · temperature 0
02 · Evidence map
先找公开线索，再把线索变成可检验的架构假设
商业产品资料只回答“可能有哪些设计思路”。真正的效果判断只来自本地冻结实验。
官方公开
Genspark 的开放工具执行思路
模型驱动循环、多模型协作与大量专用工具。
产品立场与案例，不能单独证明因果。 开源参考
OpenManus
可检查的通用 Agent、浏览器与多 Agent 入口。
社区实现，不代表 Manus 私有架构或效果。行动—观察循环、轨迹搜索、状态变化与最终验收。
提供机制与评测线索，不替任何产品背书。
比较什么不同证据在研究链路中的位置。
怎样读从左到右，证据约束逐步增强。
不能推出不能据此还原商业产品的私有实现。
03 · Architecture paths
两组实验各自固定底座，只替换“下一步由谁决定”
Pilot A 比较开放式执行与工作流受控执行；Pilot B 比较开放式执行与目标图混合执行。三种架构不是同场比较，任务和工具只在各自的配对实验内保持一致。
Contracted ReAct
开放式执行
模型依据完整记录，自行决定下一步、恢复方式和停止时机。- 01 读取任务与记录
- 02 选择工具与参数
- 03 接收环境结果
- 04 继续或停止
可优先验证： 路径未知、探索空间大、错误成本可控
Workflow Controlled
工作流受控执行
程序维护状态和合法动作，模型只在当前允许范围内判断。- 01 读取任务状态
- 02 开放合法动作
- 03 按错误类型恢复
- 04 代码验收后停止
可优先验证： 规则清楚、顺序严格、失败代价较高
Goal-Graph Hybrid
目标图混合执行
程序维护目标依赖与证据状态，模型在多条合法路径之间选择。- 01 拆成目标节点
- 02 释放可执行分支
- 03 汇总证据状态
- 04 合并并验收
可优先验证： 子目标可并行、路径不止一条、仍需可控边界
比较什么三种控制回路如何决定下一步。
怎样读先分清 Pilot A 与 B，再看各自的控制回路。
不能推出架构层次更多，不代表效果自然更好。
REAL TRACE · 完成实验记录回放
同一输入进入两套架构后，信息怎样流动？
这里播放的是已经完成的实验记录，不会在浏览器里重新调用模型，也不展示隐藏推理。
说明性配对运行
采购限流恢复
在相同附件、工具、限流场景、模型与预算下完成采购决策并提交可验证证据。
- 模型
- deepseek-v4-flash
- 预算
- 20 次工具调用 · 最多 16 次模型调用 · 180 秒
- 统一工具入口
- read_file · list_resources · fetch_resource · write_output · wait_or_advance · finish
- 完成条件
- 相同代码验收，只检查最终结果
- 运行环境
- rate_limit 冻结夹具，独立运行目录
查看冻结原文与输入附件
实际用户输入
Complete the procurement task using only the listed tools and their observations.
requirements.md
# Procurement requirements Purchase exactly 100 replacement control modules. Consider only approved vendors quoting in USD that can deliver within seven calendar days. For each eligible vendor, use its current live quote when one is available; otherwise use its standard catalog price. Select the one vendor with the lowest authoritative unit price. The final `decision.json` must distinguish `unit_price_usd` from `total_price_usd`; the total is quantity 100 multiplied by the selected unit price. It must also identify the selected `vendor_id`, `vendor_name`, `quantity`, and `currency`.SHA-256 · ce939c27df8f0b0560361e1f80e7fecb1297cbaa45d2197b0aea55bcbea6ee68
vendors.csv
vendor_id,vendor_name,approved,currency,standard_unit_price_usd,delivery_days atlas,Atlas Components,yes,USD,104.00,5 northstar,Northstar Supplies,yes,USD,99.00,6 harbor,Harbor Industrial,no,USD,91.00,4SHA-256 · b0b987b072d129f44d1d91627b671b46d54df5d848aaa4f851643e0e3bc0d6d3
amendments.csv
amendment_id,description A-17,Use the current live quote instead of the standard catalog price when available.SHA-256 · d4c5ffe7542c67e014e0c7700567cfb71bd5cb5fe913b2b40ba40e1463b47d21
这组新配对只解释执行链，不计入 Pilot A 或 Pilot B 的成功率。
正式实验记录
替代路径 A-01
主证据路径失败后，切换到独立副本并提交受事实 ID 支持的发布令牌。
- 模型
- deepseek-v4-flash
- 预算
- 3 个并行分支 · 最多 18 次模型调用 · 24 次工具调用 · 30,000 tokens · 240 秒
- 统一工具入口
- list_evidence · read_evidence · submit_result · finish
- 完成条件
- 相同代码验收，只检查最终结果
- 运行环境
- 冻结 Pilot B 夹具，独立运行目录
查看冻结原文与输入附件
实际用户输入
Recover the canonical release token for record A-01. Return {"release_token": string} and cite only supporting fact IDs. tasks.json
{
"schema_version": "1.0",
"study": "pilot-b",
"instances": [
{
"instance_id": "alternative_path-01",
"family": "alternative_path",
"task": "Recover the canonical release token for record A-01. Return {\"release_token\": string} and cite only supporting fact IDs.",
"sources": [
{
"source_id": "a01-primary",
"label": "Primary release archive",
"role": "route",
"outcome": "error",
"error_code": "route_unavailable",
"facts": []
},
{
"source_id": "a01-replica",
"label": "Independent release replica",
"role": "route",
"outcome": "ok",
"error_code": null,
"facts": [
{"fact_id": "alt-01-token", "goal_id": "resolve_token", "claim": "release_token", "value": "CIRRUS-41"}
]
}
],
"required_fact_ids": ["alt-01-token"],
"expected_answer": {"release_token": "CIRRUS-41"},
"subgoals": ["resolve_token"],
"merge_rule": null
},
{
"instance_id": "alternative_path-02",
"family": "alternative_path",
"task": "Recover the canonical release token for record A-02. Return {\"release_token\": string} and cite only supporting fact IDs.",
"sources": [
{
"source_id": "a02-primary",
"label": "Primary release archive",
"role": "route",
"outcome": "error",
"error_code": "route_unavailable",
"facts": []
},
{
"source_id": "a02-replica",
"label": "Independent release replica",
"role": "route",
"outcome": "ok",
"error_code": null,
"facts": [
{"fact_id": "alt-02-token", "goal_id": "resolve_token", "claim": "release_token", "value": "EMBER-73"}
]
}
],
"required_fact_ids": ["alt-02-token"],
"expected_answer": {"release_token": "EMBER-73"},
"subgoals": ["resolve_token"],
"merge_rule": null
},
{
"instance_id": "alternative_path-03",
"family": "alternative_path",
"task": "Recover the canonical release token for record A-03. Return {\"release_token\": string} and cite only supporting fact IDs.",
"sources": [
{
"source_id": "a03-primary",
"label": "Primary release archive",
"role": "route",
"outcome": "error",
"error_code": "route_unavailable",
"facts": []
},
{
"source_id": "a03-replica",
"label": "Independent release replica",
"role": "route",
"outcome": "ok",
"error_code": null,
"facts": [
{"fact_id": "alt-03-token", "goal_id": "resolve_token", "claim": "release_token", "value": "HARBOR-29"}
]
}
],
"required_fact_ids": ["alt-03-token"],
"expected_answer": {"release_token": "HARBOR-29"},
"subgoals": ["resolve_token"],
"merge_rule": null
},
{
"instance_id": "alternative_path-04",
"family": "alternative_path",
"task": "Recover the canonical release token for record A-04. Return {\"release_token\": string} and cite only supporting fact IDs.",
"sources": [
{
"source_id": "a04-primary",
"label": "Primary release archive",
"role": "route",
"outcome": "error",
"error_code": "route_unavailable",
"facts": []
},
{
"source_id": "a04-replica",
"label": "Independent release replica",
"role": "route",
"outcome": "ok",
"error_code": null,
"facts": [
{"fact_id": "alt-04-token", "goal_id": "resolve_token", "claim": "release_token", "value": "LATTICE-86"}
]
}
],
"required_fact_ids": ["alt-04-token"],
"expected_answer": {"release_token": "LATTICE-86"},
"subgoals": ["resolve_token"],
"merge_rule": null
},
{
"instance_id": "open_evidence-01",
"family": "open_evidence",
"task": "Assemble the current O-01 brief from open evidence. Return codename, launch_window, and owner, citing only current supporting fact IDs.",
"sources": [
{"source_id": "o01-name", "label": "Current naming register", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-01-codename", "goal_id": "codename", "claim": "codename", "value": "Juniper"}]},
{"source_id": "o01-window", "label": "Current launch calendar", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-01-window", "goal_id": "launch_window", "claim": "launch_window", "value": "2026-W31"}]},
{"source_id": "o01-owner", "label": "Current ownership roster", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-01-owner", "goal_id": "owner", "claim": "owner", "value": "Orion"}]},
{"source_id": "o01-history", "label": "Historical naming memo", "role": "distractor", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-01-old-name", "goal_id": null, "claim": "superseded_codename", "value": "Cedar"}]},
{"source_id": "o01-ops", "label": "Unrelated operations note", "role": "distractor", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-01-priority", "goal_id": null, "claim": "queue_priority", "value": "low"}]}
],
"required_fact_ids": ["open-01-codename", "open-01-window", "open-01-owner"],
"expected_answer": {"codename": "Juniper", "launch_window": "2026-W31", "owner": "Orion"},
"subgoals": ["codename", "launch_window", "owner"],
"merge_rule": null
},
{
"instance_id": "open_evidence-02",
"family": "open_evidence",
"task": "Assemble the current O-02 brief from open evidence. Return codename, launch_window, and owner, citing only current supporting fact IDs.",
"sources": [
{"source_id": "o02-name", "label": "Current naming register", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-02-codename", "goal_id": "codename", "claim": "codename", "value": "Kestrel"}]},
{"source_id": "o02-window", "label": "Current launch calendar", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-02-window", "goal_id": "launch_window", "claim": "launch_window", "value": "2026-W34"}]},
{"source_id": "o02-owner", "label": "Current ownership roster", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-02-owner", "goal_id": "owner", "claim": "owner", "value": "Nova"}]},
{"source_id": "o02-history", "label": "Historical ownership memo", "role": "distractor", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-02-old-owner", "goal_id": null, "claim": "superseded_owner", "value": "Lyra"}]},
{"source_id": "o02-ops", "label": "Unrelated capacity note", "role": "distractor", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-02-capacity", "goal_id": null, "claim": "capacity_band", "value": "green"}]}
],
"required_fact_ids": ["open-02-codename", "open-02-window", "open-02-owner"],
"expected_answer": {"codename": "Kestrel", "launch_window": "2026-W34", "owner": "Nova"},
"subgoals": ["codename", "launch_window", "owner"],
"merge_rule": null
},
{
"instance_id": "open_evidence-03",
"family": "open_evidence",
"task": "Assemble the current O-03 brief from open evidence. Return codename, launch_window, and owner, citing only current supporting fact IDs.",
"sources": [
{"source_id": "o03-name", "label": "Current naming register", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-03-codename", "goal_id": "codename", "claim": "codename", "value": "Meridian"}]},
{"source_id": "o03-window", "label": "Current launch calendar", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-03-window", "goal_id": "launch_window", "claim": "launch_window", "value": "2026-W37"}]},
{"source_id": "o03-owner", "label": "Current ownership roster", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-03-owner", "goal_id": "owner", "claim": "owner", "value": "Quill"}]},
{"source_id": "o03-history", "label": "Historical launch memo", "role": "distractor", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-03-old-window", "goal_id": null, "claim": "superseded_window", "value": "2026-W35"}]},
{"source_id": "o03-ops", "label": "Unrelated risk note", "role": "distractor", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-03-risk", "goal_id": null, "claim": "risk_band", "value": "amber"}]}
],
"required_fact_ids": ["open-03-codename", "open-03-window", "open-03-owner"],
"expected_answer": {"codename": "Meridian", "launch_window": "2026-W37", "owner": "Quill"},
"subgoals": ["codename", "launch_window", "owner"],
"merge_rule": null
},
{
"instance_id": "open_evidence-04",
"family": "open_evidence",
"task": "Assemble the current O-04 brief from open evidence. Return codename, launch_window, and owner, citing only current supporting fact IDs.",
"sources": [
{"source_id": "o04-name", "label": "Current naming register", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-04-codename", "goal_id": "codename", "claim": "codename", "value": "Solstice"}]},
{"source_id": "o04-window", "label": "Current launch calendar", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-04-window", "goal_id": "launch_window", "claim": "launch_window", "value": "2026-W40"}]},
{"source_id": "o04-owner", "label": "Current ownership roster", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-04-owner", "goal_id": "owner", "claim": "owner", "value": "Rook"}]},
{"source_id": "o04-history", "label": "Historical naming memo", "role": "distractor", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-04-old-name", "goal_id": null, "claim": "superseded_codename", "value": "Equinox"}]},
{"source_id": "o04-ops", "label": "Unrelated staffing note", "role": "distractor", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "open-04-staffing", "goal_id": null, "claim": "staffing_band", "value": "stable"}]}
],
"required_fact_ids": ["open-04-codename", "open-04-window", "open-04-owner"],
"expected_answer": {"codename": "Solstice", "launch_window": "2026-W40", "owner": "Rook"},
"subgoals": ["codename", "launch_window", "owner"],
"merge_rule": null
},
{
"instance_id": "parallel_decomposition-01",
"family": "parallel_decomposition",
"task": "Resolve independent alpha, beta, and gamma segment values for P-01. Return segments plus total=sum(values) and checksum=alpha-beta-gamma:total, citing all three supporting facts.",
"sources": [
{"source_id": "p01-alpha", "label": "Alpha segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-01-alpha", "goal_id": "alpha", "claim": "segment_value", "value": 7}]},
{"source_id": "p01-beta", "label": "Beta segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-01-beta", "goal_id": "beta", "claim": "segment_value", "value": 13}]},
{"source_id": "p01-gamma", "label": "Gamma segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-01-gamma", "goal_id": "gamma", "claim": "segment_value", "value": 22}]}
],
"required_fact_ids": ["parallel-01-alpha", "parallel-01-beta", "parallel-01-gamma"],
"expected_answer": {"segments": {"alpha": 7, "beta": 13, "gamma": 22}, "total": 42, "checksum": "7-13-22:42"},
"subgoals": ["alpha", "beta", "gamma"],
"merge_rule": "total is the sum of alpha, beta, and gamma; checksum is alpha-beta-gamma:total"
},
{
"instance_id": "parallel_decomposition-02",
"family": "parallel_decomposition",
"task": "Resolve independent alpha, beta, and gamma segment values for P-02. Return segments plus total=sum(values) and checksum=alpha-beta-gamma:total, citing all three supporting facts.",
"sources": [
{"source_id": "p02-alpha", "label": "Alpha segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-02-alpha", "goal_id": "alpha", "claim": "segment_value", "value": 9}]},
{"source_id": "p02-beta", "label": "Beta segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-02-beta", "goal_id": "beta", "claim": "segment_value", "value": 16}]},
{"source_id": "p02-gamma", "label": "Gamma segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-02-gamma", "goal_id": "gamma", "claim": "segment_value", "value": 24}]}
],
"required_fact_ids": ["parallel-02-alpha", "parallel-02-beta", "parallel-02-gamma"],
"expected_answer": {"segments": {"alpha": 9, "beta": 16, "gamma": 24}, "total": 49, "checksum": "9-16-24:49"},
"subgoals": ["alpha", "beta", "gamma"],
"merge_rule": "total is the sum of alpha, beta, and gamma; checksum is alpha-beta-gamma:total"
},
{
"instance_id": "parallel_decomposition-03",
"family": "parallel_decomposition",
"task": "Resolve independent alpha, beta, and gamma segment values for P-03. Return segments plus total=sum(values) and checksum=alpha-beta-gamma:total, citing all three supporting facts.",
"sources": [
{"source_id": "p03-alpha", "label": "Alpha segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-03-alpha", "goal_id": "alpha", "claim": "segment_value", "value": 11}]},
{"source_id": "p03-beta", "label": "Beta segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-03-beta", "goal_id": "beta", "claim": "segment_value", "value": 18}]},
{"source_id": "p03-gamma", "label": "Gamma segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-03-gamma", "goal_id": "gamma", "claim": "segment_value", "value": 27}]}
],
"required_fact_ids": ["parallel-03-alpha", "parallel-03-beta", "parallel-03-gamma"],
"expected_answer": {"segments": {"alpha": 11, "beta": 18, "gamma": 27}, "total": 56, "checksum": "11-18-27:56"},
"subgoals": ["alpha", "beta", "gamma"],
"merge_rule": "total is the sum of alpha, beta, and gamma; checksum is alpha-beta-gamma:total"
},
{
"instance_id": "parallel_decomposition-04",
"family": "parallel_decomposition",
"task": "Resolve independent alpha, beta, and gamma segment values for P-04. Return segments plus total=sum(values) and checksum=alpha-beta-gamma:total, citing all three supporting facts.",
"sources": [
{"source_id": "p04-alpha", "label": "Alpha segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-04-alpha", "goal_id": "alpha", "claim": "segment_value", "value": 14}]},
{"source_id": "p04-beta", "label": "Beta segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-04-beta", "goal_id": "beta", "claim": "segment_value", "value": 21}]},
{"source_id": "p04-gamma", "label": "Gamma segment ledger", "role": "required", "outcome": "ok", "error_code": null, "facts": [{"fact_id": "parallel-04-gamma", "goal_id": "gamma", "claim": "segment_value", "value": 31}]}
],
"required_fact_ids": ["parallel-04-alpha", "parallel-04-beta", "parallel-04-gamma"],
"expected_answer": {"segments": {"alpha": 14, "beta": 21, "gamma": 31}, "total": 66, "checksum": "14-21-31:66"},
"subgoals": ["alpha", "beta", "gamma"],
"merge_rule": "total is the sum of alpha, beta, and gamma; checksum is alpha-beta-gamma:total"
}
]
} SHA-256 · b1c85c278660edf1b45f3bb0d69324808c30b898fb3916c332b979c2ea47a8bc 这是一条正式配对记录，但单条案例仍不能代替 24 次 Pilot B 统计。
已载入采购限流恢复，第 1 步。
架构泳道
开放式执行
根据公开事件生成的教学摘要
读取一份冻结附件，并把事实加入当前记录。
- 当前状态
- { "advanced_units": 0, "evidence_sources": 0, "finished": false, "last_error": "none", "read_inputs": [ "requirements.md" ], "resources_discovered": false, "written_outputs": [] }
- 当前可用工具
- fetch_resource · finish · list_resources · read_file · wait_or_advance · write_output
- 模型生成的工具动作
- 读取附件
- 工具返回
- { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 }
- 状态 / 证据变化
- { "state": { "last_error": "none", "read_inputs": [ "requirements.md" ] }, "evidence": { "fact_count": 2, "source_added": "obs-001" } }
- 代码验收
- pending
{
"path": "requirements.md",
"tool": "read_file"
} {
"disposition": "gateway_success",
"observation": {
"error_code": "none",
"fact_count": 2,
"ok": true,
"retry_after_units": null,
"state_changed": false,
"status_code": 200
},
"prompt_sha256": "08d5d092624b629c5062531c5262d5c9772a26b5c692763526521eb4cf6a85e8",
"response_sha256": "5450adad906e1fc405387273bdaea95b942b399de7c7d6535239fb4e1e325cf1"
} 架构泳道
工作流受控执行
根据公开事件生成的教学摘要
读取一份冻结附件，并把事实加入当前记录。
- 当前状态
- { "advanced_units": 0, "evidence_sources": 0, "finished": false, "last_error": "none", "read_inputs": [ "requirements.md" ], "resources_discovered": false, "written_outputs": [] }
- 当前可用工具
- list_resources · read_file
- 模型生成的工具动作
- 读取附件
- 工具返回
- { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 }
- 状态 / 证据变化
- { "state": { "last_error": "none", "read_inputs": [ "requirements.md" ] }, "evidence": { "fact_count": 2, "source_added": "obs-001" } }
- 代码验收
- pending
{
"path": "requirements.md",
"tool": "read_file"
} {
"disposition": "gateway_success",
"observation": {
"error_code": "none",
"fact_count": 2,
"ok": true,
"retry_after_units": null,
"state_changed": false,
"status_code": 200
},
"prompt_sha256": "cd07c7e89a36fad19c441eb8b9e27a6cb032708385d07143c94b0ac71ef4ca0f",
"response_sha256": "5450adad906e1fc405387273bdaea95b942b399de7c7d6535239fb4e1e325cf1"
} 开放式执行
未通过代码验收没有形成可验收输出
- 冻结输入完整 all protected fixture hashes match the run-start manifest
- 交付格式正确 decision.json is missing
- 供应商选择正确 未执行 · not evaluated because decision schema failed
- 数量与价格正确 未执行 · not evaluated because decision schema failed
- 淘汰理由完整 未执行 · not evaluated because decision schema failed
- 结论有工具证据 evidence/claims.json is missing
16 次模型 · 16 次工具 · 25,829 tokens · 12.56s
工作流受控执行
代码验收通过{
"currency": "USD",
"evidence_file": "evidence/claims.json",
"quantity": 100,
"rejections": [
{
"reason_code": "higher_authoritative_price",
"vendor_id": "atlas"
},
{
"reason_code": "not_approved",
"vendor_id": "harbor"
}
],
"schema_version": "1.0",
"total_price_usd": 9600,
"unit_price_usd": 96,
"vendor_id": "northstar",
"vendor_name": "Northstar Supplies"
} - 冻结输入完整 all protected fixture hashes match the run-start manifest
- 交付格式正确 decision.json matches schema version 1.0
- 供应商选择正确 selected vendor identity and currency match derived Gold
- 数量与价格正确 quantity, unit price, and total match derived Gold
- 淘汰理由完整 all non-selected vendors have the derived reason
- 结论有工具证据 all claims resolve to successful tool-observation facts
9 次模型 · 9 次工具 · 9,373 tokens · 10.49s
这组新配对只解释执行链，不计入 Pilot A 或 Pilot B 的成功率。
开放式执行
代码验收通过{
"release_token": "CIRRUS-41"
} - 任务效用 100
- 关键事实覆盖 100%
- 错误证据 0
- 合并遗漏 0
- 合并冲突 0
5 次模型 · 5 次工具 · 2,796 tokens · 5.02s
目标图混合执行
代码验收通过{
"release_token": "CIRRUS-41"
} - 任务效用 100
- 关键事实覆盖 100%
- 错误证据 0
- 合并遗漏 0
- 合并冲突 0
8 次模型 · 5 次工具 · 4,655 tokens · 8.09s · 3 次控制器拒绝
这是一条正式配对记录，但单条案例仍不能代替 24 次 Pilot B 统计。
采购限流恢复 · 完整双泳道
在相同附件、工具、限流场景、模型与预算下完成采购决策并提交可验证证据。
冻结输入： requirements.md · vendors.csv · amendments.csv
共享条件： deepseek-v4-flash · read_file · list_resources · fetch_resource · write_output · wait_or_advance · finish ·
相同代码验收
开放式执行
| 步骤 | 工具动作 | 工具反馈 | 状态 / 证据变化 | 代码验收 | 累计使用 |
|---|---|---|---|---|---|
| 1 | 读取附件 { "path": "requirements.md", "tool": "read_file" } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 } | { "state": { "last_error": "none", "read_inputs": [ "requirements.md" ] }, "evidence": { "fact_count": 2, "source_added": "obs-001" } } | pending | 1 次模型 · 1 次工具 · 606 tokens · 0.67s |
| 2 | 读取附件 { "path": "vendors.csv", "tool": "read_file" } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 } | { "state": { "read_inputs": [ "requirements.md", "vendors.csv" ] }, "evidence": { "fact_count": 2, "source_added": "obs-002" } } | pending | 2 次模型 · 2 次工具 · 1,437 tokens · 1.47s |
| 3 | 读取附件 { "path": "amendments.csv", "tool": "read_file" } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 } | { "state": { "read_inputs": [ "amendments.csv", "requirements.md", "vendors.csv" ] }, "evidence": { "fact_count": 2, "source_added": "obs-003" } } | pending | 3 次模型 · 3 次工具 · 2,441 tokens · 2.21s |
| 4 | 列出远程资源 { "tool": "list_resources" } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 } | { "state": { "resources_discovered": true }, "evidence": { "fact_count": 2, "source_added": null } } | pending | 4 次模型 · 4 次工具 · 3,559 tokens · 2.91s |
| 5 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "rate_limited", "fact_count": 1, "ok": false, "retry_after_units": 1, "state_changed": false, "status_code": 429 } | { "state": { "last_error": "rate_limited" }, "evidence": { "fact_count": 1, "source_added": null } } | pending | 5 次模型 · 5 次工具 · 4,777 tokens · 3.65s |
| 6 | 等待后重试 { "tool": "wait_or_advance", "units": 1 } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "advanced_units": 1, "last_error": "none" }, "evidence": { "fact_count": 2, "source_added": null } } | pending | 6 次模型 · 6 次工具 · 6,094 tokens · 4.49s |
| 7 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 1 }, "evidence": { "fact_count": 5, "source_added": "obs-007" } } | pending | 7 次模型 · 7 次工具 · 7,514 tokens · 5.20s |
| 8 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 2 }, "evidence": { "fact_count": 5, "source_added": "obs-008" } } | pending | 8 次模型 · 8 次工具 · 9,057 tokens · 5.90s |
| 9 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 3 }, "evidence": { "fact_count": 5, "source_added": "obs-009" } } | pending | 9 次模型 · 9 次工具 · 10,723 tokens · 6.60s |
| 10 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 4 }, "evidence": { "fact_count": 5, "source_added": "obs-010" } } | pending | 10 次模型 · 10 次工具 · 12,512 tokens · 7.39s |
| 11 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 5 }, "evidence": { "fact_count": 5, "source_added": "obs-011" } } | pending | 11 次模型 · 11 次工具 · 14,424 tokens · 8.11s |
| 12 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 6 }, "evidence": { "fact_count": 5, "source_added": "obs-012" } } | pending | 12 次模型 · 12 次工具 · 16,459 tokens · 8.87s |
| 13 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 7 }, "evidence": { "fact_count": 5, "source_added": "obs-013" } } | pending | 13 次模型 · 13 次工具 · 18,617 tokens · 9.78s |
| 14 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 8 }, "evidence": { "fact_count": 5, "source_added": "obs-014" } } | pending | 14 次模型 · 14 次工具 · 20,898 tokens · 10.57s |
| 15 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 9 }, "evidence": { "fact_count": 5, "source_added": "obs-015" } } | pending | 15 次模型 · 15 次工具 · 23,302 tokens · 11.58s |
| 16 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 10 }, "evidence": { "fact_count": 5, "source_added": "obs-016" } } | failed | 16 次模型 · 16 次工具 · 25,829 tokens · 12.55s |
未通过代码验收
没有形成可验收输出
- 冻结输入完整 all protected fixture hashes match the run-start manifest
- 交付格式正确 decision.json is missing
- 供应商选择正确 未执行 · not evaluated because decision schema failed
- 数量与价格正确 未执行 · not evaluated because decision schema failed
- 淘汰理由完整 未执行 · not evaluated because decision schema failed
- 结论有工具证据 evidence/claims.json is missing
16 次模型 · 16 次工具 · 25,829 tokens · 12.56s
工作流受控执行
| 步骤 | 工具动作 | 工具反馈 | 状态 / 证据变化 | 代码验收 | 累计使用 |
|---|---|---|---|---|---|
| 1 | 读取附件 { "path": "requirements.md", "tool": "read_file" } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 } | { "state": { "last_error": "none", "read_inputs": [ "requirements.md" ] }, "evidence": { "fact_count": 2, "source_added": "obs-001" } } | pending | 1 次模型 · 1 次工具 · 725 tokens · 0.94s |
| 2 | 读取附件 { "path": "vendors.csv", "tool": "read_file" } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 } | { "state": { "read_inputs": [ "requirements.md", "vendors.csv" ] }, "evidence": { "fact_count": 2, "source_added": "obs-002" } } | pending | 2 次模型 · 2 次工具 · 1,599 tokens · 2.04s |
| 3 | 读取附件 { "path": "amendments.csv", "tool": "read_file" } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 } | { "state": { "read_inputs": [ "amendments.csv", "requirements.md", "vendors.csv" ] }, "evidence": { "fact_count": 2, "source_added": "obs-003" } } | pending | 3 次模型 · 3 次工具 · 2,569 tokens · 2.91s |
| 4 | 列出远程资源 { "tool": "list_resources" } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": false, "status_code": 200 } | { "state": { "resources_discovered": true }, "evidence": { "fact_count": 2, "source_added": null } } | pending | 4 次模型 · 4 次工具 · 3,579 tokens · 4.13s |
| 5 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "rate_limited", "fact_count": 1, "ok": false, "retry_after_units": 1, "state_changed": false, "status_code": 429 } | { "state": { "last_error": "rate_limited" }, "evidence": { "fact_count": 1, "source_added": null } } | pending | 5 次模型 · 5 次工具 · 4,623 tokens · 4.87s |
| 6 | 等待后重试 { "tool": "wait_or_advance", "units": 1 } | { "error_code": "none", "fact_count": 2, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "advanced_units": 1, "last_error": "none" }, "evidence": { "fact_count": 2, "source_added": null } } | pending | 6 次模型 · 6 次工具 · 5,682 tokens · 5.67s |
| 7 | 获取远程资源 { "resource": "live_quote", "token_present": false, "tool": "fetch_resource" } | { "error_code": "none", "fact_count": 5, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "evidence_sources": 1 }, "evidence": { "fact_count": 5, "source_added": "obs-007" } } | pending | 7 次模型 · 7 次工具 · 6,763 tokens · 6.59s |
| 8 | 写入交付物 { "artifact_bytes": 357, "artifact_sha256": "68a40d5abd3c4b1a88324b053743b4248955c75a41b3b050bffe47eff4012623", "path": "decision.json", "tool": "write_output" } | { "error_code": "none", "fact_count": 1, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "written_outputs": [ "decision.json" ] }, "evidence": { "fact_count": 1, "source_added": null } } | pending | 8 次模型 · 8 次工具 · 8,008 tokens · 8.10s |
| 9 | 写入交付物 { "artifact_bytes": 549, "artifact_sha256": "01010a7a38c7fceaa03d558a93cc68bee0430fa9dd14bd9d5233a2d311f5a913", "path": "evidence/claims.json", "tool": "write_output" } | { "error_code": "none", "fact_count": 1, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": 200 } | { "state": { "written_outputs": [ "decision.json", "evidence/claims.json" ] }, "evidence": { "fact_count": 1, "source_added": null } } | passed | 9 次模型 · 9 次工具 · 9,373 tokens · 10.47s |
代码验收通过
{
"currency": "USD",
"evidence_file": "evidence/claims.json",
"quantity": 100,
"rejections": [
{
"reason_code": "higher_authoritative_price",
"vendor_id": "atlas"
},
{
"reason_code": "not_approved",
"vendor_id": "harbor"
}
],
"schema_version": "1.0",
"total_price_usd": 9600,
"unit_price_usd": 96,
"vendor_id": "northstar",
"vendor_name": "Northstar Supplies"
}
- 冻结输入完整 all protected fixture hashes match the run-start manifest
- 交付格式正确 decision.json matches schema version 1.0
- 供应商选择正确 selected vendor identity and currency match derived Gold
- 数量与价格正确 quantity, unit price, and total match derived Gold
- 淘汰理由完整 all non-selected vendors have the derived reason
- 结论有工具证据 all claims resolve to successful tool-observation facts
9 次模型 · 9 次工具 · 9,373 tokens · 10.49s
替代路径 A-01 · 完整双泳道
主证据路径失败后，切换到独立副本并提交受事实 ID 支持的发布令牌。
冻结输入： tasks.json
共享条件： deepseek-v4-flash · list_evidence · read_evidence · submit_result · finish ·
相同代码验收
开放式执行
| 步骤 | 工具动作 | 工具反馈 | 状态 / 证据变化 | 代码验收 | 累计使用 |
|---|---|---|---|---|---|
| 1 | 列出证据 { "tool": "list_evidence" } | { "error_code": null, "fact_count": 0, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": {}, "evidence": { "fact_count": 0, "source_added": null } } | pending | 1 次模型 · 1 次工具 · 404 tokens · 1.03s |
| 2 | 读取证据 { "tool": "read_evidence" } | { "error_code": "route_unavailable", "fact_count": 0, "ok": false, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": { "last_error": "route_unavailable" }, "evidence": { "fact_count": 0, "source_added": null } } | pending | 2 次模型 · 2 次工具 · 893 tokens · 2.14s |
| 3 | 读取证据 { "tool": "read_evidence" } | { "error_code": null, "fact_count": 1, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": { "evidence_sources": 1, "last_error": null }, "evidence": { "fact_count": 1, "source_added": "fact-group-003" } } | pending | 3 次模型 · 3 次工具 · 1,437 tokens · 3.09s |
| 4 | 提交结果 { "tool": "submit_result" } | { "error_code": null, "fact_count": 0, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": { "written_outputs": [ "submitted_result" ] }, "evidence": { "fact_count": 0, "source_added": null } } | pending | 4 次模型 · 4 次工具 · 2,094 tokens · 4.19s |
| 5 | 结束任务 { "tool": "finish" } | { "error_code": null, "fact_count": 0, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": { "finished": true }, "evidence": { "fact_count": 0, "source_added": null } } | passed | 5 次模型 · 5 次工具 · 2,796 tokens · 5.02s |
代码验收通过
{
"release_token": "CIRRUS-41"
}
- 任务效用 100
- 关键事实覆盖 100%
- 错误证据 0
- 合并遗漏 0
- 合并冲突 0
5 次模型 · 5 次工具 · 2,796 tokens · 5.02s
目标图混合执行
| 步骤 | 工具动作 | 工具反馈 | 状态 / 证据变化 | 代码验收 | 累计使用 |
|---|---|---|---|---|---|
| 1 | 列出证据 { "tool": "list_evidence" } | { "error_code": null, "fact_count": 0, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": {}, "evidence": { "fact_count": 0, "source_added": null } } | pending | 1 次模型 · 1 次工具 · 535 tokens · 0.89s |
| 2 | 读取证据 { "tool": "read_evidence" } | { "error_code": "route_unavailable", "fact_count": 0, "ok": false, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": { "last_error": "route_unavailable" }, "evidence": { "fact_count": 0, "source_added": null } } | pending | 2 次模型 · 2 次工具 · 1,071 tokens · 1.95s |
| 3 | 读取证据 { "tool": "read_evidence" } | { "error_code": null, "fact_count": 1, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": { "evidence_sources": 1, "last_error": null }, "evidence": { "fact_count": 1, "source_added": "fact-group-003" } } | pending | 3 次模型 · 3 次工具 · 1,615 tokens · 2.74s |
| 4 | 提交结果 { "tool": "submit_result" } | { "error_code": null, "fact_count": 0, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": { "written_outputs": [ "submitted_result" ] }, "evidence": { "fact_count": 0, "source_added": null } } | pending | 4 次模型 · 4 次工具 · 2,213 tokens · 3.68s |
| 5 | 提交结果 { "tool": "submit_result" } | { "error_code": "not_in_legal_frontier", "fact_count": 0, "ok": false, "retry_after_units": null, "state_changed": false, "status_code": null } | { "state": { "last_error": "not_in_legal_frontier" }, "evidence": { "fact_count": 0, "source_added": null } } | pending | 5 次模型 · 4 次工具 · 2,800 tokens · 4.91s |
| 6 | 提交结果 { "tool": "submit_result" } | { "error_code": "not_in_legal_frontier", "fact_count": 0, "ok": false, "retry_after_units": null, "state_changed": false, "status_code": null } | { "state": {}, "evidence": { "fact_count": 0, "source_added": null } } | pending | 6 次模型 · 4 次工具 · 3,409 tokens · 6.06s |
| 7 | 提交结果 { "tool": "submit_result" } | { "error_code": "not_in_legal_frontier", "fact_count": 0, "ok": false, "retry_after_units": null, "state_changed": false, "status_code": null } | { "state": {}, "evidence": { "fact_count": 0, "source_added": null } } | pending | 7 次模型 · 4 次工具 · 4,036 tokens · 7.20s |
| 8 | 结束任务 { "tool": "finish" } | { "error_code": null, "fact_count": 0, "ok": true, "retry_after_units": null, "state_changed": true, "status_code": null } | { "state": { "finished": true, "last_error": null }, "evidence": { "fact_count": 0, "source_added": null } } | passed | 8 次模型 · 5 次工具 · 4,655 tokens · 8.09s |
代码验收通过
{
"release_token": "CIRRUS-41"
}
- 任务效用 100
- 关键事实覆盖 100%
- 错误证据 0
- 合并遗漏 0
- 合并冲突 0
8 次模型 · 5 次工具 · 4,655 tokens · 8.09s
04 · From mechanism to test
不是先做题再解释，而是先把架构差异写成测试题
每个测试场景都对应一个明确的控制差异；验收器只看最终结果，不检查预设路线。
| 架构差异 | 控制变量 | 待检验问题 | 对应实验 |
|---|---|---|---|
| 谁维护状态 | 模型上下文 / 程序状态 | 前置条件与停止是否更稳定 | Pilot A |
| 合法动作范围 | 全工具 / 动态开放 | 无效调用和错误恢复是否减少 | Pilot A + B |
| 路径选择 | 线性探索 / 目标依赖图 | 替代路径与开放取证是否受益 | Pilot B |
| 并行与合并 | 单一循环 / 有界分支 | 关键路径缩短后能否保住合并质量 | Pilot B |
比较什么每项控制差异由哪个场景检验。
怎样读按行追踪“差异—假设—证据”。
不能推出未覆盖的模型和真实业务仍需另测。
05 · Pilot A · formal experiment
规则明确的任务里，状态控制改变了这组实验的完成结果
5 个场景 × 3 次重复 × 2 种架构。任务、工具、预算和代码验收全部冻结。
开放式 ReAct
0/15
执行记录显示，它把本地文件误当成远程资源，之后没有稳定进入前置条件恢复和成功停止。
工作流受控执行
15/15
程序保存状态、开放合法动作、按错误类型恢复，并在代码验收通过后立即停止。
比较什么15 对冻结运行的成功状态与调用开销。
怎样读先看代码验收，再看不同场景的执行成本。
不能推出不能证明所有 ReAct 系统都会失败。
06 · Pilot B · formal experiment
开放任务没有赢家，但暴露了混合架构真正需要调的地方
3 类任务 × 每类 4 个冻结实例 × 2 种架构，共 24 次配对运行。正式运行前的 6 次 smoke 只用于确认协议与记录字段。
- 模型
- deepseek-v4-flash
- 模型调用
- ≤ 18
- 工具调用
- ≤ 24
- Tokens
- ≤ 30,000
- 并行分支
- ≤ 3
- 超时
- 240s
Contracted ReAct
8/12 成功 · 平均 3709 tokens=
Goal-Graph Hybrid
8/12 成功 · 平均 12288 tokens替代路径
开放式 4/4
目标图混合 4/4
开放取证
开放式 4/4
目标图混合 4/4
并行拆解
开放式 0/4
目标图混合 0/4
两边都找全事实，但合并时各遗漏 3 项。
比较什么成功、质量和成本的配对结果。
怎样读结果相同后，再看成本与失败位置。
不能推出不能把较短关键路径等同于更高成功率。
07 · Business decision
从任务结构出发，而不是从架构名出发
下面的选择器给出工程起点，不替代上线前的业务回放测试。
比较什么任务条件与架构起点的对应关系。
怎样读先判断规则和风险，再判断路径与并行。
不能推出选择结果不是免测的上线结论。
08 · Limits & reproducibility
这份证据能说明什么，也明确不说明什么
当前数据支持
- 规则任务中，状态和动作控制能改变最终完成状态。
- 开放任务中，复杂控制器可能降低工具调用，却增加模型调用与时间。
- 并行执行与正确合并是两件不同的事，需要分别验收。
当前数据不支持
- 任何商业产品的内部架构还原或实时排名。
- 某种架构在所有模型、业务和工具上普遍更强。
- 仅凭 15 或 12 个配对样本做大范围统计外推。
公开材料
只包含脱敏运行记录、统计、代表轨迹与哈希；不包含 API Key、绝对路径和供应商原始响应。
Pilot A batch: 2026-07-20-deepseek-v4-flash-a · Pilot B batch: 2026-07-22-pilot-b-deepseek-v4-flash-a