Planning、Reflection 与 Verifier：规划、自我修正与验证闭环

1
步骤数量
2
依赖关系
3
环境不确定性
4
工具动态性
5
失败成本
6
可逆性
7
延迟预算
8
是否有明确验证器

1
Autonomy =
2
  allowed action set
3
  + max steps/time/cost
4
  + side-effect ceiling
5
  + data access scope
6
  + human escalation rule

1
from typing import Literal
2
from pydantic import BaseModel, Field
3

4

5
class PlanStep(BaseModel):
6
    step_id: str
7
    objective: str
8
    capability: str
9
    dependencies: list[str]
10
    required_facts: list[str]
11
    preconditions: list[str]
12
    success_conditions: list[str]
13
    failure_routes: dict[str, str]
14
    side_effect: Literal["none", "read", "write", "external"]
15
    reversible: bool
16
    max_attempts: int = Field(ge=1, le=3)
17
    timeout_seconds: float
18
    estimated_cost: float | None
19

20

21
class Plan(BaseModel):
22
    plan_id: str
23
    goal: str
24
    assumptions: list[str]
25
    steps: list[PlanStep]
26
    terminal_conditions: list[str]
27
    replan_conditions: list[str]
28
    plan_version: int

1
Complete：覆盖目标必要条件
2
Grounded：只引用可用能力和已知事实
3
Ordered：依赖与副作用顺序正确
4
Minimal：无无关步骤
5
Verifiable：每步有外部成功条件
6
Recoverable：失败有明确路由
7
Budgeted：时间/成本/次数有限
8
Policy-compliant：计划本身不包含越权动作

1
decision code
2
selected capability
3
assumptions
4
evidence refs
5
rejected alternatives（简短结构化）

1
先生成整体计划
2
-> 按顺序执行
3
-> 每步更新状态
4
-> 必要时重规划

1
只详细规划接下来 1-3 步
2
远期步骤保持粗粒度
3
新事实到达后继续展开

1
Goal
2
-> Milestones
3
-> Subtasks
4
-> Executable Actions

1
query_order ───┐
2
               ├-> decide_eligibility
3
query_logistics┘
4
search_policy ─┘

1
计划引用的工具是否存在？
2
当前主体是否可用？
3
参数能否从已有/可获取事实得到？
4
工具是否支持当前地域/租户？

1
订单当前 version 是多少？
2
前置步骤是否真的完成？
3
人工确认是否仍有效？
4
业务状态是否在等待期间变化？

1
class Assumption(BaseModel):
2
    assumption_id: str
3
    statement: str
4
    source: Literal["user", "tool", "policy", "model"]
5
    evidence_refs: list[str]
6
    status: Literal["unverified", "verified", "invalidated"]
7
    invalidates_steps: list[str]

1
step_id 唯一
2
依赖存在
3
DAG 无意外环
4
至少一个终止路径
5
所有失败码有路由

1
capability 在当前 Registry Snapshot
2
输入字段有来源
3
副作用与计划声明一致
4
timeout/max_attempts 合法

1
主体/资源权限
2
高风险动作前有确认/HITL
3
不可逆动作在必要读取之后
4
敏感数据不流向未授权工具

1
class PlanValidation(BaseModel):
2
    valid: bool
3
    errors: list[str]
4
    warnings: list[str]
5
    unreachable_steps: list[str]
6
    missing_verifiers: list[str]
7
    policy_version: str

1
class StepAttempt(BaseModel):
2
    attempt_id: str
3
    step_id: str
4
    plan_version: int
5
    started_at: str
6
    input_fact_refs: list[str]
7
    capability_snapshot: str
8
    status: Literal[
9
        "running",
10
        "succeeded",
11
        "failed_retryable",
12
        "failed_terminal",
13
        "outcome_unknown",
14
        "cancelled",
15
    ]
16
    observation_refs: list[str]
17
    error_code: str | None

1
Expected Success
2
Expected Business Rejection
3
Transient Technical Failure
4
Permanent Technical Failure
5
Unexpected State
6
Outcome Unknown
7
Security/Policy Denial

1
new verified facts
2
new satisfied preconditions
3
completed milestone
4
reduced uncertainty
5
external state transition

1
{
2
  "step_id": "query_logistics",
3
  "attempt": 2,
4
  "input_versions": {"order": 8},
5
  "observation": "logistics://ORD-A1B2C3D4?version=3",
6
  "progress": ["fact.logistics_status"],
7
  "cost_usd": 0.002,
8
  "latency_ms": 184
9
}

1
前置假设被推翻
2
工具不可用/权限改变
3
业务状态冲突
4
新必需事实出现
5
Verifier 拒绝结果
6
计划超预算
7
用户改变目标

1
class PlanRepairRequest(BaseModel):
2
    goal: str
3
    current_plan: Plan
4
    completed_steps: list[str]
5
    durable_fact_refs: list[str]
6
    external_effect_refs: list[str]
7
    failed_step: str
8
    failure_code: str
9
    invalidated_assumptions: list[str]
10
    remaining_budget: dict

1
KEEP 已完成且仍有效
2
INVALIDATE 依赖错误假设
3
REPLACE 失败步骤
4
INSERT 新查询/确认/补偿
5
REORDER 尚未执行步骤
6
STOP 转人工或终止

1
外部 operation_id
2
幂等记录
3
业务最终状态
4
可补偿性

1
plan v1: steps A B C D
2
step A/B complete
3
step C failure
4
plan v2: KEEP A/B, REPLACE C with C1/C2, KEEP D dependency updated

1
暂停调度旧计划新 Step
2
-> 标记未执行步骤 cancelled_by_goal_change
3
-> 对已执行副作用列出状态/补偿可能
4
-> 重新确认新目标和约束
5
-> 生成新 goal_id + plan_id
6
-> 保留两者因果关联

1
class GoalRevision(BaseModel):
2
    previous_goal_id: str
3
    new_goal_id: str
4
    user_message_ref: str
5
    kept_effects: list[str]
6
    compensation_candidates: list[str]
7
    cancelled_step_ids: list[str]
8
    confirmed_at: str

1
{
2
  "from_version": 1,
3
  "to_version": 2,
4
  "kept": ["S1"],
5
  "invalidated": ["S2"],
6
  "replaced": {"S2": ["S2a", "S2b"]},
7
  "inserted": ["S2c"],
8
  "reason_refs": ["error://logistics_record_not_found"],
9
  "external_effects_preserved": []
10
}

1
class ReflectionResult(BaseModel):
2
    failure_category: str
3
    failed_step_ids: list[str]
4
    evidence_refs: list[str]
5
    root_cause_hypotheses: list[str]
6
    confidence: float
7
    proposed_repairs: list[str]
8
    reusable_lesson_candidate: str | None
9
    requires_human_review: bool

1
计划
2
实际 Observation
3
工具错误码
4
Verifier 结果
5
状态 diff
6
预算

1
候选经验
2
-> 多 Case 验证
3
-> 人工/离线评估
4
-> 作用域与有效期
5
-> 才能提升为 procedural memory/skill change

1
代码/报告等高价值产物
2
有明确 verifier
3
失败可修复
4
剩余预算充足

1
简单 FAQ
2
已明确权限拒绝
3
不可逆动作之后的盲重试
4
无新证据的重复自评

1
报告结构
2
代码可读性
3
解释是否清楚
4
遗漏风险

1
Schema 是否合法
2
测试是否通过
3
数据库状态是否变为 refunded
4
金额是否不超过上限
5
每个 claim 是否有证据支持

1
L0 Static / Schema
2
L1 Deterministic Rule
3
L2 Environment / Tool Check
4
L3 Model-based Judge
5
L4 Human Review

1
class VerificationResult(BaseModel):
2
    verdict: Literal["pass", "fail", "unknown"]
3
    check_id: str
4
    verifier_version: str
5
    claim: str
6
    evidence_refs: list[str]
7
    failed_constraints: list[str]
8
    confidence: float | None
9
    repairable: bool
10
    recommended_route: str

1
确定性检查优先
2
Verifier 只看必要证据，不看生成者自辩
3
使用不同 rubric/prompt/model（按风险）
4
对 Judge 做人工校准
5
环境状态优先于语言评价

1
明确 pass
2
明确 fail
3
证据不足 unknown
4
表面合理但引用不支持
5
最终答案对但轨迹越权
6
措辞不同但语义正确
7
对抗性自辩/奖励劫持

1
               Gold Pass  Gold Fail  Gold Unknown
2
Pred Pass
3
Pred Fail
4
Pred Unknown

1
class VerifierPolicy(BaseModel):
2
    verifier_id: str
3
    version: str
4
    pass_threshold: float
5
    human_review_band: tuple[float, float]
6
    max_rechecks: int
7
    fail_open: bool = False

1
old/new confusion matrix
2
verdict distribution shift
3
false-pass regression
4
repair/human traffic delta
5
cost and latency delta

1
class RetryContext(BaseModel):
2
    error_code: str
3
    attempt: int
4
    max_attempts: int
5
    side_effect: str
6
    outcome_known: bool
7
    idempotency_supported: bool
8
    remaining_time_seconds: float
9
    remaining_cost_usd: float

1
delay = min(cap, base * 2^attempt) + random_jitter

1
原动作
2
补偿动作
3
补偿前置条件
4
补偿权限
5
补偿失败路由

1
高风险不可逆动作
2
规则/证据冲突
3
Verifier unknown
4
超过自动金额阈值
5
多次 repair 失败
6
用户明确要求人工

1
{
2
  "run_id": "run_01J...",
3
  "goal": "处理未发货退款",
4
  "plan_version": 2,
5
  "completed_steps": ["query_order", "query_logistics", "search_policy"],
6
  "verified_facts": ["order://...@v8", "logistics://...@v3"],
7
  "evidence_bundle": "evb://...",
8
  "proposed_action": "refund 12900 cents",
9
  "verifier_result": "unknown: conflicting presale metadata",
10
  "allowed_decisions": ["approve", "reject", "request_more_info"],
11
  "expires_at": "2026-07-15T11:00:00Z"
12
}

1
验证 review token
2
检查决定未过期
3
刷新外部业务状态
4
重新运行必要 Policy/Verifier
5
确认参数与批准内容一致

1
{
2
  "goal": "判断并处理订单退款",
3
  "steps": [
4
    {"id": "S1", "capability": "query_order", "depends_on": []},
5
    {"id": "S2", "capability": "query_logistics", "depends_on": ["S1"]},
6
    {"id": "S3", "capability": "search_policy", "depends_on": ["S1", "S2"]},
7
    {"id": "S4", "capability": "verify_eligibility", "depends_on": ["S3"]},
8
    {"id": "S5", "capability": "request_confirmation", "depends_on": ["S4"]},
9
    {"id": "S6", "capability": "create_refund", "depends_on": ["S5"]},
10
    {"id": "S7", "capability": "verify_final_state", "depends_on": ["S6"]}
11
  ]
12
}

1
{
2
  "status": "failed_terminal",
3
  "code": "logistics_record_not_found",
4
  "order_version": 8
5
}

1
{
2
  "failure_category": "missing_external_record",
3
  "failed_step_ids": ["S2"],
4
  "root_cause_hypotheses": [
5
    "订单尚未生成物流单，not-found 可能等价于 not-shipped"
6
  ],
7
  "proposed_repairs": [
8
    "查询订单 fulfillment_type 和 promised_ship_at",
9
    "使用 no-logistics-record 规则分支"
10
  ]
11
}

1
KEEP S1
2
REPLACE S2 with:
3
  S2a query_fulfillment_details
4
  S2b classify_no_logistics_record
5
UPDATE S3 required facts
6
KEEP S4-S7 with dependencies updated

1
{
2
  "verdict": "pass",
3
  "check_id": "refund_eligibility_v4",
4
  "evidence_refs": [
5
    "order://ORD-A1B2C3D4?version=8",
6
    "policy://refund/v18#R001"
7
  ],
8
  "failed_constraints": [],
9
  "repairable": false,
10
  "recommended_route": "request_confirmation"
11
}

1
create_refund -> accepted(operation_id=OP-88)
2
query_refund(OP-88) -> completed(refund_id=RF-77)
3
query_order -> status=refunded, version=9

1
Plan Validity
2
Capability Grounding Accuracy
3
Dependency Accuracy
4
Required-step Recall
5
Unnecessary-step Rate
6
Precondition Coverage
7
Expected Cost/Time Error

1
Step Success Rate
2
Progress per Step
3
No-progress Loop Rate
4
Repair Success Rate
5
Completed-work Reuse Rate
6
Duplicate Side-effect Rate
7
Replan Count per Success

1
Failure Attribution Accuracy
2
Useful Repair Suggestion Rate
3
Reflection-induced Regression Rate
4
Cost/Latency Delta
5
Net Task Success Lift

1
Verifier Precision/Recall
2
False-pass Rate
3
False-fail Rate
4
Unknown Calibration
5
Evidence Completeness
6
Inter-rater Agreement

1
A fixed workflow baseline
2
B planner only
3
C planner + repair
4
D planner + repair + verifier
5
E D + reflection
6
F E + human gate

1
Verified Task Success
2
Unsafe Action Rate
3
Latency
4
Cost per Verified Success
5
Human Escalation

1
{
2
  "run_id": "run_01J...",
3
  "goal_id": "goal_v1",
4
  "plan_versions": [1, 2],
5
  "planner": "planner@6",
6
  "capability_snapshot": "tools@42",
7
  "observations": ["obs://S1-A1", "obs://S2-A1"],
8
  "plan_diffs": ["diff://plan-1-2"],
9
  "reflections": ["reflection://R1"],
10
  "verifiers": ["refund-rule@4", "final-state@3"],
11
  "terminal_verdict": "pass",
12
  "verified_outcome_ref": "refund://RF-77"
13
}

1
Kept Valid Step Ratio
2
Unnecessarily Invalidated Step Rate
3
Repeated External Call Rate
4
Plan Edit Distance
5
New Failure Introduced Rate

1
未知工具
2
缺依赖
3
循环依赖
4
写操作早于确认
5
无终止条件
6
预算超限
7
不可逆动作无 verifier

1
只读工具瞬时失败
2
写工具 outcome_unknown
3
业务版本冲突
4
计划中途权限撤销
5
用户改变目标
6
并行 Observation 版本不一致

1
正确结果/错误结果
2
证据不足
3
冲突证据
4
对抗性自辩文本
5
相同模型偏差
6
Verifier 超时

1
Reflection 前进程死亡
2
新 Plan 保存后旧 Worker 继续执行
3
人工批准同时计划被修复
4
补偿动作失败
5
旧计划 Step 被重复投递