Agent 评估、测试集与可观测性：从最终答案到轨迹级归因

1
使用跨租户数据得到正确答案
2
重复执行退款后生成正确话术
3
引用旧规则但结论碰巧相同
4
人工修正了工具参数但未记录

1
先查订单再查物流
2
先并行查订单/物流再汇合

1
from typing import Literal
2
from pydantic import BaseModel
3

4

5
class AgentEvalCase(BaseModel):
6
    case_id: str
7
    dataset_split: str
8
    scenario_family: str
9
    risk_level: Literal["low", "medium", "high", "critical"]
10
    input: dict
11
    identity: dict
12
    initial_environment_ref: str
13
    pinned_versions: dict
14
    expected: dict
15
    allowed_trajectory: dict
16
    forbidden: dict
17
    graders: list[str]
18
    tags: list[str]
19
    source_provenance: dict

1
{
2
  "intent": "refund_request",
3
  "required_fact_refs": ["order", "logistics"],
4
  "required_policy_ids": ["R001"],
5
  "decision": "refund_eligible",
6
  "terminal_state": "awaiting_confirmation",
7
  "environment_assertions": {
8
    "refund_count": 0,
9
    "order_status": "paid"
10
  }
11
}

1
{
2
  "tools": ["create_compensation"],
3
  "data_namespaces": ["tenant/other"],
4
  "claims": ["退款已完成"],
5
  "side_effects": ["refund_created"]
6
}

1
agent/workflow
2
model/adapter
3
prompt
4
tool registry
5
policy
6
RAG index
7
memory snapshot
8
evaluator/grader
9
environment fixture

1
Intent Accuracy / Macro-F1
2
Slot Precision/Recall/F1
3
Clarification Precision/Recall
4
Multi-intent Coverage

1
Required Context Recall
2
Context Precision
3
Stale Context Rate
4
Unauthorized Context Rate
5
Memory Write Accuracy
6
Conflict/Deletion Accuracy

1
Recall@K / MRR / nDCG
2
Required Evidence Recall
3
Effective-version Accuracy
4
Citation Precision/Recall
5
Claim Entailment
6
Evidence Sufficiency

1
Plan Validity
2
Dependency Accuracy
3
Required-step Recall
4
Forbidden-step Rate
5
No-progress Loop Rate
6
Step Efficiency
7
Repair Success

1
Need-tool Accuracy
2
Tool Selection Accuracy
3
Argument Field/Exact Accuracy
4
No-tool Accuracy
5
Tool Success
6
Idempotency Violation
7
Final-state Verification

1
Responsibility Accuracy
2
Action Accuracy
3
Policy Compliance
4
Unsafe Action Rate
5
Unauthorized Data Exposure
6
Human Gate Precision/Recall

1
Schema Validity
2
Faithfulness
3
Helpfulness
4
Citation Correctness
5
Sensitive Data Leakage
6
Overclaim Rate

1
Verified Task Success
2
Resolution Rate
3
Human Escalation
4
p50/p95/p99 Latency
5
Tokens/Cost per Verified Success
6
Retry/Fallback/Cancel Rate
7
User Correction/Satisfaction

1
Tool Success Rate = succeeded tool calls / attempted tool calls
2
Tool Selection Accuracy = correct next-action cases / eligible cases
3
Verified Task Success = verified successful runs / all eligible runs
4
Human Escalation Precision = necessary escalations / all escalations

1
只在成功解析输出上算准确率
2
排除 timeout/cancel 后算任务成功
3
只在模型决定调用工具的 Case 上算 tool accuracy
4
把人工修正后的结果算模型成功

1
total model + retrieval + tool + infra + human review cost
2
------------------------------------------------------------
3
verified successful tasks

1
Micro：按 Case 数量，容易被高频简单场景主导
2
Macro：每 scenario family 等权
3
Risk-weighted：高影响 Case 权重更高

1
query_order < create_refund
2
search_policy < decision
3
confirmation < create_refund
4
create_refund < verify_final_state

1
class TrajectoryExpectation(BaseModel):
2
    required_nodes: list[str]
3
    optional_nodes: list[str]
4
    forbidden_nodes: list[str]
5
    precedences: list[tuple[str, str]]
6
    max_node_visits: dict[str, int]
7
    max_steps: int
8
    required_gates: list[str]

1
任何写工具前必须 Authz/Policy
2
参数变化后确认失效
3
跨租户读次数必须为 0
4
最终成功必须有环境验证

1
def grade_precedence(events, before, after) -> bool:
2
    before_positions = [e.sequence for e in events if e.node == before]
3
    after_positions = [e.sequence for e in events if e.node == after]
4
    return bool(before_positions and after_positions) and max(before_positions) < min(after_positions)

1
steps above shortest acceptable path
2
duplicate tool calls
3
repeated queries
4
context/token per completed requirement

1
Prompt Injection
2
跨租户/权限
3
工具误用
4
Memory/RAG 投毒
5
重复副作用
6
资源耗尽

1
dataset: orderflow-agent-eval@2026-07-15.3
2
case_count: 1280
3
splits:
4
  golden: 420
5
  adversarial: 180
6
  regression: 260
7
  long_tail: 300
8
  online_holdout: 120
9
schema_version: "4"
10
label_policy: orderflow-rubric@7
11
source_cutoff: 2026-06-30
12
content_digest: sha256:...

1
同一 thread/order/user 跨 split
2
同一模板只替换 ID
3
线上 Regression 同时进入训练与 final test
4
Judge calibration Case 被写进 rubric 示例
5
公开 Benchmark 答案被模型记忆

1
draft
2
-> reviewed
3
-> active
4
-> flaky/quarantined
5
-> superseded
6
-> retired

1
rows: scenario/risk/failure type
2
columns: normal/boundary/error/adversarial/recovery
3
cells: case count + last pass + owner

1
业务规则/决策表
2
历史工单
3
线上失败
4
Support/SME 访谈
5
Threat Model
6
边界值/状态机
7
合成扩展

1
数据库 fixture
2
确定性规则引擎
3
人工政策 Owner
4
可执行测试
5
多 Reviewer adjudication

1
answer/decision
2
required facts/evidence
3
allowed/forbidden actions
4
acceptable trajectories
5
human escalation
6
severity/impact

1
multiple acceptable outcomes
2
requires clarification
3
requires human
4
insufficient policy

1
hard gates: safety/authz/environment/schema
2
soft scores: helpfulness/style/efficiency

1
class GraderResult(BaseModel):
2
    grader_id: str
3
    grader_version: str
4
    verdict: Literal["pass", "fail", "unknown", "error"]
5
    score: float | None
6
    reason_codes: list[str]
7
    evidence_refs: list[str]
8
    severity: str | None
9
    latency_ms: int
10
    cost_usd: float

1
1. Grader contract/error check
2
2. Critical hard gates
3
3. Required environment/trajectory checks
4
4. Soft quality scores
5
5. Cost/latency budgets
6
6. Human adjudication band

1
known pass/fail/unknown fixtures
2
边界值
3
缺失 Trace/证据
4
乱序/重复事件
5
adversarial text
6
旧 Schema migration

1
class JudgeRubric(BaseModel):
2
    rubric_id: str
3
    version: str
4
    criteria: list[dict]
5
    evidence_requirements: list[str]
6
    allowed_labels: list[str]
7
    abstain_label: str
8
    examples: list[dict]

1
Position bias
2
Verbosity/length bias
3
Style bias
4
Self/model-family preference
5
Reference-answer anchoring
6
Prompt injection in candidate

1
clear pass/fail
2
subtle unsupported claim
3
correct but concise/verbose pair
4
same answer swapped order
5
adversarial self-praise
6
insufficient evidence/abstain
7
unsafe trajectory with good final text

1
HTTP request
2
└── agent.run
3
    ├── context.build
4
    ├── model.route
5
    ├── model.call
6
    ├── rag.retrieve
7
    ├── tool.query_order
8
    ├── policy.evaluate
9
    ├── human.wait
10
    ├── tool.create_refund
11
    └── state.verify

1
Run：完整任务
2
Span：一段有开始/结束的操作
3
Event：Span/Run 中的离散变化
4
Link：异步/跨 Agent 因果关联

1
run/thread/task/tenant refs
2
agent/workflow/node version
3
model/provider/route
4
prompt version
5
tool/server/version
6
policy/index/memory snapshot
7
attempt/error/retry
8
token/cost/latency
9
state before/after hash

1
API Key/token/cookie
2
完整 PII
3
支付凭据
4
受限文档全文
5
隐藏推理
6
未授权跨租户内容

1
redaction policy/version
2
content hash
3
secure source reference
4
data classification
5
truncated flag

1
100% high-risk action/audit link
2
100% errors/security events（按数据策略）
3
adaptive sample slow/high-cost
4
low-rate normal success

1
producer span -> message/task id -> consumer span link
2
human wait span/event -> resume span
3
remote task -> artifact id -> local verify span

1
error/security
2
high latency/cost
3
fallback/retry
4
human override
5
new version
6
rare scenario

1
SDK/Runtime events
2
-> local batch/queue
3
-> redaction/classification
4
-> OTel collector
5
-> trace/log/metric backends
6
-> eval/failure consumers

1
% runs with terminal event
2
% model calls with usage
3
% tool calls with proposal/auth/result
4
% writes with final-state verification
5
% high-risk actions linked to audit/confirmation

1
Prompt/Model
2
Context/Retriever
3
Tool/Adapter
4
Workflow/Runtime
5
Policy/Data
6
Evaluator/Test

1
1. 环境终态错？
2
2. 找 first divergent invariant
3
3. 对应 Span/State diff
4
4. 检查上游必要输入是否存在
5
5. 用 Oracle 替换怀疑模块重放
6
6. 最小化 Case
7
7. 分类 root cause 与 contributing factors
8
8. 修复后跑局部 + 全量回归

1
Oracle Context + original Planner
2
original Context + Oracle Planner
3
Oracle Tool Result + original Decision
4
original Trace + Oracle Verifier

1
{
2
  "node": "verify_eligibility",
3
  "expected": {"policy_ids": ["R001"], "product_type": "normal"},
4
  "actual": {"policy_ids": ["R001"], "product_type": null},
5
  "first_missing_field": "product_type",
6
  "source_node": "query_order_adapter"
7
}

1
scenario
2
risk
3
language
4
tool
5
error type
6
tenant class
7
sequence length

1
核心成功率不得降
2
critical safety 必须 100% pass
3
p95 latency/cost 有预算
4
长尾可接受 trade-off 需 Owner 批准

1
hard:
2
  critical_safety_pass: 1.0
3
  unauthorized_data_exposure: 0
4
  duplicate_side_effect: 0
5
  schema_validity_gte: 0.999
6
soft:
7
  task_success_delta_gte: -0.005
8
  cost_per_success_delta_lte: 0.10
9
  p95_latency_delta_lte: 0.10

1
PR fast smoke
2
merge full golden/regression
3
nightly multi-seed/adversarial/OOD
4
release shadow/canary

1
random baseline
2
100% errors/security/high-risk
3
uncertainty/disagreement
4
new model/tool/policy version
5
long/high-cost traces

1
explicit user correction
2
human override
3
tool/environment outcome
4
support ticket
5
model judge（辅助）

1
按用户/租户稳定分桶，避免一次会话跨版本
2
记录 eligibility/exposure
3
避免共享缓存/人工 Reviewer 污染
4
预定义主/护栏指标和停止规则

1
critical safety immediately
2
model/judge disagreement
3
unknown/new failure
4
random quality sample

1
理解能力维度
2
复用环境/指标思想
3
做模型/框架初筛
4
发现测试盲区

1
你的业务规则正确
2
你的权限安全
3
你的工具稳定
4
你的成本/SLO 可接受
5
你的数据分布表现

1
Benchmark capability -> 业务能力假设 -> 本地 Case -> 环境成功条件