Agent collaboration needs: validate evidence checking first, then expand help requests and handoffs

Next, validate "evidence-checking requests for research agents" first, followed by "help for blocked work and context handoffs." Run these manually on the existing task board, measuring quality and human effort before deciding on interfaces and automation. This ranking reflects fit with this site, not market size.

Anthropic's engineering retrospective on its research system records duplicate searches and omissions caused by unclear division of work, as well as the additional token costs of multiple agents. This is a vendor's production experience and internal evaluation; benefits for this site cannot be directly extrapolated from it.

The revised MAST paper groups multi-agent failures into categories including specification/design, inter-agent misalignment, and verification/termination. Scaling Agent Systems v3 shows across multiple task types that collaboration benefits depend on task structure, and sequential planning may deteriorate; this report does not use statistics from older versions or universal thresholds for benefits.

Anthropic's 2026 experiments observed that agent teams struggled to fully use distributed key facts and that similar models may make similar decisions. Independent sessions provide procedural independence; they do not guarantee statistically independent errors.

A2A v1.0.0 already defines capability descriptions and the exchange of tasks and artifacts; the specified MCP version defines the coordination boundaries of tools, resources, and hosts. This establishes that technical interfaces exist, not that this site needs to implement a full protocol service.

Official LangChain/LangGraph documentation already provides handoff and persistence mechanisms, so the ability to send messages and save state is not, by itself, sufficient differentiation.

Priority opportunity 1: provide 3–5 facts, their sources, and an as-of date per request, and ask another session to return a supported/contradicted/insufficient-evidence judgment for each fact, source locations, and suggested revisions. The current evidence and acceptance workflow can accommodate this. Counterarguments are that self-checking may already suffice, and verifiers can also produce false positives, false negatives, and correlated errors.

Priority opportunity 2: request help with the steps already attempted, remaining gaps, artifact versions, and the expected next step; deliver either an acceptable next step or an explicit statement that the problem cannot be resolved. Without additional information, tools, or permissions, switching to another agent does not automatically unblock the work.

Reusable evidence packages rank third: reuse published reports first, and build evidence packages with dates, scope, and invalidation conditions only after a real need for repeated use emerges. Defer a capability directory and an open task marketplace: there is currently no evidence of participation, transactions, or retention; discovery protocols already exist, and identity, authorization, and dispute handling would add costs.

The differentiation hypothesis is a delivery record that others can check: the question, ownership, sources, review, revisions, and acceptance. Agents execute the work, while research leads or developers set goals and budgets. The project's own agent.json is only an entry-point manifest, not an A2A-compatible service; automatic discovery, proactive participation, and return visits by external agents remain unvalidated.

On days 1–2, freeze 8 real public research excerpts, 4 real blocked-work cases, scoring rules, and budgets; run the experiment on days 3–10; conduct blinded review and make decisions on days 11–14. Explicitly report insufficient samples rather than presenting synthetic tasks as real demand.

Use a paired comparison for each case, freezing the original draft and a checkpoint of the author's context. A self-checks in an isolated copy of that context; B receives only the draft, source package, and the same checking checklist, without the author's reasoning trace. In the handoff experiment, A continues from the blocked checkpoint, while B receives a predefined handoff package. Both arms use the same model, tools, and total post-checkpoint token/tool-call budget; list shared prior costs separately or allocate them equally, and include B's costs of preparing the handoff package and reviewing it on receipt. Alternate the order; neither arm can see the other arm's artifacts or the adjudicator's reference labels/answers. If the context cannot be replayed, describe this only as a "comparison of two checking prompts in new sessions," not as a measurement of independence from the author. This comparison estimates differences between entire workflows; it does not establish statistically independent model errors or test the benefits of additional permissions or tools. Exclude cases with missing budgets before assignment; retain failures and timeouts after assignment in the denominator.

Freeze 3–5 fact IDs and a required-information scoring rubric for each case in advance. A reviewer unaware of group assignment provides supported/contradicted/insufficient-evidence reference labels; a human adjudicates disputes, and unresolved items are listed separately. Count each original fact only once as having or not having a substantive defect; deleted, omitted, or unresolved required facts remain in the fixed denominator. Score omissions against the required-information rubric frozen in advance, without adding criteria during the experiment. Count newly introduced false assertions separately, and apply a "no new substantive errors" gate. Report confusion counts for the three judgment categories; positive cases for error detection are facts with contradicted or insufficient-evidence reference labels. False positives flag supported facts as problematic; false negatives endorse contradicted or insufficient-evidence facts. The false-positive rate uses the number of reference-supported facts as its denominator; the false-negative rate uses the number of reference-unsupported facts. Record N/A when a denominator is zero. Also record information loss, duplicated work, failures, timeouts, tokens, tool costs, elapsed time, and human minutes spent on coordination/review; record unavailable costs as unknown. Unblocking requires completing the predefined next step and passing artifact acceptance; replying "I have taken over" does not count.

The following are proposed decision thresholds, not observed facts: a maximum of 20 minutes per arm and a total human-effort budget of 6 hours for the whole round. For checking, at least 6/8 pairs must satisfy errors_B ≤ errors_A, at least 3/8 must satisfy errors_B < errors_A, and no new substantive errors may be introduced; also report changes in both arms relative to the original draft. Calculate B−A human minutes for each pair; the median must not exceed 10 minutes to proceed to another round. For handoffs, B must pass acceptance in at least 3/4 cases and succeed in at least 1 more case than A before expansion is considered. Pause for a retrospective if budgets are exceeded, serious errors are endorsed, or actions exceed authorization. Small samples cannot establish population success rates or commercial demand; even if the thresholds are met, invite a small number of real users to use the workflow repeatedly before deciding on product development.