5-agent SOP-encoded pipeline — 124.3 tokens/LoC, executability 3.75/4.
Recent activity
Version cuts and proof, newest first — the living track record.
Spec sheet
The benchmark fields — designed for comparison across teams.
- Topology
- Pipeline
- Agent count
- 5
- Platform
- MetaGPT
- Runs on
- MetaGPT ×5
- Industries
- software-delivery
- Task kinds
- software-developmentrequirements-analysiscode-generationqa
- Trust tier
- Self-Reported
- Proof entries
- 1
Topology & roster
Sequential pipeline with shared message pool. Product Manager → Architect → Project Manager → Engineer → QA Engineer. Agents publish structured messages and subscribe to task-relevant information. Sequential workflow prevents cascading hallucination via intermediate verification at each step.
System wiring
Node details
Typical Pipeline layout — schematic, not verified wiring
HumanHuman operatorHuman gate
- Tool
- Human operator
- Autonomy
- Human-gated
- directs → Stage 1 agent
BuilderStage 1 agent
- Tool
- Stage 1 agent
- Autonomy
- Runs autonomously
- hands off to → Stage 2 agent
- directs ← Human operator
BuilderStage 2 agent
- Tool
- Stage 2 agent
- Autonomy
- Runs autonomously
- hands off to → QA reviewer
- hands off to ← Stage 1 agent
QAQA reviewer
- Tool
- QA reviewer
- Autonomy
- Runs autonomously
- delivers to → Output artifact
- hands off to ← Stage 2 agent
ResourceOutput artifact
- Tool
- Output artifact
- Autonomy
- Runs autonomously
- delivers to ← QA reviewer
How a typical Pipeline team handles a task
Typical Pipeline layout — schematic, not verified wiring
Task arrives
Human operator directs Stage 1 agent.
The builders execute
Stage 1 agent and Stage 2 agent build the work.
Independent review gates the work
QA reviewer reviews the work. This reviewer is autonomous and separate from the agent that built the work, so the check is independent of its author.
The artifact lands
The artifact lands in Output artifact: QA reviewer contributes via "delivers to".
Human holds the last word
Human operator holds final approval.
Replicate a typical Pipeline setup
Typical Pipeline layout — schematic, not verified wiring
Ingredients
- HumanHuman operator
- BuilderStage 1 agent
- BuilderStage 2 agent
- QAQA reviewer
- ResourceOutput artifact
Setup order
- 1.Provision the substrate: Output artifact.
- 2.Wire Stage 1 agent: it receives "directs" from Human operator and sends "hands off to" to Stage 2 agent. Wire Stage 2 agent: it receives "hands off to" from Stage 1 agent and sends "hands off to" to QA reviewer. Wire QA reviewer: it receives "hands off to" from Stage 2 agent.
- 3.Give QA reviewer an independent workspace/verdict channel: "delivers to" to Output artifact.
- 4.Declare the human gate: Human operator holds final approval.
Performance metrics
Windowed metrics with provenance. [unknown] means it was not tracked — an honest hole beats an invented figure.
SoftwareDev benchmark; vs ChatDev 248.9. Source: arXiv 2308.00352 [evidence_linked]
With executable feedback loop. MBPP: 87.7% Pass@1. Source: arXiv 2308.00352 [evidence_linked]
3.75/4; vs ChatDev 2.25. Source: arXiv 2308.00352 Table 3 [evidence_linked]
With executable feedback loop. Source: arXiv 2308.00352 [evidence_linked]
Token economics
Cost transparency is part of the honesty architecture. [unknown] means it was not tracked — not that it is zero.
Blueprint
Operational DNA — why it works, how it was built, and how it is overseen. Not files for sale; knowledge of the design.
SOPs give each agent a structured, verifiable output format — reducing hallucination cascades. The shared message pool is more efficient than direct dialogue. Token efficiency of 124.3 tokens/LoC (vs 248.9 for ChatDev) reflects the structured communication overhead reduction.
SOPs encoded as prompt sequences for each role. Publish-subscribe message pool eliminates one-to-one communication overhead. Executable feedback loop: runtime code execution and iterative debugging yield 4.2% and 5.4% improvements in Pass@1 on HumanEval and MBPP respectively. HumanEval: 85.9% Pass@1; MBPP: 87.7% Pass@1.
SOP verification at each pipeline stage — agents check intermediate results against structured specifications. The QA Engineer formulates test cases and validates code quality as the final stage.
Proof (1)
The team's shared track record — tasks, incidents, lessons, milestones. Per-entry provenance tags are always visible.
- ArtifactAug 1, 2023evidence-linked
MetaGPT paper published (arXiv 2308.00352)
Five-agent SOP pipeline. HumanEval: 85.9% Pass@1; MBPP: 87.7% Pass@1. Token efficiency: 124.3 tokens/LoC vs ChatDev's 248.9. Executability: 3.75/4 vs ChatDev's 2.25.
https://arxiv.org/abs/2308.00352
Sign in to add a proof entry.
Sign inAttestations (0)
Named third-party statements from people with first-hand experience. Attestations are what separates Peer-Attested from Evidence-Linked.
No attestations yet. Worked with this configuration or agent? Attest to it using the form below — attestations are named third-party statements and are what separates Peer-Attested from Evidence-Linked.
Sign in to attest to this team.
Sign in