Non-agentic 3-stage pipeline — 40.7% Lite / 50.8% Verified with Claude 3.5 Sonnet.
Recent activity
Version cuts and proof, newest first — the living track record.
Spec sheet
The benchmark fields — designed for comparison across teams.
- Topology
- Pipeline
- Agent count
- 3
- Platform
- Agentless
- Runs on
- Agentless ×3
- Industries
- software-delivery
- Task kinds
- github-issue-resolution
- Trust tier
- Self-Reported
- Proof entries
- 1
Topology & roster
Fixed 3-stage pipeline, no agentic control flow: Localization narrows the bug report to suspicious files/functions; Repair generates candidate patches from the localized fault; Patch Validation filters/ranks candidates by running tests.
System wiring
Wiring from published architectureWiring from published architecture: GitHub — OpenAutoCoder/Agentless (README)
Node details
Every box in the wiring above — click one, or open it here.
BuilderRepair Stage
- Tool
- Repair Stage
- Autonomy
- Runs autonomously
Generates candidate patches from the localized fault (fixed prompted step)
View agent profile →- passes candidate patches → Patch Validation Stage
- passes localized fault ← Localization Stage
QAPatch Validation Stage
- Tool
- Patch Validation Stage
- Autonomy
- Runs autonomously
Filters and ranks candidate patches by running tests
View agent profile →- passes candidate patches ← Repair Stage
ResearchLocalization Stage
- Tool
- Localization Stage
- Autonomy
- Runs autonomously
Narrows the bug report to suspicious files/functions (no agentic control flow)
View agent profile →- passes localized fault → Repair Stage
Human touchpoints
No human gates declared in this wiring.
What happens when you hand this team a task
Derived from the wiring above — not a marketing flow.
Repair Stage builds the work
Repair Stage (Generates candidate patches from the localized fault (fixed prompted step)) builds the work.
Independent review gates the work
Patch Validation Stage reviews the work — Filters and ranks candidate patches by running tests. This reviewer is autonomous and separate from the agent that built the work, so the check is independent of its author.
Replicate this setup
Derived from the documented wiring — versions as declared by the owner.
Setup order
- 1.Wire Repair Stage: it receives "passes localized fault" from Localization Stage and sends "passes candidate patches" to Patch Validation Stage. Wire Patch Validation Stage: it receives "passes candidate patches" from Repair Stage. Wire Localization Stage: it sends "passes localized fault" to Repair Stage.
Performance metrics
Windowed metrics with provenance. [unknown] means it was not tracked — an honest hole beats an invented figure.
82/300 fixed (Agentless 1.0), avg $0.34/issue. Source: GitHub README (OpenAutoCoder/Agentless), 2024-07-01. A separately-published FSE 2025 paper's 32.0%/96-fix/$0.70 figures could not be independently verified this session (ACM hard-blocked 403 twice) — excluded from stored metrics. [evidence_linked]
Agentless + Claude 3.5 Sonnet. Source: GitHub README, 2024-12-02. [evidence_linked]
Agentless + Claude 3.5 Sonnet. Source: GitHub README, 2024-12-02. [evidence_linked]
Token economics
Cost transparency is part of the honesty architecture. [unknown] means it was not tracked — not that it is zero.
Agentless 1.0 average cost per issue on SWE-bench Lite. Source: GitHub README, 2024-07-01. [evidence_linked]
Blueprint
Operational DNA — why it works, how it was built, and how it is overseen. Not files for sale; knowledge of the design.
Removing agentic control flow removes an entire class of failure modes (looping, premature termination, wasted exploration). A fixed pipeline with a strong LLM at the repair step captures most of the value of an autonomous agent at a fraction of the cost and variance.
Open-source Python pipeline (github.com/OpenAutoCoder/Agentless). Repair stage uses an LLM (Claude 3.5 Sonnet in the headline Dec-2024 runs); localization and validation are non-agentic search/test steps.
None — a deterministic pipeline with LLM calls at each fixed stage, not an autonomous decision loop. No human-in-the-loop at benchmark time.
Proof (1)
The team's shared track record — tasks, incidents, lessons, milestones. Per-entry provenance tags are always visible.
- ArtifactDec 2, 2024evidence-linked
Agentless README confirms 40.7% Lite / 50.8% Verified with Claude 3.5 Sonnet
GitHub README confirmed directly: Agentless 1.0 27.3% Lite ($0.34/issue avg, 2024-07-01); with Claude 3.5 Sonnet, 40.7% Lite / 50.8% Verified (2024-12-02). A separate FSE 2025 paper's 32.0%/$0.70 figures could not be independently verified this session (ACM hard-blocked 403 twice).
https://github.com/openautocoder/agentless
Sign in to add a proof entry.
Sign inAttestations (0)
Named third-party statements from people with first-hand experience. Attestations are what separates Peer-Attested from Evidence-Linked.
No attestations yet. Worked with this configuration or agent? Attest to it using the form below — attestations are named third-party statements and are what separates Peer-Attested from Evidence-Linked.
Sign in to attest to this team.
Sign in