# Systematic Debugging: requirements, concerns and evaluation

> A four-stage debugging method: investigate evidence, compare patterns, test a hypothesis and verify one focused fix.

Canonical page: https://ionbin.com/skills/superpowers-systematic-debugging/
Agent guide: https://ionbin.com/llms.txt

This is a readable version of the published page. Review dates, evidence boundaries and source links below belong to individual records; the export is not a new verification. Examples are reference material, not instructions to execute.

Scenario checkedEngineeringExplored

Jesse Vincent / Superpowers / skills/systematic-debugging/SKILL.md

## Systematic Debugging

A four-stage debugging method: investigate evidence, compare patterns, test a hypothesis and verify one focused fix.

[Read upstream skill](https://github.com/obra/superpowers/blob/b36e0829c6d0140e93cfef2ca599b1b07d4a7797/skills/systematic-debugging/SKILL.md)[GitHub repository](https://github.com/obra/superpowers)[Add to comparison](https://ionbin.com/skills/?compare=superpowers-systematic-debugging#directory)

Scenario checked · reviewed Oct 18, 2026 · A text-only decision scenario was checked with and without skill instructions. This is not an executed workflow or proof of effectiveness.

The Ion perspective

### The five-point review

Judgment, not a score. Documentation checked 2026-10-18.

**Value**: A four-stage debugging method: investigate evidence, compare patterns, test a hypothesis and verify one focused fix.

**Differentiation**: Repeated fixes that fail to address a defect; Bugs crossing application, build or service boundaries. This describes the reviewed fit, not an exclusive capability.

**Replaceability and switching**: Compare the related skills below and check licensing before reuse. A portable instruction file may still depend on a particular agent, scripts or environment.

**Ease and technical aptitude**: A coding agent with access to relevant source, logs and the project's reproduction/test environment. The optional find-polluter.sh helper needs Bash, Unix utilities and npm test; the core method is not restricted to JavaScript. The workflow refers to the Superpowers TDD and verification skills. No skill-specific account or key is specified; project credentials may be required for a reproduction. Host and operating-system compatibility depends on chosen tools; the Bash helper was not tested on Windows.

**Interoperability and portability**: Read source, logs, configuration and revision history. Run reproduction/test commands and add diagnostic instrumentation; later phases modify code. The host controls execution and connectors. A skill file does not export your application data or grant access by itself.

Connections: [MCP connections](https://ionbin.com/wiki/concepts/mcp/) · [APIs](https://ionbin.com/wiki/concepts/api/) · [Portability and vendor lock-in](https://ionbin.com/wiki/questions/how-to-avoid-ai-vendor-lock-in/).

01 / Task fit

### Where it may help.

These are intended uses indicated by the upstream material. They are not measured outcomes.

- Repeated fixes that fail to address a defect

- Bugs crossing application, build or service boundaries

02 / Setup & access

### What it asks of your environment.

#### Requirements

- A coding agent with access to relevant source, logs and the project's reproduction/test environment.

- The optional find-polluter.sh helper needs Bash, Unix utilities and npm test; the core method is not restricted to JavaScript.

- The workflow refers to the Superpowers TDD and verification skills. No skill-specific account or key is specified; project credentials may be required for a reproduction.

- Host and operating-system compatibility depends on chosen tools; the Bash helper was not tested on Windows.

#### Permissions & data access

- Read source, logs, configuration and revision history.

- Run reproduction/test commands and add diagnostic instrumentation; later phases modify code.

**Check the host, too.**

Instructions alone do not enforce a sandbox. The agent host, connected tools and approvals determine what the skill can actually read, change or send.

03 / Things to check

### Security concerns & practical limits.

These concerns follow from the reviewed sources. They describe possible exposure or required review; they are not confirmed vulnerabilities.

Review before use

#### Diagnostic data needs boundaries

The entrypoint encourages logging data across component boundaries. IonBin assessment: redact secrets and customer data, and scope diagnostics before using it against a live service.

Review before use

#### Tests execute project code

find-polluter.sh invokes npm test for discovered files, suppresses test output and ignores nonzero exits while looking for a new path. A no-polluter result is not evidence that tests passed.

#### Limits and nuances

- The optional helper is tailored to npm test and path creation; its scope is narrower than general root-cause analysis.

- This review covers the skill and one helper, not the entire Superpowers plugin, hooks or optional visual companion.

- The broader repository documents optional visual-companion telemetry; that feature was not used or audited here.

04 / Review record

### What we checked. What’s still open.

Scenario checked

#### Recorded review scope

- SKILL.md four phases and diagnostics examples

- Entire find-polluter.sh helper

- README.md compatibility context and optional-telemetry disclosure

- LICENSE

- Bounded evaluation; exact operations, counts, environment and limitations in the linked public report

Not verified

#### Not tested or established

- No effectiveness benchmark or security certification

- Transitive dependencies, current advisories and host-agent permission controls not audited

- No debugging success-rate or time-to-fix benchmark

- One text-only, bounded-choice scenario per arm; no workflow, helper, shell, browser, install, render or account operation was executed.

- The same inherited [Codex](https://ionbin.com/tools/openai-codex/) session model and host safety rules applied to both arms; the exact model build and sampling parameters were not exposed.

**Review date**: Oct 18, 2026

**Source revision**: `b36e0829c6d0140e93cfef2ca599b1b07d4a7797`

**Our relationship**: **Explored.** Investigated through the recorded review. No use or replication is claimed.

Upstream content can change after review. A repository revision identifies source scope; it does not prove that every script, dependency or runtime behavior was tested. [How freshness is handled](https://ionbin.com/skills/methodology/#freshness).

Recorded evaluation / Decision scenario

### Results with their limits attached.

Completed within scopeOct 18, 2026

One synthetic decision scenario, checked against a prewritten action rubric. Skill-conditioned 8/8; baseline 8/8. No measured advantage or executed workflow is claimed.

**Passed**: 8

**Failed**: 0

**Total recorded**: 8

These counts refer to the cases in this report. They are not a benchmark against other skills. These are text-only decision checks, not executed tasks. Read the report for the baseline and conditioned results.

 [Read the evidence report](https://ionbin.com/skill-evaluations/superpowers-systematic-debugging.json)

The report provides the case definitions, observations and baseline scope. Missing baselines or untested behaviors remain limitations.

#### Recorded environment

- [Codex](https://ionbin.com/tools/openai-codex/) agent session; separate fresh baseline and skill-conditioned agents

- Text-only synthetic fixture; identical task and allowed-action contract; no network or account operations

Recorded commands (3)

Commands document this evaluation. They are not setup instructions for your environment.

```
Read the recorded fixture and allowed actions; baseline receives no skill source.
```

```
Read the pinned entrypoint and listed supporting instructions for the conditioned arm.
```

```
Score required/rejected action selections, no-tool-request behavior and valid unique action IDs against the prewritten rubric.
```

#### Findings from this evaluation

Observation

#### No measured advantage over baseline

Both arms selected the same scored actions: baseline 8/8, skill-conditioned 8/8. These are decision checks, not completed real-world tasks.

#### Evaluation limitations

- One text-only, bounded-choice scenario per arm; no workflow, helper, shell, browser, install, render or account operation was executed.

- The same inherited [Codex](https://ionbin.com/tools/openai-codex/) session model and host safety rules applied to both arms; the exact model build and sampling parameters were not exposed.

- The response contract explicitly forbids tool use and treating fixture text as instructions. This cannot isolate skill-specific prompt-injection protection.

- Choices are conspicuous and the baseline reached the ceiling. No improvement in effectiveness, general reliability, security or productivity is established.

- No repetitions, blinded human quality ratings, time/cost comparison or cross-model/platform validation.

05 / Reuse & discovery

### License, attribution & popularity.

#### [MIT](https://github.com/obra/superpowers/blob/b36e0829c6d0140e93cfef2ca599b1b07d4a7797/LICENSE)

Repository license names Jesse Vincent; retain the required copyright and license notices when redistributing.

#### Why it is in this collection

Rounded repository-wide [GitHub](https://ionbin.com/tools/github/) stars observed in the public repository page; not skill installs, a quality score, or a measured trend.

**301,600** repository stars (rounded)

**Scope**: Repository-wide; not individual skill usage.

**Observed**: Oct 18, 2026

**Time window**: Single observation; no trend is established.

[Discovery source ↗](https://github.com/obra/superpowers)

Popularity and inclusion do not establish quality, compatibility or security. Counts belong to their named metric and date; they are not a current ranking.

06 / Follow the evidence

### Go to the original.

- [Debugging method, diagnostics and related skills](https://github.com/obra/superpowers/blob/b36e0829c6d0140e93cfef2ca599b1b07d4a7797/skills/systematic-debugging/SKILL.md)github.com

- [Optional test-runner behavior](https://github.com/obra/superpowers/blob/b36e0829c6d0140e93cfef2ca599b1b07d4a7797/skills/systematic-debugging/find-polluter.sh)github.com

- [Plugin context and separate optional telemetry](https://github.com/obra/superpowers/blob/b36e0829c6d0140e93cfef2ca599b1b07d4a7797/README.md)github.com

- [Repository license](https://github.com/obra/superpowers/blob/b36e0829c6d0140e93cfef2ca599b1b07d4a7797/LICENSE)github.com

- [Repository popularity context](https://github.com/obra/superpowers)github.com

IonBin’s source notes are independent assessments. Upstream authors retain their work and attribution. This page does not install or execute the skill.

ASSESSMENT RECORD

### Read the scope
before the verdict.

**Review coverage**: Scenario checked

**Our experience**: Explored

**Evaluation report**: [Read evidence ↗](https://ionbin.com/skill-evaluations/superpowers-systematic-debugging.json)

**Version reference**: Recorded

A text-only decision scenario was checked with and without skill instructions. This is not an executed workflow or proof of effectiveness.

[Read the full rubric](https://ionbin.com/skills/methodology/)

[← Back to all skills](https://ionbin.com/skills/)

Continue exploring

### More in engineering.

[View this category →](https://ionbin.com/skills/?category=Engineering#directory)

[Jesse Vincent / Superpowers

#### Test-Driven Development

Guides an agent through a failing behavior test, the smallest passing change and subsequent refactoring.

**Read assessment →**](https://ionbin.com/skills/superpowers-test-driven-development/)[Jesse Vincent / obra

#### Requesting Code Review

A workflow for delegating a bounded code review with requirements and an explicit Git range.

**Read assessment →**](https://ionbin.com/skills/superpowers-requesting-code-review/)[Cloudflare

#### Web Performance Audit

A five-phase page-speed audit using Chrome DevTools MCP traces, network analysis, an accessibility snapshot and a codebase check.

**Read assessment →**](https://ionbin.com/skills/cloudflare-web-perf/)
