How to write a reproducible bug report for an AI-built app

A reproducible bug report for an AI-built app names the environment, exact steps, synthetic input, expected outcome and observed outcome. Add evidence of the committed state where possible and a clear retest condition. Describe impact in business terms without guessing at severity or blaming the builder. Another person should be able to follow the report and observe the same failure.

This guide is for founders and agency owners preparing an English-language app or integration for users in the US, UK, Canada or Australia. The method is about minimum reproduction and falsifiable retest. Country-specific legal or compliance requirements need their own qualified review; this guide does not establish them.

Define the outcome before running a check

Write the expected result in business language before examining the implementation. Ask which record should exist, which user may access it and what should happen when the action cannot finish. If two people disagree on that policy, record a product decision rather than calling one implementation a bug. Agree the environment and permission boundary before testing.

Worked synthetic example

On release demo-7, Chromium, signed-in member: open Enquiry, enter synthetic details, intercept POST with 500, submit. Expected: error and retained input. Actual: ‘Sent’ and empty form. Retest: 500 never redirects; 201 with committed receipt redirects exactly once.

These rows are test expectations, not observations from a client app. Mark each result NOT RUN until it has actually been executed. A blocked credential, unavailable provider or missing fixture belongs in a blocked column. Do not count it as a pass to make the report look complete.

Practical verification sequence

1. Record release, browser, role and authorized environment.
2. Reduce the steps to the smallest repeatable failure.
3. Separate expected policy from actual observation.
4. Attach redacted evidence and remaining uncertainty.
5. Define the acceptance assertion before commissioning a repair.

Run one case at a time and reset the fixture between cases where the previous action changes state. Keep the initial state reproducible: a named synthetic account, an agreed role and a known record. If the case fails intermittently, record the timing and repeat conditions. Do not erase a failure by rerunning until it happens to pass; investigate what changed.

Turn a finding into a decision

A useful finding separates expected behavior, observed behavior and the reason it matters. Attach the smallest reproduction you can produce. Name the affected journey and the conditions you tested. If you cannot inspect a downstream record, say that the downstream result is unverified. This lets the owner choose whether to repair, add observability, change the product policy or defer release.

For a repair, agree the acceptance condition before implementation. Retest the original failure and one adjacent ordinary path, then record the release identity. A narrow repair should not quietly become an unrelated redesign. When a broader issue appears, revise the scope and explain the dependency.

Sources and scope limits

Use the primary documentation for the underlying platform or verification contract. Documentation is a reference, not evidence that your app conforms. The worked example above is original and synthetic. It makes no claim about client outcomes, defect rates or every AI-generated app.

Next step

Discuss a scoped review when the failure affects a critical journey or you need implementation and retest evidence. Book a discovery call with the actual stack and a description of the broken outcome. Do not send production secrets, source archives or customer data in the initial enquiry.

Frequently asked questions

What should reproduction demonstrate?

The expected result is: 500 enquiry response. Establish the initial state, execute the case and inspect the resulting business state. This example is synthetic; your product policy determines the actual assertion.

What evidence should I keep?

Keep the release identity, role, environment, steps, expected result, observed result and status. Redact personal data, tokens and secrets. A passing screenshot alone may miss a failed backend effect.

How should I handle observed?

For this example, expect: False success. If that cannot be demonstrated, record fail or blocked with the missing evidence. Do not infer coverage for other roles, integrations or future releases; this is not a security certification.

When should I request paid help?

Request a scoped review when a critical journey fails repeatedly, the expected policy is unclear, or you cannot determine what committed. Bring reproduction steps and the actual stack; share access only through an agreed process.