Business teams are now building their own software with AI tools like Claude, Cursor, Lovable and Replit. An operations manager can ship a working internal app in a weekend. Most of these apps never pass through IT, never appear in the application inventory, and never get tested against the controls your audit plan assumes.
Our team audits and repairs AI built applications for a living. The same seven control failures show up again and again. None of them need deep coding knowledge to detect. Each one below comes with the evidence an auditor can request to spot it.
The seven failures
1. Secrets stored in the code
AI tools often paste API keys, database passwords and payment tokens straight into the source code. Sometimes the keys ship inside the browser code, where any user can read them.
How to spot it: Ask for the output of a secret scanning tool such as Gitleaks or GitHub secret scanning, which checks the full Git history for exposed keys. Ask where credentials are stored. "In the code" or "I'm not sure" is a finding.
2. Access checks that only exist on screen
The app hides a button from regular users, and the server still accepts the request behind it. OWASP ranks broken access control as the top web application risk. Apps built on Supabase often ship with row level security switched off, so any logged in user can read every record.
How to spot it: Ask for a test showing that User A cannot view or edit User B's records through the API. If nobody can produce one, assume the control is missing.
3. No change management
Code goes from the AI chat straight to production. There is no pull request, no second reviewer, and no record of who approved what.
How to spot it: Look at the repository history. Direct commits to the main branch with no approvals mean segregation of duties has failed. Some apps have no repository at all and live only inside the builder tool.
4. Tests that were edited to pass
When a test fails, an AI assistant will sometimes change the expected answer to match the broken output. The test suite stays green and proves nothing.
How to spot it: Ask which business rules the tests cover, such as pricing, tax, or dosage limits. Then check the history of those test files for changed expected values with no linked ticket.
5. Silent data corruption
AI written import code often assumes formats, units, currencies and time zones. When an upstream file changes, nothing crashes. The numbers are simply wrong.
How to spot it: Reconcile a sample of records against the source system. Ask what happens when an input file has an unexpected column or unit. "It would still load" is a finding.
6. No audit trail
Many vibe coded apps let users delete records permanently and keep no log of changes. When something goes wrong, nobody can say who changed what.
How to spot it: Pick a record and ask who last changed it, when, and what the old value was. If the app cannot answer, logging is missing.
7. Unknown dependencies
AI tools pull in open source packages freely. Some are outdated with known vulnerabilities. Some names are invented by the model outright, a problem documented in a 2025 USENIX Security study on package hallucinations. Attackers now register those invented names with malicious code, a tactic known as slopsquatting.
How to spot it: Ask for a software bill of materials (SBOM) or a dependency scan from a tool such as Dependabot, Snyk or npm audit. No list of dependencies means no one is managing them.
Evidence request list
These map to the IT general control domains most audit plans already use, so they can be added to an existing request list.
# | Control failure | ITGC domain | Evidence to request |
|---|---|---|---|
1 | Secrets in code | Logical access | Secret scan report, credential storage location |
2 | Screen only access checks | Logical access | API test proving users cannot reach each other's records |
3 | No change management | Change management | Repository history, branch protection settings, approval records |
4 | Edited tests | Change management | Test coverage of business rules, history of test file changes |
5 | Silent data corruption | Data integrity | Reconciliation sample, input validation rules |
6 | No audit trail | IT operations | Change log for a sampled record, delete policy |
7 | Unknown dependencies | IT operations | SBOM or dependency scan report |
Where to start
The hardest step is finding these apps. Ask each department one question: "Is anyone using a tool that someone here built with AI?" Add every answer to the application inventory, then run the seven checks above on anything that touches customer data, money, or regulated records.
If the checks turn up failures, most apps can be repaired in place. A structured vibe coding cleanup and fix usually starts with secrets and access control, then moves to change management and logging.
What are others seeing? If your team has already added AI built apps to its audit scope, I'd like to hear how you found them and which controls failed first.
Abd works at Clixlogix, where the team audits and stabilizes AI built applications.
Replies