Skip to content

Guide

A practical checklist for reviewing AI-generated code before you merge

Ten things to check in code an assistant wrote, from invented dependencies and browser-only auth checks to swallowed errors and tests that assert nothing.

· 9 min read · By the vetkit team

A stack of ten glass checklist tiles, most lit and a few still unlit, beside a small group of code blocks waiting for review.A stack of ten glass checklist tiles, most lit and a few still unlit, beside a small group of code blocks waiting for review.

Code from an assistant usually compiles, reads well and does roughly what you asked. That is exactly why it is easy to wave through. The mistakes are rarely syntax errors. They are decisions the model made quietly: a dependency it assumed exists, a permission check it put in the wrong place, an error it decided to ignore.

This is the checklist we use when we review AI-assisted changes. It is not about whether the code is elegant. It is about the handful of problems that turn into incidents.

Why AI code needs a different review

When a colleague writes code, you can usually trust the parts they did not mention. They knew the codebase, tested the change locally and would have said if something was unfinished. A model has none of that context unless you gave it, and it tends to fill gaps with something plausible rather than with a question. So the review shifts from “is this well written” to “what did it assume”.

1. Dependencies

Check every package the change adds. Does it exist on the registry? Is it the package the official docs recommend, or a similar-sounding one? Was it published last week? Models sometimes invent package names, and some of those names get registered by attackers. Our guide to slopsquatting covers how to check a package in a minute.

Also look at versions. A version pinned to something that was never released will break the next fresh install, and an unpinned dependency means the code you reviewed is not the code that runs.

2. Secrets and public variables

Search the diff for keys, tokens and passwords, including in test files and examples. Then look at environment variables with public prefixes such as VITE_, NEXT_PUBLIC_ or EXPO_PUBLIC_. Anything with those prefixes is copied into the browser bundle at build time. An assistant may rename a secret to one of them to get a feature working in the browser. This guide walks through the LLM API key version of that mistake.

3. Where access is checked

Find the line that decides whether the current user may do this. If it is in a React component, a route guard or a hidden button, it is a user experience feature, not security. Anyone can call your API directly. The check has to happen on the server, for every request, against the resource being accessed: not only “is this user signed in” but “does this user own this invoice”.

Watch for new API routes with no auth middleware at all, and for backends like Supabase or Firebase where the database is reachable from the browser. There, the protection is the database’s own rules. A Supabase table without row level security is open to anyone with your public anon key.

4. Untrusted input

Follow user input from where it enters to where it is used. String-built SQL, shell commands assembled from request fields, file paths from query parameters, HTML rendered from user content and URLs fetched on a user’s behalf are the classic injection, path traversal, XSS and SSRF patterns. Assistants still produce them, mostly when the surrounding code already does something similar.

text
// Looks harmless, is an injectiondb.query(`SELECT * FROM users WHERE email = '${req.query.email}'`); // Parameterizeddb.query('SELECT * FROM users WHERE email = $1', [req.query.email]);

5. Unfinished code

Long generations sometimes stop short. Search the change for placeholders: TODO, “implement later”, “rest of the code here”, functions that return hard-coded sample data, mock credentials, and files with names like utils copy.ts. Also look for blanket suppressions such as // @ts-ignore, eslint-disable at the top of a file, or any used to silence the type checker. They often hide the exact problem the model could not solve.

6. Error handling

A catch block that logs and continues, or catches everything and returns an empty result, turns a loud failure into silent data loss. Ask what should happen when this call fails, and whether the code does that. Be suspicious of retries without limits and of errors that are converted into success responses.

7. Tests

Generated tests are worth reading line by line. Common problems: tests that never assert anything, tests that mock the very function they are testing, assertions that only check that a function was called, and .only or .skip left behind. A test suite that passes is not evidence until you know it can fail. Break the code on purpose once and see whether a test notices.

8. Timeouts and blocking calls

Outbound HTTP calls without a timeout are a production incident waiting for a slow dependency. Many popular clients wait indefinitely by default, including Python’s requests and Go’s http.DefaultClient. In Node.js, synchronous file operations or password hashing inside a request handler block every other request on that process. These rarely show up in local testing because your laptop and your dependencies are fast and idle.

9. Data access

Look for queries inside loops (the N+1 problem), list endpoints that return every row without pagination, and database clients created on every request instead of shared. Check migrations carefully too: a schema sync that runs on startup, or a migration that drops and recreates a table, can delete production data on the next deploy.

10. LLM features

If the change calls a model, treat the model’s output as untrusted input. It should never be executed, concatenated into SQL or rendered as raw HTML. Check that the API key stays on the server, that the route calling the model requires a signed-in user and has a rate limit, and that output length is capped. If an agent has tools, check what those tools can do and whether text from documents or web pages can steer them.

Make it routine

Ten checks per pull request is a lot to do by hand, and the point of a checklist is that you stop relying on memory. Most of these items can be automated, at least partly. That leaves your attention for the things tools are bad at, such as whether the change does what the product actually needs.

vetkit groups these checks into suites: security (secrets, vulnerable dependencies, code vulnerabilities, infrastructure configuration), code quality, sustainability, AI code risks (invented packages, unfinished code, app platform pitfalls, LLM and agent security) and production readiness (timeouts, blocking calls, database usage, operational basics). It reads your code without installing, building or running it, and reports what it finds with the file, line and a suggested fix. Like any static analysis, it can miss things and it can be wrong, so it supports a review rather than replacing one.

Request early access

Tell us a little about your team and the repositories you want to audit. We onboard teams in small batches and will reply by email.

We use these details only to reply to your request. Privacy policy