I hired my own AI to hack my product. It found 19 ways in.

I pointed my own AI build system at my own product and told it to find every way in. It came back with fifty-nine claims. Nineteen survived scrutiny. One of them was critical — and sitting directly above that code was a comment of mine asserting the exact protection that wasn't there.

This is Thread 16 of Loomwork — patient work, woven in public. New here? Thread 01 explains why this newsletter exists, in 2 minutes. Threads 12 and 13 were about protecting what an agent touches — the credential vault, the write-access gates. This one hunts for everything those two chapters didn't already catch.

The hard problem here isn't "did I get hacked." It's harder than that: how do you find the vulnerabilities you don't know to look for, with no security team, without either scaring yourself into over-claiming risk or reassuring yourself into missing something real?

Four rules came out of it. They're the transferable part — the findings themselves are mine, but the rules are the method, and I'd defend all four in a design review.

Rule 1 — Refute before you count.

A single broad pass asking one model "find security bugs" either hallucinates plausible-sounding risk or misses the real ones — it has no way to check its own work. I ran an 11-dimension audit instead — auth, API routes, server-action authorization, RLS, secrets and bundle leakage, credential crypto, XSS/injection, headers, dependency CVEs, prompt injection, live probing — and then made every claim survive an independent skeptic whose only job was to refute it.

The raw output was 59 claimed findings. 23 didn't survive: a "path injection" that turned out to be URL-encoded and inert, a Next.js CVE that doesn't apply to how this app uses middleware. What was left was 36 confirmed, deduped to 19 canonical — 1 Critical, 6 High, 5 Medium, 5 Low, 2 Info. The discipline of throwing two-thirds of the raw claims away mattered more than the tool that generated them. An audit without a refutation step isn't an audit; it's a list.

Rule 2 — Fix the pattern, not the instance.

The Critical and three of the Highs were the same bug wearing different clothes: server actions accepting a userId as a parameter and using it as the authorization scope — while the Supabase client behind them uses a service-role key that bypasses row-level security entirely. An attacker literally supplies the id they're checked against.

Patching each call site individually would have left the next one open. Instead: one shared requireUserId() helper that derives the id from the session, never the caller, used everywhere. That single refactor closed the Critical and three Highs at once.

The part I only learned later is the part that matters. Re-verifying before publishing this, I checked the ten action modules written after the audit — the board, the credential store, the conversation layer, and seven others that didn't exist when the finding was filed. Every one of them derives identity from the session. The refactor didn't just close four findings; it became the house habit. That's the actual test of a pattern fix: not whether the old call sites are clean, but whether the new ones were never written wrong.

Rule 3 — Trust the code, never the comment.

The database had no real backstop: two of the busiest tables carried zero row-level-security policies. That's a finding. This is the one that stayed with me — comments in the code claimed "RLS enforced via auth.jwt()" when the client was built with a service-role key that never touches a JWT. Not out of date. Never true.

A docstring is a claim, not a control, and I'd been reading that one as proof for months. So the fix couldn't just be a corrected comment — a corrected comment is exactly as trustworthy as the wrong one was. Real policies went in, the false comments were rewritten, and a check went into the build that fails when any new query is missing its ownership filter. The durable fix for a lying comment is a test, because a test is the only kind of claim that has to keep being true.

Rule 4 — Stage the fix that can fail silently.

Those policies are a dormant backstop. The app still reads with the service-role key, which bypasses them by design. The change that makes row-level security actually enforce — switching user-facing reads to the anon key plus a Clerk-issued JWT — is bigger and riskier than it looks, and I deliberately did not rush it.

Here's the reason, and it's the general one: a careless flip returns empty results to real users instead of erroring. It fails quieter than the hole it closes. A loud failure gets fixed in an hour; a silent one ships, and you find out when someone tells you their data is gone. Any fix whose failure mode is quieter than the bug gets its own gate and its own week — never the tail end of an audit sprint.

The other fifteen.

An unauthenticated endpoint was running a real paid Sonnet call for anyone on the internet — confirmed live, not theoretical: a bare curl returned a real completion in 2.16 seconds. Fixed with an auth gate plus a rate limiter built on infrastructure I already had, no new vendor. A cron guard failed open when its secret was blank. Config fields could point outbound fetches at internal metadata hosts. External PR and issue text was being pasted straight into the model's instructions with nothing separating "data to review" from "commands to follow" — which is its own chapter, and the subject of the next one. Seventeen of nineteen are now fixed, each behind an independent reviewer's approve and a security GO before merge.

What I'm deliberately not doing yet.

One finding is genuinely still open, and one isn’t a finding at all. The open one is the enforcement cutover from Rule 4 — real policies exist, but the service-role key still bypasses them day to day. The other is that sign-ups are open to the public, which the audit flagged and I closed as a decision, not a gap. That distinction matters more than the count: a tracker that quietly reclassifies decisions as findings is how a security posture starts lying to you. And it's exactly why the cutover is next rather than optional — open sign-ups are the reason the database-level backstop has to stop being a plan.

A server action is a POST endpoint with better manners — its arguments are exactly as attacker-controlled as any URL parameter, no matter how trusted the code around it feels.

Nineteen findings isn't a scary number. Finding them yourself, before anyone else does, is the whole point.

— Vamshi

Loomwork is by Vamshi Krishna Veggalam. Building Foundry at mindloomhq.com, with the full archive at loomwork.dev. Find me on LinkedIn.