AI Writes Code Like a Confident Intern
Vibes & Verdicts — Issue #5
THE VIBE
AI Writes Code Like a Confident Intern
I need to tell you about the scariest thing I’ve found in my own codebase.
A few weeks into building Batesly, I set up a code review workflow using Claude Code. I wrote a prompt that tells it to scan my entire repository, classify every issue by severity, and fix the critical stuff automatically. I expected it to find some lint errors. Maybe some unused variables. General housekeeping.
It found 47 issues.
One of them was a multi-tenancy data isolation bug. The permission check was filtering by user but not by organization. The app “worked” perfectly in testing — because I was only testing with one organization.
Think about it… the AI wrote code that handled authentication correctly but failed at authorization. It checked “is this a valid user?” but not “does this user’s organization own this document?” The app looked secure. It felt secure. It was not secure.
This is what I mean when I say AI writes code like a confident intern. The output is syntactically correct, follows patterns, handles the obvious cases, and looks professional. But it makes assumptions. It takes shortcuts. It misses the edge cases that only matter when real data and real people are involved.
Here’s a taxonomy of AI code bugs I’ve encountered, ranked by how much they kept me up at night:
Authorization gaps (terrifying): Missing org_id checks on database queries. Permission checks that verify roles but not resource ownership. API endpoints that authenticate users but don’t authorize the specific action. These look correct at a glance and fail catastrophically in production.
Silent failure modes (sneaky): Error handling that catches exceptions and does nothing. API calls that fail and return empty arrays instead of errors. Database queries that return no results because of a bad filter, and the UI just shows “no documents found” instead of flagging the problem. The app never crashes. It just quietly gives you wrong data.
Hardcoded assumptions (annoying): Values that should be environment variables baked into the code. URLs pointing to localhost that work in development and break in production. Default admin credentials that were supposed to be temporary. These aren’t security vulnerabilities per se, but they’re the kind of thing that makes your ops life miserable.
Optimistic data handling (subtle): Code that assumes data will always be in the expected format. No null checks. No type validation on user inputs. No handling for the case where a related record has been deleted. These bugs hide until someone does something unexpected — which, with real users, happens immediately.
The review workflow I built to catch these is simple: I build features in Replit, sync to GitHub, then run Claude Code against the repository with a comprehensive review prompt. Claude Code identifies issues, classifies them by severity, and fixes the critical ones. I review the fixes, push the clean code back to GitHub, and pull it into Replit.
It’s effective (but not very elegant). And it’s the difference between shipping software and shipping liability.
If you’re vibe coding anything that handles real data — user information, financial records, sensitive documents — you need a review workflow. The AI that writes your code is talented and careless. Just like an intern. And just like an intern, it needs a senior review before anything goes to production.
THE VERDICT
Quick Takes
“No-code” platforms are creating a wave of insecure applications and nobody’s talking about it. When you make it easy for non-engineers to build software, you also make it easy for non-engineers to build insecure software. The tools have a responsibility to bake security in by default. Most don’t.
The duty of technology competence applies to AI-generated code too. Lawyers have an ethical obligation under ABA Model Rule 1.1 to understand the technology they use. If you’re using AI to build tools that handle client data, “I didn’t write the code” isn’t a defense. You chose the tool. You shipped the product. The responsibility is yours.
Security audits aren’t just for enterprise companies. I ran a comprehensive security audit on Batesly when it was still just me using it. Found and fixed 47 issues. The cost: zero dollars (Claude Code). The time: one evening. The peace of mind: considerable. If you’re building anything that will ever touch user data, do this now, not later.
THE CONFESSIONAL
The Multi-Tenancy Bug Was Live for Weeks
That data isolation bug I mentioned? It wasn’t in some unreleased feature. It was in the production codebase for multiple weeks before I caught it.
No one exploited it. There were no other tenants in the system yet. The exposure was theoretical. But “no one exploited it” is not the same as “it was fine.” If I’d onboarded two firms before running that audit, one firm’s attorneys could potentially have accessed another firm’s documents.
For a legal tech platform. Where the entire value proposition is trust and confidentiality.
I’m sharing this because I think the vibe coding community needs more honesty about the risks. Building fast is great. Shipping features is great. But if you’re not auditing what you ship, you’re gambling. And when the stakes are client data, the house edge is not in your favor.
Thanks for reading.
I’m Rachel — a practicing attorney building legal tech through vibe coding. Every week I share what I’m learning, what’s working, and what I probably shouldn’t admit publicly.
Next week: “Your Favorite Legal Tech Company Hates You” — Per-seat pricing, vendor lock-in, and why the legal tech industry is overdue for a reckoning.
Know someone building without a CS degree? Forward this. They’ll thank you… maybe?