It’s really eye opening to work with these tools on a codebase you know deeply because these problems are everywhere.
However if I opened an unfamiliar project in another language and I wanted to add a little feature with no intention of maintaining it, I’d happily accept the changes and loop until it worked well enough for my temporary needs.
The scary middle is when you’re dealing with coworkers who don’t care about anything other than closing tickets and collecting credit. With enough of a token budget you can now wrap loops around an LLM and have it try things until the program appears to work. Ask it to do a code review and then submit the PR without having understood what it was doing. There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing.
I was working on creating a next-n-actions predictor for one of our use cases and not paying much attention for a PoC. I was fairly happy with the progress for a few days, before actually reading the eval code and seeing that we leaked the final state in every eval.
It's nice to let claude run loose on porting from framework to framework (port my code from TRL to NemoRL to Tinker to VeRL) but looking at what it does in the intermediate steps makes me want to claw my eyes out. And getting it to adhere to our domain model (e.g. we have an SFTConfig and a .to_trl(), or a Row and a .to_harmony()) is impossible.
Most of the time my pushbacks are true improvements, but I've seen a couple of instances where the LLM was happy to downgrade their own good solution.
Although this was more of an issue when using opusplan as Sonnet loves to do this. I only use Opus now if using Claude.
I've used this a bunch as a suffix to try to prevent that, works OK in most cases, but not always obviously, works better in the system/developer prompt if you have access to those. Seems I've used that about ~1000 times since 2025/08 when I started using codex (- transcription duplications, so maybe 1/2 of that?).
$ rg -a -o "Answer grounded in truth" ~/.codex/sessions | wc -l
1046Indeed, it's easy to surface this by sending one model a "Review" of their proposal to another, then bounce them back and forward, ask which one is best and both models will almost always say something like "The other proposal/review is better", I'm guessing because somehow they think it comes from the human, and "human is always right" or something.
Dude, the fucking model is great for sure, but there is nothing behind the illusion. It doesn't know if something is right or wrong - simpler or harder to reason about etc
It's just generating text, in a coherent manner while following rhetoric processes as a solid attempt at logical thinking
Why is that so hard for people to grok?
Our industry (and society after) is beyond doomed with people seeing these self affirmations as anything like "insightful" validation.
That fundamentally wouldn't happen if it wasn't just an illusion.
There is value in it for sure and I can use it to write a lot of simple code, which is 99.99% of enterprise software - but that's another topic.
Something that can write a correct code snippet or even larger program that accepts the correct input and provides the correct output and otherwise is consistent with the given spec is doing something substantially more than just autocomplete.
> It's just generating text, in a coherent manner while following rhetoric processes as a solid attempt at logical thinking
So yeah, I do agree that they can make a very reasonable amount of reasoning. As a matter of fact, they reason about things better then an average Joe off the street ime.
That's entirely unrelated to what I said though, I think you misinterpreted/misunderstood what I wrote earlier.
They can make solid attempts at reasoning, its just not grounded in reality. It just applies these rhetoric processes to the current text - but it doesn't understand wherever it's actually correctly reasoned. Hence the answer "you're right to push back on this" is just the model being a sycophant. The sentence does not mean that anything of value has been communicated in either direction, and thinking that it has means the person in question is suffering from ai psychosis
It all seems to behave correctly and then you run your test suite, and your e2e tests start failing in weirs ways, a few but not many accounting discrepancies, and everything else passes. You spend a lot of time asking it to explain what's happening, you give it the data to browse, and it keeps giving you very plausible explanations of "found the issue, the data shows this clearly, there fore the bug is here, all I need to do is fix this thing", and it does this, and it still fails.
When you open the hood, man, the code salad, the 100s of unnecessary, and complex and duplicate abstractions, the stacked mistakes and lazy corrective attempts, the comment pollution that overrides your instructions across sessions.
You realize that there are things and concepts that it just cannot wrap it's "mind" around and you need to grab the wheel for a bit, make the corrections, remove all the comment litter, commit and then hand the wheel back and tell it to "look at the last commit so see what I mean. explain to me what you did wrong and update all documentation, memory and context with this new understanding".
So if you have no experience in the field, you won't even know how to test, how to find that there is an issue, the appearance of "working" and the AI's confidence will trip you in prod so hard.
I don't want to start a fight or anything but IME Codex has a bit more of a spine. If you point out something weird, it sometimes gives a good reason for it. Whereas Claude will always say "whoopsie you're right as always sir" even when it's me who missed something.
But your comment just made me think whether this tendency for LLMs to resort to flattery when found out is a built in strategy to distract the user from the error prone fragility of much of the output? It's perhaps a stretch to think these canned responses were put in strategically, but the result is that the user's attention may be deflected to contemplating their own superior knowledge and insight, and bask in the glory of all that, but then forgot to appreciate that 'Hey, chatLLM is just making all this stuff up/doesn't know which way is up/or down!'
Not sure if there are sycophancy benchmarks for coding agents
It measures whether models push back on bullshit prompts or just go along with it, and Claude models are all the top performers.
I have a Rails background, so maybe KISS is more engrained in my philosophy than whatever training material was used on AI. At least it isn't heavily pushing design patterns...
then you add the simplicity / lessons of clojure of using simple datastructures & functions - simply agents become frustrating - cz most of the things I need to get done are done in a few lines
majority of the time is spent thinking by me to save a few lines.
You all know the feeling: you see a code review from _that person_ and you know its gonna be a long day. And you know they are going to fight you every step of the way and say “but it works” when you leave a comment about their code being hard to maintain.
Not coworkers, but I started getting contributions on public GitHub repos that attempted to close issues tagged with the default "good first issue" label. Got real excited when one project I'm stoked for got its first contribution, until I looked at the PR. The account it was tied to was someone looking for work. Looked like what a model would output for a LinkedIn Job seeker NPC--im sure you can imagine.
Same could be said w.r.t interacting with LLMs on stuff you are an expert on.
The thing is laborious, over-does it, slow and wasteful.
If the "big ball of spaghetti" theory holds, where software companies who can't manage the debt stumble over themselves as they continue to add to the big ball of spaghetti code, I guess we'll see a row of companies declaring "software bankruptcy" or something in some/many months, depending on how well these workspaces learn to care slightly more and get better at pushing back against slop.
I don't think you will, because that would require the business to recognise the problem. That might happen in companies where the leadership team are engineers but it will never happen if they're not.
Instead you'll see:
- Churn in the dev team with senior developers leaving rather than try to deal with the mess
- Large scale projects to refactor or rewrite entire codebases, which will inevitably fail because you can't rewrite a big ball of spaghetti because you can't tell what it actually does (especially if it's in a language that allows side effects, or you've used a strategy like 'exceptions as flow of control').
- Companies just getting slower and slower to deliver anything. That's probably fine in many cases where they're big enough to still carry on without growing much, but anyone in the company will see their career die and pay rises dry up.
- Eventually, maybe, you'll see 'tech debt fixing' service companies start up to leverage AI in the effort to fix these problems. (AWS have a thing called 'Amazon Modernization Lab' that is exactly that, but only for companies running old tech on their services.)
But that's based on "spherical economy in a frictionless vacuum" type assumptions.
In the real world, in addition to the problems others have noted of it being hard to identify and fix the specific sources of problems, we have so much consolidation that it doesn't matter if something from any of the tech giants starts getting buggier and slower. What are you* going to do—switch from Windows to Linux, just because it's getting a bit buggy? Or worse, switch away from Banner, or Salesforce?
We cannot depend on "market forces" to prove whether LLM-assisted coding is actually a good idea. We have to push for universal personal accountability for the code we commit (at least internally; I'm not calling for legal liability here!). Which is, unquestionably, going to be a huge uphill slog.
* where "you" in this case is an average PC user, or a large institution
People call coding agents bad because they don't know the asinine meaningless conventions at their particular company while they themselves write awful abstractions and brittle tightly coupled systems, but hey, at least they know how to write a for loop how their particular company likes.
And how long does it take a coding agent to output a thousand lines of code versus a human? The worst human at any company was rate limited by themselves. Those 'average enterprise' programmers aren't going away, they're the ones now spending tens of thousands on coding agents and filling your codebase with even more garbage without bothering to review an iota of it.
In the past, a team of five mid devs and one good one would be fine, because that good one would ride herd on the mid ones. But now those mid ones are slamming out robot code that they're incapable of meaningfully reviewing (because it's better than they are already), and they're just overwhelming the good dev's capacity.
The solution, of course, is to fire them all -- they're worthless now -- but this is not going to happen quickly, and it's probably for the best that it doesn't.
Sometimes the human is faster.
I've seen someone duplicate a class file (already filled with duplicate methods) rather than subclassing, and when called out on this it was because properties were private.
This was a team with just me and him in it, it didn't even really benefit from things being private.
That said, the really important lesson I've learned over the years is that terrible code and practices are almost irrelevant: this app won awards and was highly regarded.
An average enterprise developer would never add bloat like that up-front, unless if the ability to change the order was a requirement.
Obviously a stable order can be easily derived from the ID or a creation time (if available).
Setting a position however requires extra steps to ensure the integrity of the sequence.
I see things like that all the time, and it's always stuff that grows the code base and adds unnecessary complexity.
I've seen countless vibecoded implementations that look exactly like that. Especially painful is agents adding the same utility functions in each and every file instead of properly reusing or splitting things.
And then I have to fix them.
i've never seen even a junior do something as crazy as displaying a page sheet ui from literally a color object, yes, a literal color...
Why is this worse than splitting it across 1k files?
(Personally my threshold is around 2-5 thousand lines per file depending on what it is; but that's me working solo, obviously I'll follow whatever standards any team I'm in gives me).
I'm not making an argument in favor of people using LLMs for this, but people were doing this before we had LLMs it was just usually a bit slower. I can't even say it usually doesn't work out long term because I worked with a lot of guys who did this and took a ton of Adderall while working practically around the clock. Every incentive structure in the organizations rewarded it along with social credibility from more junior engineers. (The last cowboy I worked with who pulled this shit ended up becoming the most senior engineer in the company, a multi-millionaire and worshipped like a god by 90% of the mostly fresh grads we were hiring).
The problem is when invariably these people burn out eventually and leave, they leave a massive vacuum in their stead. Not from load they were carrying but creating.
I think the larger the organization I've been at, the more they reward the people making huge commits on nights and weekends. Worse, they could get away with TBRing their shit and merging it without review.
LLMs are often all of the bad habits and organizational problems that we already carryied just being speedrun. There are some places doing it right, but they already were.
Could you be more specific what "right" is?
> I can't even say it usually doesn't work out long term because I worked with a lot of guys who did this and took a ton of Adderall while working practically around the clock. Every incentive structure in the organizations rewarded it along with social credibility from more junior engineers. (The last cowboy I worked with who pulled this shit ended up becoming the most senior engineer in the company, a multi-millionaire and worshipped like a god by 90% of the mostly fresh grads we were hiring).
I'm having a tough time believing this, it sounds like you're trying to backwards rationalize more productive engineers were "on drugs" and they delivered but "did it wrong"
https://en.wikipedia.org/wiki/Michael_Crichton#%22Gell-Mann_...
Having worked 20 years in this field and managed a few projects, no, I wouldn't make a dozen mistakes, because I would refuse to take on work I can't responsibly do.
Invasive and risky work IS the thing I want to be working on because it's the place where I can be most valuable, but part of my value comes from asking the right people the right questions. If I'm working on something invasive and risky, I'm going to work directly with the people who wrote it, and only when THEY think I understand it well enough am I venturing in alone.
Absent access to the people who wrote the code, I'm going to start by writing tests around the code and spend a lot of time checking my initial assumptions upon reading the code, because I know that I don't know what I don't know.
Yeah, if I did foolishly just started making changes, I'd make mistakes but that's missing the point: a good senior engineer knows not to do that.
That's the failure point of AI: it's arrogant. It will provide you statements without any idea if they're true and make changes without any idea if they're correct. It will never tell you "I don't know how to do that" or even "I am not sure if this is correct". It just does the work with infinite confidence even when that confidence is not justified and often it will be just as hard to figure out if the AI's work is correct as it would be to do the work yourself.
...ah, what a boon it would be to be working with code written by people still working at the organization!
(No shade, just being wistful; I happen to have a history of coming in and having to deal with some messy codebases from the guy who just retired...)
I agree with your take, but AI is exactly as arrogant as the human driving it.
It sounds like you've not conditioned your Claude to stop being a sycophant yet?
(There are workplaces where that's the norm, I know -- it tends to be a thing with smaller teams with codebases that everyone understands fully, and much less a thing with larger teams where different people have areas of the code they understand more than others.)
With AI code, though, it's _your code_ and you can't give it a lgtm, you actually need to dig at it until you do fully understand it, fully agree with it, and could justify it to a hostile reviewer. It's a different level of rigor.
Not all engineers apply that rigor, though, which becomes a problem.
I’m not saying you must see into the soul of every line, but “no idea what I’m looking at, LGTM” misses the point of code review.
I have never been on a team where that’s okay.
If it’s not good it’s not good.
The problem is this. Human cognitive resources are finite, so we inevitably become shallow outside our own expertise. There is no programmer who can do everything well. And as systems grow in scale, they become more modularized and fragmented, making it impossible to understand the whole system. So what should we do about this? That's always the question.
In the end, do I choose not to use AI, finish the project with uneven code outside my domain, and deliver it? Or do I use AI and deliver a program that is uniform and consistent, but not in my own style? I still don't know. I haven't found the answer yet.
My position is that AI could be useful to find the potential places for these changes, but it should be someone who's capable of thinking to implement them.
For me, AI has been a godsend for productivity because it's great at what I'm bad at. I'm not spending 99% of my day grinding away at C++ code; I'm never writing enough for it to become a world class language expert. I'm jumping between SQL queries, CSS, Java, bezier curves, Python, and shell. If I need to write something in a language I touch infrequently (e.g. Go or Ruby) it's nice to have individual blocks of code generated for me, so that I'm not slowed down by my ignorance on a language's iterator syntax, or whatever.
In the end, an exceptionally skilled programmer might be able to keep their core domain intact, but I think the vast majority would find that very difficult. So it might be possible once you cross a certain threshold, but considering the sheer amount of code required to deliver a single modern program, it's hard to know which parts to focus on. However, my perspective might be different because I'm coming from the point of view of delivering a working program, not from the perspective of open source development
When using power tools you make all the measurements and decisions, you just hammer screw drill and cut faster. You cannot power tool your way to building a things that you don’t know how to build.
The other interesting thing about this is it works with smaller models and uses a fraction of the compute.
In the past, I wrote code by first writing English pseudo-code as a series of self-documenting comments. These would be declarative assertions of what the code will do. (For example, "Method returns true if array values are within 0.5% of spherical.") I then wrote the real code next to each comment.
My current workflow is mostly the same as before, but as soon as I think there's nothing creative left to do, I allow AI to take a pass at it, insisting it include verbose comments. Next I read everything; its comments are often redundant but allow me to internalise the logic/intent more quickly. I make any corrections myself. And I strip any pointless AI comments.
In short, I stay in full control of the architecture while tasking AI with the grunt work, the implementation details, and the superficial correctness.
I use it rarely. I did have it rewrite some code, mainly from one language to another. That works really well. I also had it rewrite a database interface, which also seems to work (no time to test it thoroughly, yet, so it's not in production). But I'll be damned if I let it write new features. I've debugged other people's code, and it ain't fun. Debugging 10kLOC AI code sounds like hell to me.
Pinky promise that's enough to get good output.
Pinky promise we won't invent yet another body of work the whole industry must adopt to get good output.
Pinky promise the AI tool will properly read all your work
And then of course we are told you must never trust its output !? You must review all code it produces line by line and grok it fully !
And now we have: keep challenging it, keep rejecting it, keep interrogating it... That's just fancy words for spend more money (tokens)
It doesn't seem like AI users are very good at telling how much or how well they're using it.
Good ol' software architecture tricks can also help you slot "vibe coded" components into a larger system safely.
I wish it were clearer in these kinds of posts how "I use AI code I don't understand" is so different from "I use libraries written by other people I don't understand", or "I work in a large codebase which was 99% written by other people, and I haven't seen all of it", or even "I use software written by other people I don't understand".
Besides, this post has nothing specific to code produced by an LLM, and placing AI in the stated reasons feels completely arbitrary, or is rather a fallacy of our times:
- I reject [AI] code when I can’t explain the approach in my own words.
- I reject [AI] code when the diff is bigger than the problem.
- I reject [AI] code when it introduces abstractions before proving they’re needed.
- I reject [AI] code when it works locally but makes the system harder to reason about.
- I reject [AI] code when I’m trusting the output more than my understanding.
These are the people who spit out an incredible volume of code with AI, to the point reviews simply can’t keep up.
The last person who said this to me works in embedded, where we look at the assembly all the time. Scary.
If you're given two embedded devices and both pass the same testing, how would you tell which one was 100% AI code and which was beautifully handcrafted line by line?
Also, when something invariably doesn’t work (maybe I told Claude “delay 1 sec after each swing of the axe the robot makes if the proximity sensor trips to avoid the puppy that walks across the ax’s path once every month”, and meant to type “2 sec”), I still have to go down to the level of the code sometimes. I’m sure the counter argument is “well then that just means your testing wasn’t good enough”. Sure, but I’ve never seen any project with hardware in the loop where the testing was good enough 100% of the time. Sometimes it’s hard to test once in a month type events in a regression test suite.
FWIW I hover around 80-90% code AI written these days. I still look at every line of code it makes.
No amount of reading code or auditing or testing gets you 100% bug free solutions. It's possible, but nobody outside of maybe NASA will foot the bill for that.
My point is that why does it matter who or what wrote the code if errors are inevitable anyway? You plan what you do when you encounter one and limit the blast radius. If you find a process that can cut out a category of bugs, you implement it when you encounter it.
Why do we allow human written code to have more errors than AI generated code? Or is it just that both create different type of errors?
I'm more interested right now in what does that abstraction look like for AI generated code. Is there some reasonable solution wherein a sandboxed component in the enterprise architecture has various attributes (e.g. the bytes i stuff into this file store component are always the exact bytes i get back from it) confirmed by methods other than a human reading its code? Those methods, are they cheaper, faster, safer than just having a human do it?
If your enterprise architects have to read every line of code in your system today then i'd claim your architecture practices have room to mature. What can derived from that, and in which scenarios, for the purposes of safely leveraging immutable write-only code? I'm not interested in evolving the code (lines of code spent to solve a business problem was never an asset, it was always a cost) if it wasn't hand crafted by a human, i still have the requirements so i can just regenerate the entire thing with the revised requirement.
You don't look at the code, but use tooling to create a chart of the calls, data models etc. Then you can look at that and see the complexity.
...and we already had these tools in the early 2000s, when (can't remember which) no-code fad was running about. You know the ones where you just draw the boxes and lines and poof the code is generated =) There were also tools that did the reverse.
However if you’re highly familiar with a domain then LLMs are much less useful.
(For as long as that's true, "software developer" is still a job. It's not clear for how long it will be true.)
Meanwhile, those codebases often require a ton of boilerplate and drudgery to get anything done.
In these spaces it's very easy to read and comprehend AI generated output and review it fairly quickly. So the time savings from dealing with all that boilerplate and conforming with all that existing infrastructure are potentially substantial.
Now we are getting to the point where we are speed-running the deskilling of engineers into comprehension debt and they themselves rapidly losing confidence in reviewing code they did not write.
I think this blog post [0] is the best example of what could go entirely wrong and even worse when you do not know the technology.
If you cannot explain a change even when "the CI is green" or "all tests passing", I will immediately reject it.
Maybe great for vibe coding prototypes, but it all changes when that code is deployed onto mission critical systems. Just ask Amazon with Kiro. [1]
[0] https://sketch.dev/blog/our-first-outage-from-llm-written-co...
[1] https://www.reuters.com/business/retail-consumer/amazons-clo...
LLMs are perfect for quick prototypes, speed runs, learning, etc., but if the code really matters its still not clear cut. I think the definition of what "really matters" is very project dependent of course As an extreme example you would want to understand every line of the code for the control system runs an MRI machine or a jet engine since bugs might mean life or death. Depositing money into the wrong account might not kill anyone but could lead to severe economic losses. But, then again, even problems in far less consequential software may be drastically sub-economic (i.e. saving $1000 on the implementation might cost $10000 if customers aren't happy and fails to re new). Pick your scenario I guess.
The problem is, this isn't going to change regardless of how well a new model scores on a benchmark. It seems actually AGI is needed.
Adequate often means done and cheap
Stakeholder needs: What people wants to get done with the product
Management needs: How to manage the spending of resources (time, money,…) to create the product
Engineering needs: What is the product
You have to balance the three. Sometimes it’s simple and easy to get right. Sometimes it’s complex enough, you’re never truly sure until the product is out in the wild.
Software is malleable and we can do easily do iterations which is not possible with hardware. But today, we have a skew towards engineering, where the whole focus is to create a solution, whatever that is. No understanding of the problem, no proper allocation of resources, just do something. Even if it is plastering over the crack for the eleventh time.
It really, REALLY depends what you're working on. If you're throwing together an internal tool or simple dashboard, it doesn't really matter what the code looks like. But if you're writing software that other programs will depend on, bad design choices ripple out and affect another generation of software. Imagine slop in the linux kernel, in google chrome, or in your compiler or runtime. Its not acceptable.
I know a lot of people spend their careers writing end user software and web UIs. AI is increasingly a good choice for this sort of code. But that's not all of us. And its not all of the software being written.
The plan also solves "I can't explain this code" because you wrote the plan before the build, so you can explain it.
After tracking some internal metrics recently we found plan review costs 0.7 hours on average compared to PR review that costs 16 hours. We rejected 13 out of 165 plans meaning no code was written.
The one gap this doesn't close is that the agent drifts from the plan. We run a separate adversarial check that compares the diff against the approved plan and flags anything the plan didn't specify. That catches scope drift without reading every line.
I try to make sure the architecture docs of the code base are refreshed regularly based on recent changes, so it's easier for humans and AI agents to make sense of the code.
I also regularly stop all other developments and just focus on auditing the code base with these AI's to make sure they are secure, robust, clean, and well structured and well tested -- some refactoring would be needed most of the time, and it's well worth it.
With this approach, nowadays I often merge code from AI without completely understanding what it's doing, but seems the code has been working so far. :)
I do sometimes have to steer the discussions between the AI's to the right direction, if they deviate too far away from the real problem, either because they miss some context, or because my original description of the problem was misleading.
To do that formally, I have a mechanism built-in the review loop where if a comment on a github issue or PR is signed as "-- Human Reviewer", then all AI agents have to treat the comment as the highest priority item to address.
I am still skeptical on this method's ability to deliver polished products though. I've kept an eye out on it in the OSS world and don't think I've seen big anything yet.
Each implementation is also reviewed by me before merging to master. I complete PRs only when I'm satisfied with the implementation, my feedback is addressed, and I fully understand what is going on. Agents are the replacement for typing and productivity multipliers.
I have big picture view of the product, each plan implements only a part of it, scoped to avoid merging unreviwed slop. Probably slower, but result is much better.
Being able to step back and say "this was a failure and we need to discard the day's work and start over" is still hard with LLMs.
It's like if I 3D-printed something I haven't modelled myself and the print goes wonky. I don't spend days trying to glue and file it back together. I chuck it in the bin and start a new one.
But if I had handcrafted the same item over multiple days, of course I'd try to salvage it - because there was a sunk cost of me spending time doing it.
But with the agent, you know that the change will be relatively quick and easy, so the bar to tell it to shift approaches is much, much lower.
> I reject AI code when I can’t explain the approach in my own words.
I think that's the key problem. LLMs turn code into big, black boxes. Sure, theoretically nothing stops me from reading all that code. I don't, however, because it's wasted effort. The time it takes me to really understand the code is IMO better spent just writing it myself. Once written, I have a very good understanding. Read ten times, not so much.
It reminds me of pen and paper. Journaling the old way remains the best way to learn something, but writing on a computer is much more convenient.
What I'm hoping to build ultimately is something that works more like a pair-programming partner than existing harnesses do. I want the user to be an engaged part of the development process all the way through, I don't want the agent disappearing to work on its own. I even want to make it possible for users to swap into the driver role and have the LLM automatically assume the role of navigator when that happens.
There's more info in the readme (actually the readme is all that exists so far, I wanted to get the idea straight in my head first):
https://gitlab.com/philbooth/opair
Even if nobody else uses it, I hope it will be a useful tool for myself and help me find a way to work with LLMs that doesn't harm my mental models, which is what I feel current harnesses do.
Suppose you were legally liable for your code misbehaving in a way that led to harm. Would you behave differently?
And do you do this by choice? Or is this the case of an employer forcing you to vibecoded while skipping your due diligence as the author of that code?
I know that's not your call but IME it's simply not true: rarely do products win by simply being faster than their competition at delivering more features to market.
But the AI age has led to a panic among leaders as FOMO has taken over the industry. I can only hope one day that fever breaks.
I'm not optimistic.
Anyway, we're in this sh.t together so stay strong, keep your head up, and try not to compromise your ethics. The industry is seriously f.cked right now and it's going to be a rough ride for a while...
The last ones, I worked on in Industry are retail7 apps, Migros Self scanning client, EDEKA, LIDL and so on customer facing apps.
My private interest is more in electronics.
Thanks for sharing!
What I found myself doing is operating in two modes: 1. For projects that require my attention, I plan and instruct LLM, when needed will draft some code and ask agent to make it better or finish the mundane part (write code and leave gaps with comments asking agent to finish) 2. Full automode where I use spec driven development and TDD - I only ask for changes based on existing PRD, which agent also have to update. Here I do not look at the code at all.
Seems to be working just fine.
And the industry is rushing towards it, whilst failing to train people who are able to fix it
When implementing its often a lot of misses with a few golden hits. The other day it used flex for a table layout while our app uses tables everywhere sigh.
Another typical one is that it tends to prefere frontend aggregation and looping of data instead of letting the database and backend deal with it.
Using mix of claude, cursor composer and codex.
How do you verify that it works?
json='{ "left":2, "right":2 }';
result="$(
perl -e '($_)=<>; / "left":(\d+), "right":(\d+)/; print $1 + $2, "\n";' <<< "$json";
)";
printf '%s\n' "$result";
printf '%s\n' "$(( 2 + 2 ))";However, if AI provides a solution, as the person using AI, one should conduct research before making a decision. This is not in conflict with or hindered by the use of the ideas provided by AI.
The obvious counterargument is "well, just ask the AI for those answers," but the AI lacks the context and experience that you have. Sometimes, genuinely, the user really is just "holding it wrong," but none of the current AI models would ever admit that, and you'd spend hours trying to solve an unsolvable problem.
For example, I use a vibecoded internal tool written in Go. I don’t even know how to write Go. Haven’t read a single line of the code. I just wanted to move from bash scripts to using cloud SDKs for performance reasons.
But the internal tool is a convenience tool, and you can do everything it does using alternative methods. So if it break, there is no real negative impact besides personal convenience of anyone using it. There’s some documentation on how to do everything manually if needed.
Here’s another example: you’re making a static website. No JavaScript, no interactivity. Truly, what could go wrong? And while I do understand HTML a lot better than Go, it wouldn’t really matter if I didn’t.
What is this supposed to mean? How is a “cloud sdk” more performant than a shell script?
There’s a bit less waiting around.
Linking a huge file consuming clients’s bandwith for no reason. Embedding PII in the html source? And if setting up your own server, misconfiguring it?…
You also don’t need to know how to read HTML to recognize large files. You can catch issues like this with a simple website performance testing tool like pagespeed.web.dev
I’m also not sure how PII would enter the HTML source.
Human developer can work on a program incrementally, ensuring at each step that it is mostly correct.
But LLMs can't think, they fake reasoning and explore problem space in random walk until they stumble into something that looks like a solution. And these "solutions" will have hilarious and absolutely unexpected failure modes.
TLDR: Keeping your codebase human readable and reason-about-able is not just helping humans to stay relevant. It will save costs for LLMs to maintain it.
If you use AI at the very start of a project, replace slightly with greatly. AI loves to write abstractions and indirection and add complexity wherever it can. And it does so really, really, really badly. AI is great at writing procedural code, but it's a world class shit-for-brains at architecture. It has no taste, no restraint, no appreciation of simplicity.
And it wouldn't be so bad, except it's ALSO a complete toaster when it comes to naming things.
Agents respond really well to feedback! They have no ego and they’ll happily improve code if told where and how. But you need to provide the tools that provide that feedback without your involvement - otherwise you can’t scale.
All the linting and autoformatting you can put in, is a good start. Next, create custom scripts that check for every single dumb AI-ism you can think of, tell the agent about them, tell it to use them to check its work, and put them in hooks so the harness refuses to let the agent stop until all your linters show no errors.
Then, keep iterating basically forever. Any dumb AI-ism you see, make a linter for it, give it to the agent, and enforce it using the harness.
I’ve spent months doing this. When I review a PR - which was built by the agent with TDD so it definitely works - I’m no longer asking if it did dumb stuff or confirming it conformed to the architecture or duplicated code or missed opportunities for reuse. That’s all linted for. I don’t worry about duplication or outdated docstrings/comments because the self review caught all that. I mostly read it to look for opportunities to make the feature even better & more useful.
If this makes no sense or you disagree it’s possible, my contact details are on my profile and I’ll be happy to give a demo.
Incidentally I also don't understand the drive to scale up. Show me a successful tech company and I'll show you a company that won, not by delivering code the fastest, but by delivering the right product with the right features at the right time.
Hell, Anthropic itself is the perfect example: they're doing well because unlike their competitors they realized the real revenues come from enterprise not consumer. They're winning by identifying the right market and giving them the right product.
Then have a look at https://github.com/cadamsdotcom/CodeLeash/blob/main/scripts/... (which was test-driven alongside https://github.com/cadamsdotcom/CodeLeash/blob/main/tests/un...)
The script can exit 2 to block the agent, and whatever it prints to stderr is shown to the agent. That’s a pretty darn flexible way to enforce whatever you like.
Despite this being in the codebase I still have no idea what python’s ast stuff is or does - I just let the agent rip, ensured it did TDD and reviewed it all to make sure the tests & code looked reasonable. I didn’t write this code and don’t want to. But I’ve watched it catch hundreds of dumb AI-isms, and watched the agent go “okay” and fix them ;) it’s been paying for itself over and over for months :)
"TDD" isn't some magic trick. The tests codify the expected behavior. But if you don't review them for correctness, if you let the LLM build them blindly, then you have no idea what those tests assert and can make no claims about whether the code then does what you expect.
That's fine. That's your choice.
But you have to acknowledge you've chosen to accept that you personally cannot vouch for the quality or correctness of that code.
I fully expect this to be the direction the industry goes, where increasingly complex systems exist that no human actually understands or can reason about.
I think it's bad for the industry. Very bad.
But I'm not making those decisions, so... it is what it is, I guess.
I design everything with plan mode and review every line. Nothing happens to my codebase that I don’t decide should happen. With my way of working, tech debt doesn’t exist because I never have to create it.
You’ve made a bunch of assumptions you’re not conscious of. And now you’re blaming me for that.
Open your mind, you never know what you might (un)learn.
The thesis of the post is (paraphrasing): "if an AI wrote it, and I don't immediately grok it or if the code quality is low, I throw it away, even if on the surface it seems to work, because simply 'working' isn't enough to say a piece of code is acceptable."
I'd add as a corollary "and therefore I would never want to be accountable for that code."
If you're reviewing every line then it sounds like you have no argument with the writer and I don't understand what your point is.
Your very first paragraph says:
> If you reject AI code that works then your mindset is still too hands on. Put another way - you still have some loops to work on taking yourself out of.
But if you do indeed "review every line" then you seem pretty damn in the loop yourself and I don't understand what you think taking oneself out of the loop is.
The comment was motivated by the complaint that first-draft code from an agent can be brought up in quality significantly with a little bit of engineering.