Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick.
If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.
Edit:
Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.
The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.
What you point out is slowness once you hit the entry point. Go, or similar languages, as a server language platform could have solved that problem from a computational perspective. But it did not for the most part. In my opinion people choose the faster stuff because it's cool and they have more wiggle room to cram in to get back to the slow status quo.
Everything is overengineered, software or distributed architectures, sound to naive human logic but alien to computers. It's an cultural problem, development is so deeply entrenched into "business logic" that the minimal viable and computational economic solution isn't even on the table. I don't even think it has to do with cost or feasibility, it's just that your random e-com manager wouldn't know what to do with you, if a programmer really starts talking about hardcode tech stuff.
Things like this really pushed me to everything local systems. I’ll move actual files around if I want to do anything on the network. Or sometimes even use cables! Shock, horror!
I have been using the proprietary NVIDIA drivers for more than 2 decades on various hardware, both desktops and laptops, mostly with Gentoo Linux.
Opening an OpenGL context or any other OpenGL operations have always been instant.
In similar style I recently wiped a device that I thought had firmware that was slow to boot but it turns out that a hang and subsequent timeout due to something I had long ago misconfigured had been obscured by the previous setup that defaulted to hiding all details during boot.
Try turning off wifi and see if it improves.
You are starting to get software with ticket driven development layered around glue code for existing libraries. I see no problem with libraries, it is just the architecture and vertical understanding that leaves a lot of performance on the table, because refactoring insanity takes resources and a lot of talking, understanding and convincing to be done.
Doing this across big teams starts to have downsides. So one team doesn't have a particular use case implemented or understood it and does not want to support it and you run code to compensate for this.
Best example in a monolith case is oracle...
"Do not fall into the trap of anthropomorphizing Larry Ellison. You need to think of Larry Ellison the way you think of a lawnmower. You don’t anthropomorphize your lawnmower, the lawnmower just mows the lawn - you stick your hand in there and it’ll chop it off, the end. You don’t think "oh, the lawnmower hates me" – lawnmower doesn’t give a shit about you, lawnmower can’t hate you. Don’t anthropomorphize the lawnmower. Don’t fall into that trap about Oracle."
I feel this is so much real. Like if there were two kinds of companies: the ones who deliver, they care about their product but most of all they care about their customers, the managers get their hands dirty and everyone pushes towards the same direction; then there are companies in which you open a ticket and wait for two weeks for something that should take 5 minutes, customers and product don’t matter because you’re focused in cost attribution and no body does anything if it doesn’t come in your JIRA board, the managers are all coming from consultancy companies and all they do is finding someone to blame.
The problem usually comes from inappropriately arranging the systems of record such that information needs to be communicated beyond the scope of one computer in order to satisfy a single logical request.
Moving information between physical processors tends to be significantly more expensive than local computation over that same information. JSON serialization is a really good example of this. You need a network with bandwidth in excess of 10 GbE to begin overtaking simdjson.
SSR or SPA doesn't really matter if the server still takes a minimum of 300ms to compose any kind of response due to how its database or other infrastructure is set up. Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful. Stringing the user along with psychological tricks is a lot cheaper than hiring a skilled developer to do it the right way.
We've come full circle.
* Must feel very responsive * Network requests can take up to 500ms end to end
This gave me a chuckle because I personally hate things like watching the browser jump through 50+ redirects when logging into a website.
With a fast backend ~1-5ms response times (not even that fast). Streaming compression over something like SSE to keep your response sub 1kb packet (roughly an ethernet MTU).
With a push based model, pushing data to a user is half their RTT latency. They will only experience their full RTT on actions they trigger.
Now the network to you is distance to the server (not your rail/nextjs backend taking 400ms). Things like 4G and 3G are fine. The real problem is when you have such bad signal you effectively have no down or up.
When it comes to improving performance by a few ms, or implementing a new feature, business people will always choose a new feature, unless the current performance is unbearably slow (we're talking regular 1.5s+ wait times for BE response).
And it's not even a modern problem, legacy software written 20 years ago has the same latency than most modern backends from my experience.
Even though benchmarks suggest this sort of performance should be trivial, most real-world servers I have interacted with do not reliably managed to process a request, make a roundtrip to the DB, and return a response in <5ms
Don’t know about that, pretty much everything is hosted on Cloudflare, Azure or S3 these days and all of them have at least one CDN on each continent.
I'm making a simple plugin for Gimp that sends the active layer to a model with a prompt; on Macs by default the UI waits for the request to come back or timeout; on PCs it doesn't, so the modified layer appears unexpectedly. The Mac experience is better IMHO and I will replicate it in the PC version rather than the other way around.
The optimisations the article is talking about would help even with this problem if backends responded faster - although not as much as actually avoiding unnecessary network requests in the first place, of course.
I wish Atlassian listened to you.
second is on a smaller timescale. so many features and crap that is never asked for and never used is crammed into software so the systems that execute it are just juggling pretty much dead code in and stale data in their caches all days long.
https://nextjs.org/docs/app/guides/prefetching#hover-trigger...
In my view, websites should not take seconds to load with gigabit fiber. Whatever happened to "mobile first"?
I don't think the current crop of fullstack engineers would be hard pressed to know what a "markov chain" is, but in theory yes, you could emit a bunch of speculation rules[1] based on your predictions.
I should also say that markov chain based approaches have been used for fraud detection, e.g. identifying checkout anomalies by detecting the sequence of web pages that they clicked on, amongst other factors.
[1] https://developer.mozilla.org/en-US/docs/Web/API/Speculation...
This might seem wasteful, but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet. Which sort of recasts it not like wastefulness but instead like fault tolerance and preparedness.
We could maybe even dispense with the servers entirely.
There are so many different ways to build a web. Why does it feel like we've landed on the worst possible one?
This is already possible without content addressing with CDNs. They can serve content from a local cache even when the host is disconnected from the internet.
Welcome to Next.js
The .js part makes me think not.
<script type="speculationrules">
{
"prefetch": [
{
"source": "list",
"urls": ["/checkout.html", "/thank-you.html"]
}
]
}
</script>Obviously agent access goes away with internet failure but the state doesn’t need to… we use CRDTs and a virtual FS. There’s a toy-ish version of the harness at https://ourhearth.ai … if local first is interesting to you I’d love your feedback
https://github.com/eaftan/safere
https://eaftan.github.io/safere-intro/
Mine is for Java and is intended to be production grade. The first goal is to guarantee linear-time behavior to prevent ReDoS attacks. My collaborator and I have recently been optimizing it to try to surpass native RE2 in performance.
It turns out optimizations are incredibly well suited for an agentic loop. You've got concrete acceptance criteria (must show a meaninging improvement on a benchmark case, must pass tests). The agent is really, really good at using tools like a profiler and disassembler, better than I am (and I've been doing this for 20 years). It also papers over things that would take me a while to learn, like how the in-incubation Vector (SIMD) API works in Java. I understand the concept but it would take me a while to understand Java's implementation. The agent can just read the docs and go.
The key is creating a good benchmark suite and ensuring the agent doesn't ship optimizations that are too narrow or too focused on the benchmark cases. You also need a really strong test suite to make sure you're not regressing correctness. SafeRE has billions of tests; a subset of several million run on CI, and the others run on-demand.
One little thing I spotted is you use Boyer Moore Horspool for fast literal search. This is actually not linear in the worst case, although it is almost always sublinear. Worst case would be a literal composed of the same character searching a text of the same character, where it becomes quadratic.
You can actually search strings with character classes using Horspool if you want to, and I have some enhancements to basic Horspool which could maybe help. My library, byteseek [1], implements these.
I also have a much faster algorithm, HashChain [2] which also has a guaranteed linear time version. This was published in the Symposium for Experimental Algorithmics in 2024.
My Java profiling knowledge is... let's call it "antique". I was really not looking forward to ramping back up for this work. Turns out, I didn't have to do any of it. The LLM chose the tools (flight recorder) and even built a JMH (also new to me) harness to experiment with different algorithms.
About half of the optimizations were things that I would have figured out on my own; the other half were definitely "wow" moments.
The whole thing was done in a couple hours, with just a few back-and-forths. Sans AI, it would have taken a week, with nowhere near the same gain. I'm impressed.
"Figure out how to make this process fast" is really a perfect activity for LLMs. And the prompt doesn't really have to be much more sophisticated than that.
do you have some samples? It would be interesting to learn what it might be.
JMH as the framework to write microbenchmarks. It takes care of dealing with JIT warmup, etc. It's the standard way to write rigorous Java microbenchmarks.
async-profiler (https://github.com/async-profiler/async-profiler) for profiling. Java has a problem where many profilers are based on safepoints, which are biased toward particular program points. async-profiler is not biased in this way.
Java Flight Recorder for memory allocation data.
One thing I've observed in all of this is that it's really useful to have expertise in the programming language and ecosystem you're writing in, otherwise it's all Greek to you and you can't really guide the agent to do the right thing. I have opinions about e.g. profilers and I can point the agent to one that I think is more accurate than other options.
It works, it’s at least 5x faster, sometimes much more, and memory use is like 10x less and even less in cases where lots of map tiles are involved.
This shit rules.
What I’m wondering now is can we reliably evolve python and have codex act as an extremely unreliable transpiler to the rust.
Why you even start with Python at this point? Just write the Rust version straight up instead of porting things?
Personally I used to use dynamic languages for most things, because development and maintenance is so much faster and easier, particularly for larger projects (granted you know how to work with those sort of languages), but now when the LLM writes most of the code, I'm able to work as fast with Rust as with I used to be able to do with Clojure or other dynamic languages.
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke) The only novelty is that the proposer is now way better with LMs.
Further, there's a large number of reasons for software written by agents to be slow:
- LMs still don't do data or hardware-oriented design well out of the box, and therefore if you're engaging in any sort of serious novel work, beyond porting an extremely well-understood program with extremely well-understood workloads, you're going to be spending hours tracking down bad allocation decisions (c.f. why TigerBeetle doesn't use agents), which are often the root of evil (before you'd reach for anything further)
- The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
The reality is to get performant code (quickly) out of an agent, you need to know how to write performant code (and you need to know how to surface the information that you'd use to create a verifier for such a thing to the agent), which 99% of developers do not know in 2026.
Sure, agents can teach you how to do this -- but it's one of these things where iykyk.
Experience: I've poured 10s of billions of tokens into Zig with the best agents and I have the time and space to try these things.
If you want to start learning the discipline, I'd recommend matklad's + TigerBeetle blog -- as well as hardware-oriented design.
Hasn’t been my experience at all. The latest LLMs can knock out assembly optimized subroutines and benchmark 100 different variations faster than I ever could dream of.
To correct those behaviors, you're going to write tools and skills, and that's going to help, but it is still clear that you are fighting the distribution (today).
If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says.
Granted, this does require you to not use obviously dumb models, like anything you can run locally today, and at least within reasonable range of SOTA models. But they been able to do this for 6 months or more at this point.
A story.
I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a clear objective function, and chatgpt could look at other projects for ideas on how to optimise.
He proudly boasted that his resulting code outperformed my diamond-types library. I asked him if he was comparing against the native implementation, or the -Oz webassembly build, running in a wasm vm. It was the latter. When he tested it properly, his CRDT was - and is - significantly slower than diamond types. As far as I know, chatgpt still hasn't been able to catch up. I tried myself using fable. Even with reference to my source code, Fable still doesn't understand what I did in diamond types and why. (... Maybe I should document what I did!)
I think his technique itself is solid though. I tried it myself. I asked fable to write a custom binary serialization format & parser. Then optimise. Then optimise, with explicit reference to existing libraries. Optimising with reference to other code made a huge additional difference. It is now nearly as fast as those libraries. (But still not faster than them.)
My takeaway is this: I think LLMs are exceptionally good at reading and understanding code. If you guide them to do so, they're good at profiling and benchmarking. But it seems like they're not very good at coming up with novel optimisations. If you have an obviously slow program (for example, some slop claude wrote), you can often get big speedups by asking it to benchmark and optimise. But if you have a complex, already well optimised codebase, like the zig compiler, claude doesn't seem very good at figuring out novel ways to improve things on its own.
This is good news for the 95% of slow software out there. But bad news for the 5% of us who write fast code already, but want our code to go even faster.
> Maybe I should document what I did!
Please do!
Me and a bunch of friends worked on a project that used diamond-types as the backing CRDT engine. It does indeed go brrrr. But damn, it took way too long to reconstruct what it was doing (we needed some more fine grained knobs, so we were playing with the frontier directly). We eventually moved out to something a bit better documented, which was a real pity. I really liked the general architecture and simplicity (of the text-only based version at least).
Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.
TBF as an earlier commenter noted it's "just" superoptimization using an LLM as the proposer so there's a lot of relevant prior art.
A story for a story. I had my CRDT implementation in libdog, which does per-token CRDT weave/diff/merge over a DAG of git blobs. It was written by Claude 4.8 I believe, in several iterations. It was, as you may guess, a piece of neuro-slop that passed the tests by some miracle. Once I had some time to look into it, I used a trick: I supplied it with my article on Chronofolds and some helpful kicks in the butt. It implemented everything correctly on its k-th attempt, k<5. Then I used it with full intensity for three months without thinking twice. Now I have started mass-using it to resolve permalinks in the code. Like, tens of files to chronofold-ize per one commit. It is now showing up in the profile, so I may look into it once more.
Conclusion: it sort of expands its context and prompt by association. It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why. (And how can you guess if you have not had the experience. We all grew with Commodores and suchlike. I recall Spectrums, Robotrons, and Poisks. My friend sits on exam committees, says the youth arrives flatlined after 3 years of GPT. His words. Whatever.)
This is a perfect description. Last week I asked claude opus to get AAC audio working in davinci resolve on linux. It managed to add aac in mov and mp4 containers very quickly. But mkv was another matter. For mkv files, resolve doesn't use ffmpeg. Instead, it has its own parser. Claude got totally lost down a weird rabbit hole trying to add aac support to resolve's mkv code. It was really struggling. Claude even knew it was lost - it kept telling me we should cut our losses and I should just release aac support without mkv.
Eventually I gave it the executable for davinci resolve on mac, which has aac support. Claude found the corresponding part of the code for the mac version and used it as a reference. Turns out, claude had made some much earlier mistake. Just like you said, it was going down a wrong path. Then it couldn't stop itself, and it kept making it worse.
Using the mac version of the binary as a reference, claude figured out how to get everything working very quickly. But - I'm left wondering. Maybe my real mistake was using Opus and not Fable. I wonder if fable would have been smart enough to figure out the mistake and course-correct.
Reasons this doesn't follow: (1) Benchmarks never match real world use, and many optimizations the improve benchmarks degrade cases that aren't measured (think about how CPU cache behavior can be surprising) (2) In software performance optimization, frequently there is significant noise, from many sources. This makes it difficult to guarantee that a measured change is actually an improvement.
This person doesn't understand how to make efficient code. I can write code in almost any language (with a couple of exceptions) that outperforms "super-optimized assembly". Writing efficient code isn't about the language, and often isn't about the best algorithms either (but sometimes it is). Its about optimizing memory and cache use. And that's orthogonal to anything the author is writing about. Also, LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
The author is one of the most knowledgeable people about performance there is
The app and service are measurably and subjectively faster. Because I chose to have the LLM focus on solving those problems. It obviously can. It described the issues in big-O.
It is a priority problem, as it always has been, not a knowledge or skill problem, like it always has been.
> The other day, I saw a viral tweet saying [...]
I've already posted this elsewhere, but here it is again: https://news.ycombinator.com/item?id=49226923
A vibe-coded OS that runs on an 8088 with 256KB of RAM.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
"It's not the tool, it's how you use it..."
And that software is build by some of the highest paid software engineers on the planet with full access to all the LLM compute in the world.
Toxic corporate culture can go a long way to self-sabotage.
I think the "maturity and humility" phase of the LLM hype cycle is probably still 2-3 years into the future.
And that phase probably comes immediately after the "LLMs overinvestments have caused a massive global recession".
Big, feature rich, built in quick iterations - but also, in particular if you look under the hood, of extremely poor quality if measured by traditional software engineering standards (code structure as exemplified by the leaked Claude Code source code, resource usage, "buggyness" etc.).
I’ve also worked on projects where the people in charge added 1mb client-side mapping libraries to render a static map and ignored my push back. Those website were slow
I've been really, really impressed with how effective this is. I went from a 4s load on simulated slow 4g to ~750ms: https://image.non.io/speedup-graphs.webp
Side by side vid of the results: https://video.non.io/speedups.mp4
This was for https://non.io, which is something I had purposefully written to be as fast as possible (hand wrote all the comopnents, didnt even use react).
I've been considering creating a skill / utility to do this based on learnings from the speedups - would others find this kind of thing useful?
Maybe hugged but feels really sluggish to me for it is.
Should note the server is on USW and I don't have edge servers for it at the moment.
and then you threw it all away by adding transition animation
That seriously needs some optimising. For example: on click, could the bigger image be inserted behind the small one so it's hidden; on load of the bigger image, hide the smaller image; then do the slide-bigger transition with both images together?
But all these things will change in time. Hell is other people's software.
There is no technical reason that animation should take that long.
Someone said, "good enough" and let it be visibly slow. The reasons for that could be anything, including non-performant code in dependencies, written by other people. But it just does not take much computing power by today's standards to composite code-generated animation.
Somehow, despite year after year of percentage-speed hardware improvements, there are cultural and structural reasons people ship code visibly slower than it needs to be. And faster compute appears to be irrelevant.
The technical reason is you will not see all the animated beauty if it's fast, so animation forces feeds min delays for states transition
Architectural decisions have always been important.
Legacy baggage has nothing to do with it, in my opinion.
Though I must admit you can have an absolute shit ton of ridiculously complex objects in a scene, and it will run smoothly if none of them is in edit mode which is at least fairly impressive.
I disagree with the premise that this is the desired outcome. If every piece of software is bespoke and everyone’s instance of it works slightly differently, then it’s impossible to get support or a shared knowledge of how it works. There’s no “just share the excel file”, there’s no “press the this button on the left”, there’s no “oh I use program X to solve Y” (instead you have to know what you need so the custom software can solve it, but my time in startups taught me that most users don’t know what they want or need).
If only we had a sufficiently smart™ compiler, it could do the inkling, dead code elimination, and WPO across the application. (Oh we do have that, as long as you don’t write in JavaScript…)
Maybe if this philosophy / approach has a catchy name it might be more widely adopted and we all can have better more performant web and apps.
I somehow managed to avoid the entire SPA scene. I feel so vindicated because I am absolutely zooming with my tried and true Vanilla JS these days. I have tried React, Vue, Angular, and ever just plain TS, and I hate them all.
That still overkill?
I find it decently fast out the box and possible to manage as it grows.
You can't exhaustively write down all the possible ways for software to execute, especially since many programs run forever (e.g. servers, GUIs etc)
So when you say in your spec "when the user clicks here it shows this stuff" there is the tacit expectation that this will always happen and isn't dependent on the entire execution path up to this point. You also wouldn't expect a server to just finish execution after processing a request because you forgot to write "finally, cede control back to the main loop" at the end.
These assumptions are the framework. Almost everything you read is likely written in a framework, unless you limit yourself to non-technical writing for younger readers.
You could try to formalise the framework in your specs. But you'll then end up copy and pasting these paragraphs every time you want another server program. These context frameworks will probably get names, just like today.
Or you could skip that entirely and pretend you're not using a framework, but you are. It's just that the most has already learned tons of frameworks. What worries me here is stagnation. There's no way for the model to learn new frameworks. But this might be a short term effect. Maybe software in the next few years will look quite odd to those in the future as we keep writing it in the language of frameworks we already know. Maybe they'll look back at these relics and ask "what is a view, and why did the model understand that?"
I currently have one running working on a multi path problem trying to make an A* heuristic that produces the same or better score for search time, paths crossed, corners taken (plus a few other metrics, like proximity of opposite direction corners of unconnected paths.)
I have another where I'm testing how well it can iteratively rewrite the same program without regressions and successively stricter resource conditions. Instead of progressively adding lines of code to make an over engineered behemoth, starting over with the previous version as a template of functionality lets it consider the problem increasingly holistically
So going on a tangent: an additional mechanism you have that becomes feasible with LLM assistance today is proving your code correct.
Off-topic, but Dan, in the name of all that is holy, throw a `max-width: 40em;` on your blog!
A crude example is how, when building a large forum project years ago, I had a checklist I ran on every new http route: authentication, authorization, validation, etc. just because it's trivial to forget one of them. Yet now the LLM can come up with a lintable/verifiable way to ensure every route has the whole kit by construction.
Almost every way I feel like I can provide unique value to the automated AI workflow (aside from deciding what to build at all, and taste) feels like it can be automated by better markdown.
I paid for a wide screen monitor, let me use the whole width of the screen.
Whether it can apply it directly from the codebase, I’m unsure and depends on many factors.
Like we have OLAP and OLTP, there are still batch jobs that need to happen before you can see your dashboard, we have multiple users using the same resources, there are still DB Locks when writing needed, there are "first load" issues
— heck now we have to compete for resources with even more bots than ever.
Transferring 10GB or 1TB of data still not going to be instant laws of physics still apply.
You are a rust expert. Make only 5 mistakes.
:) Interesting quote though - I guess one does get to decide how much testing/debugging effort one wants now
Love the concept of agents adding epicycles to fix bugs. It maps closely to my experience.
Do you not add sleep to your programs?
The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.
(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)
I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.
This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).
I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)
The point is: Code needs to be maintained if it is to be relied on. I haven't heard good things about LLM's on that front.
Not sure how either of those things are conducive to performance
High end Windows 10/11 machines (like my Alienware I9 w/ 64GB mem + RTX4090) seem a bit sluggish. Low end machines with Windows are absolutely painful to use!
I've install debian onto my fairly ancient Lenovo X201 Tablet and it is exceptionally snappy.
That's probably one of the biggest sources of software slowness, and we can just stop using it now, with almost no trade-off. Neat!
Woke up today with 2x improvement and fully tested.
The code optimized was mostly written by humans over the years. That code was too cumbersome to optimize by humans so we never touched it.
- Your boss/most of the market doesn't care if your software is slightly slow
- Your boss/the market wants new features
It's easier now to slip in fun optimization features when no one's looking, but the incentives are still off.
If your boss doesn't care about your software being slow he's incompetent.
https://services.google.com/fh/files/blogs/google_delayexp.p...
https://business.google.com/ca-en/think/marketing-strategies...
https://medium.com/ft-product-technology/a-faster-ft-com-10e...
https://www.pingdom.com/blog/page-load-time-really-affect-bo...
It’s often been hard for me to make coworkers care as well which is even more frustrating.
Unless your job really prioritizes performance and builds in benchmarking people often focus on finishing a feature as fast as possible and ignore performance (and accessibility, bugs, UI polish, etc.)
It obviously depends a lot on what you do, but often that doesn't matter.
I haven’t seen as big of an impact on the “good” side. People with attention to detail and craft can use it with great results but they’re not pumping things out at the speed of the slip slingers.
Anything else kind of just feels like "make GTA 6, make no mistakes". The ability for you to write fast and correct software is really going to depend on well you can specify your problem, and having the ability to specify correct programs in such detail an LLM can reasonably performance tune it is a big enough reason for me to believe there will still be slow software
the OS makes a huge baseline difference
LLMs operate in abstractions. "Reasoning" is bolted on.
This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not only do people believe this, even if they didn't, they would still have the problem that most people (including many engineers) are bad at telling who is a good software engineer.
So this means that code will get worse and worse; less performant, less reliable, less secure.
It will be impossible to fix because nobody would know what the problems are let alone identify who is able to solve those problems.
I don't know how bad it has to get before people understand we have to rebuild everything from scratch on much more rigid guardrails.
The essence of the problem is that the LLM does exactly what you tell it. In the hands of a skilled engineer who understands the project, this is a superpower. In the hands of a junior, this is dangerous... Because it will do things that are insecure; it doesn't push back on a user request; it just goes ahead and tries to make it work, even if it's a bad idea and the problem is best solved in a different way.
Not even close to being true. Prior to the advent of LLMs writing good SIMD was hard. Writing good SIMD for arbitrary tasks was _extra_ hard. Having the entire application be written with SIMD in mind was nigh impossible, from simple string parsing to data pipelining. Having arbitrary OpenCL/CUDA offloading to GPUs was a monstrous task. And even the people who were experts in these domains didn't have the time to do it in an afternoon.
But now you can. You can literally instruct your agent to write SIMD accelerated code everywhere, or to optimize down to it. And it just _does_.
> The essence of the problem is that the LLM does exactly what you tell it.
This is correct. I implore people to try writing a SaaS with an LLM, but instead of going the classic agentic project style of "design me an app", tell it to use C++ or Rust, use extensive SIMD intrinsics and inline assembly wherever possible. Just off those two instructions the difference in code quality is staggering.
Architectural problems are particularly difficult for it. There are many moving parts and many ways to implement so if you want certain properties like scalability and idempotent operations, then you need to have a very clear idea what to prompt.
The other day I used Claude to improve my Rendezvous hashing (with skeleton/tree) library and it managed to solve the problem. It's a pretty small library but it used up over 50k tokens and several hours to implement the optimization and it did a good job ultimately but I had written a lot of tests for it so it's not like it was implementing from scratch.
At one point it spend at least 15 minutes trying to change the code to make a test case pass to save a few meaningless percentage points of performance... After seeing it struggle with this, I stopped it and told it that it should just increase the timeout of the test case by a few milliseconds. Any engineer would have done this but Claude didn't want to take that initiative. Instead, it was starting to over-engineer the solution just to dogmatically meet that test case instead of accepting the negligible loss of performance.
See https://deepseek.com/harness/en/
Is anyone really impressed by this gimmick anymore? Just give me a blank HTML with <table>. Its fine. I dont think anyone care.
Fun story, I was writing some 3D graphics animations for a website, and the only dependency I allowed was three.js. It ran well, but the site was loading slightly too slow so I told the LLM to optimize. It essentially hit a hard wall and told me something along the lines of "I can't optimize further, three.js is the bottleneck due to blablabla, but you don't need it. If you want I can rewrite it with pure WebGL". I said sure and load times were cut by like 90% and framerate went up too. Staggering.
I think engineers building very complex systems now have a lot of performance knobs to twiddle that would have just been too costly for human effort. Since we constrain the responsibilities of the agent slop is less of a problem. We relegate it to defined tasks with clear API boundaries and test harnesses.
LLMs have caused such an increase in ram prices that pine64 is no longer making linux machines. When you can't afford RAM, as a direct result of LLMs, it's going to make your assembly app slower too. Sorry, this is the future you chose.
They could have worked in 8GB of RAM but chose not to, and now the rest of us suffer as a result because we couldn’t afford $5000 dev laptops as daily drivers.
Make it as harder as possible to communicate your message...I am not sure if this is supposed to pass for minimalism...but looks more lie readability hostile snobbery. Good content does not excuse contempt for the person trying to read it.
If your argument is worth attention, presenting it legibly is basic respect for the reader....
On the other hand it does mean there's no excuse for writing your program using Python or Electron or whatever, which will have a big effect on performance.
I get that LLMs would make it faster to write distinct copies for different operating systems and maintain them both but thats still more costly than a single system.
Sorry if I’m misunderstanding the goal of Electron but it still seems cheaper and easier (although admittedly worse than multiple native apps)
The author of the article definitely seems to think LLMs are what enables this to happen, but I personally am much more skeptical of that. I think what it really needs is bringing engineering back into software, not just throwing LLMs at it and calling it a day.
It's messed up. Sometimes it feels like everything in the universe is aligned to ensure that skilled software engineers lead horrible lives full of anguish, frustration and powerlessness.
It feels like the economic system creates incentives which go against efficiency. So then it makes sense that software engineers would be most impacted.
There's a weird duality to our job; it's supposed to be about automation and efficiency but in reality, engineers who make the most money are those who create unnecessary complexity because it creates better lock-in and thus gives them more leverage in negotiations. And the tech monopolies they work for never seem to run out of money no matter how much enshittification occurs as a result.
Same perverse incentive for open source projects; engineers who lead projects that are full of bugs can charge more consulting fees and sell more enterprise licenses to work around the issues.
And engineers are often forced to use shitty tools by their employers so we're always stuck between a rock and a hard place with limited control over our output... Which we are blamed for when things go wrong but never given credit for when we pull heroic feats because those achievements are often overshadowed by missed deadlines resulting from being forced to use shitty software.
Adoption and quality are orthogonal, unfortunately.
Worst part is that nobody listens to us. Business people with money only listen to engineers who have money; who are mostly merchants of complexity. Engineers who care about efficiency have no voice in this market.
If you want your software to be fast, stop writing it in Python and TypeScript and instead write it in Go, Rust, C++ or C. But nobody's going to do that, because humans are emotional creatures who get attached to things (like programming languages). Or if you want your software to be fast, stop spending your time and money (tokens) on features, and start spending it on profiling log replays to find a bottleneck causing a 50ms delay for the 95th percentile. But nobody's going to do that either, because how slow is too slow? Answer: when it's slow enough to scare the shareholders or annoy the developer on their high-powered laptop.
If you want faster/more efficient software, force it to run on a 100MHz CPU with 512KB RAM and a 56k modem. You'll definitely prioritize speed then.
1. Identifying the cause of poor performance. 2. Implementation. 3. Architectural impact (performance is a classic case of a cross-cutting concern)
(I am specifically leaving out the case of realtime systems, hard or soft, where additional factors come into play.)
But the article seems to focus entirely on the second aspect, while largely ignoring the other two.
Most performance bugs are not difficult to fix [1, 2], but can sometimes be hard to identify. Implementation effort is not the driving limitation in those cases.
Conversely, other performance improvements may affect the overall design, e.g. 27% of all bugs identified in [3].
Having an obvious, self-contained optimization target with a benchmark where algorithmic optimization within a module is also the key problem seems to be the exception, not the rule.
Also, not all performance issues are created equal. Many have trivial cost.
In contrast, some of the most challenging performance issues are the ones that affect the design and architecture of the system. After all, the hard part of software engineering is not writing a small, self-contained application. It's managing system complexity, while maintaining (in the words of Fred Brooks) conceptual integrity. Fixing performance issues is at least in this regard not fundamentally different from fixing other software defects.
Unfortunately, this is an area that is also full of trade-offs, such as performance vs. architectural simplicity, or performance in one part of the system vs. performance in another part, all of which requires judgement.
For example, you may need to bypass an abstraction boundary or reorganize abstraction boundaries to improve performance. Or you may have to special-case something while keeping duplicated code at a minimum and easy to maintain.
This is not to say that agents cannot help here, too. In fact, agents can be very helpful at e.g. identifying bottlenecks that are not directly visible in a profiler or can be used quickly do comparative evaluations of the various options for an architectural change. But solving these issues is not, like with the regex example in the article, about hillclimbing towards better performance, but involves a combination of puzzle-solving and design skill, IMHO.
And finally, even a self-contained algorithmic improvement may come with an increased maintenance burden, especially around edge cases and through increased code complexity.
[1] Jin, Guoliang & Song, Linhai & Shi, Xiaoming & Scherpelz, Joel & Lu, Shan. (2012). Understanding and Detecting Real-World Performance Bugs. Sigplan Notices - SIGPLAN. 47. 10.1145/2345156.2254075.
[2] Selakovic, Marija & Pradel, Michael. (2016). Performance issues and optimizations in JavaScript: an empirical study. 61-72. 10.1145/2884781.2884829.
[3] Zhao, Yutong & Xiao, Lu & Bondi, André & Chen, Bihuan & Liu, Yang. (2023). A Large-Scale Empirical Study of Real-Life Performance Issues in Open Source Projects. IEEE Transactions on Software Engineering. 49. 924-946. 10.1109/TSE.2022.3167628.
LLMs can now write perfect code in low-level languages, that is frankly _safer_ and _exponentially_ faster than implementations in high-level languages.
When will we start using LLMs to write my web backend directly in UEFI and assembler? (Although uefi is an OS, so maybe more fair to exit the boot services)
There has never been. We are running supercomputers. And somehow every app is a dog-slow barely moving monstrosity incapable of showing a page of text without stuttering.
Yet when I ask for code it writes, by default, both slow and insecure code that mostly works. Kinda.
As I try to get AI to rewrite it into more secure, less bloated and optimized code is when it starts to randomly crash.
Then I read articles about how AI is "moving too fast" and cry.
i want the code to do this --> write the code to do this
i want the code to do this --> write the tests to specify this
the hard part is not the code or the tests. it's knowing what 'this' is supposed to be, exactly.
if you have the code, it is trivial to write the tests that will perfectly test the code. if you have the tests, it is trivial to write the code that will perfectly pass the tests.
however, none of this has anything to do with whether the code or the tests are good, and having a bad version of one will not help you write a good version of the other.
It's s about subpar models trained on subpar data doing subpar work.
The only reason why anyone takes it seriously is that we've had a glut of subpar developers for 30 years so it all balances out in the end.
I agree that there has been a glut of subpar developers for years, and that has lowered the bar significantly. This is mostly because core values shifted. So let's keep our eyes on what really matters rather than acting elitist.
In practice, no one rigorously "proves" that their program works at present, and well-written tests do suffice. The definition of "well-written" here is circular, granted, but there isn't really an alternative. Even strong programmers should be writing high-quality testing suites.
Only a moron would then use them to code up that spec.
Even if you assume that statement is true, there are techniques other than exhaustive checking/model checking. Proof assistants/theorem provers/etc. like Rocq/Isabelle/Lean are quite capable of formally verifying programs without needing to exhaustively explore the search space.
I'd question the accuracy of that statement in general as well; model checkers like CBMC/TLA+ are handy for proving properties about interesting systems. The latter, for example, sees use for verifying concurrent/distributed systems, which I think can be reasonably described as more than "small and unimpressive"
Also the architecture really matters now because a poor architecture will lead to more weird edge cases and require more exotic tests to plug all the holes.
func Benchmark(b *testing.B) {
for range b.N {
runCodeUnderBenchmark()
}
}
Regarding security - i wonder: can we develop a framework that is responsible to 100% of the security, with zero responsibility for security on the app developer?
Maybe pick languages that are more boring. That's what works for me. It worked for hiring crappy outsourced programmers and it works for AI. It seems to do well with PHP - no long running processes / side effects. it works great with golang, simple language spec, and a very common style of writing.1
People get entrenched in closedminded thinking. My guess is that this has been your canned spasm for the past year. Repeating these things won’t make them true or protect your job.
You protect your job by learning how to use new tools as the technology advances.
This sort of lazy thinking isn’t acceptable anymore. Whether this is a reflexive Luddite response or a reflection of your work ethic, neither is a good look.
> Yet when I ask for code
Engineers don't ask a tool to write code.
Most of these things are just true as of today.
> You protect your job by learning how to use new tools as the technology advances.
Sure and the more you learn the more you see how hyped they were and reality untangle
> Engineers don't ask a tool to write code.
And what they ask? They create loops or graphs? These tools ultimately vomit code anyways
The fact is that with the right spec, agentic guardrails, and evals, frontier models can now one-shot secure, performant code.
This is the job of software engineering now. Learn to properly use the tools for the job.
> one-shot secure, performant code.
I call it reverse skill issue; it feels secure and performant because you can't verify properly, it feels right but it's actually broken in many nuanced ways. You can measure performance and produce a broken measurement as well. It's not as simple as you make it
Can you give a concrete example of that being the case? I’d love to see the prompt and harness. And understand the scale of problem you’re talking about solving with a one-shot approach.
How long will it take to write the right spec? How will you know ahead of time that the spec is correct?
Which agentic guardrails do you mean? How can they be set up?
-
I’d say the job of software engineering is still firmly being able to stand by the work you present and being reasonably confident that it’s correct according to yours and others’ understanding of the problem you’re attempting to solve.
Cutting edge development practice in 2025 was using a big JavaScript framework (not quite VMs, but they inherently strain the environments they run in).
That code is run by the JavaScript interpreter VM, which is running in the browser (another VM), which was (in whole or part) written in a managed language (another VM).
That program runs on top of the OS (another VM), which runs on another VM (the default hypervisor for Windows 10/11, or the desktop environment for Linux/macOS).
Each VM lets you have an entirely new managed programming environment- you have a new program that lets you write software even quicker than before- but that comes at the expense of memory usage and (most importantly) latency. Which is why it takes modern systems a few seconds to figure out you pressed a button in your React application that's running in an isolated browser instance consuming 1GB of RAM, where systems in the 90s were instant by comparison even though they had a fraction of the processing power.
It's conceivable that in 10-20 years, LLMs could be used as VMs in and of themselves, which will still likely be using 2020s development languages. So your software is going to require at least 128GB of RAM to compile, use another 32GB to run, and it'll take twice as long to respond to clicking a button.
See: iOS 27. It’s faster than the previous version, even on very old phones.
I also think AI will contribute to removing a lot of the tedium surrounding optimization.
People are shipping all sorts of weird architectures, non-performance code, etc.
I wish that the ram scarcity would drive more performant software however I just think that the way people are shipping software currently will not lead to this
And they weren’t before AI?
But there were some sea changes from time to time.
When the iphone came out, phones became very responsive to fingers. hate the control apple gave to customers, but their guidance made apps respond in milliseconds. Before that phones were clumsy. Wasn't a desktop, but was different.
Will we have voice/chat interfaces that will be bloated... but responsive like a conversation?
First of all, coding speed is not the main bottleneck. I've been working on my own JavaScript framework. Most of my work is talking to people about challenges they have with web development, doing research on existing tools, and thinking about features to add.
Coding more slowly without LLM usage is a benefit because it helps me understand every line of code. The slower pace also helps me think about making things intuitive and reflect on my experiences with React.
While my framework is outperforming React, that isn't particularly impressive. On the other hand, it demonstrates how ignoring LLMs improves quality. Also, my monthly token spend is $0.
That is, there’s a misunderstanding of why software (traffic) was slow in the first place, and it has nothing to do with our ability to generate code (number of lanes), even if that code is “high quality”.
It's not the only reason. I think that traffic engineers lack/lacked a feel, a certain gut understanding , an inability to feel through the skin. Just look around you. See and hear them at municipal meetings. They are unable to sensually experience the effects of their work, their praxis.
The people are to blame. You get what you vote for.
Both the traffic engineer and the author of this blogpost are well-qualified enough in their fields of expertise to be careful about the claims that they make with regards to speed and efficiency.
A big issue is how those claims are interpreted by non-technical people, especially in leadership. Would hate for my CEO to stumble upon this blog post and believe, based on the title and a brief skimming, that his engineers just aren’t using AI correctly and that’s why the website still takes a few seconds to load its content.
In reality, if more people were able to go wherever they wanted to go, that's actually a good thing regardless of if the existing people didn't get to go somewhere faster.
As you imply, some of them are not because spending time on optimization seems relatively expensive compared to value. But some of them are.
Maybe the author should be a bit more careful about what titles they give their blog posts, lest they be misinterpreted by mere mortals such as myself.
Imagine traffic not getting any better after adding 1000 lanes in a medium sized town… not just to one area but to every single damn road.
Yet we’ve done this with RAM, CPU, and storage…
At the end of the day its a mindset.
Understanding.
Simplify, then add lightness.
[1] hn favorite word
And Munger's Law never cares if you hate its consequences.
AI SDLCs simply compress the same incentives that were always there: you could invest in optimizing your architecture and improving your UX, neither of which make for big bullet points that close sales, or you could invest in new features that do.
The investment that was previously measured in months of developer time may now be measured in token spend across days, but the underlying dynamics shaping where it’s directed haven’t changed.
Big tech funds startups (through obfuscated legal and financial means) and that funding goes straight back into big tech cloud hosting and SaaS services.
AI is the perfect tool for enshittification, because it can get you 80% of a finished product with a fraction of the costs it used to take, but the last 20% takes 2-5x times longer. So you just ignore the last 20%, make it up in volume by producing 100 mediocre products instead of 1 great one.
And that’s basically the state of the tech industry right now. I don’t think it’ll be like this forever, I think there will be better ways to collaborate with AI rather than wholesale delegation and prompting. But for now we’re stuck with mass enshittifation. People who have no sense of quality, taste, or craftsmanship don’t even understand, they skim over some generic blog post from Claude that takes five paragraphs to say one thing or watch the Coca Cola AI ad and don’t event notice it’s a different train in every shot and they think it’s so amazing. I hope this isn’t just the way things are now.
Eg. JIT compilers are rare because they are difficult to write. JIT compilers come with tradeoffs, they aren't an absolute boon. JIT-compiled code isn't good for short-running scripts, for example. Even normally JIT-compiled languages try to special-case parts of the code that would not benefit from compilation and interpret them instead. Another huge downside of JIT-compiled code is debugging it. JIT compilation is better for languages that don't deal with memory allocation directly, and, in general, expose fewer low-level primitives to the language user. It would be a fool's errand to make a JIT-compiled C... I bet someone tried it, but, seriously...
Another one: a solution to slow search is building an index... Boy do I have bad news for you... Even a beginner DBA knows that an index can be a curse or a blessing. There's no one size fits all solution to this.
And the author continues to misrepresent similar problems only to make his solution make sense.
problem solved
What was important for this was the fact that I can run JSON-defined e2e tests[2], also good to find frame-based leaks, stale-ID problems and general "program shows its using lots of memory in the task manager" (task managers are wildly inaccurate for this, as I found out).
So, I can just tell it "okay, loop this e2e.json test over and over again, use heaptrack, samply, find out why and exactly where it's slow, memory-hungry - find the section in the codebase, figure out a solution" and then let it run overnight. The biggest difficulty here is that many perf tools are still written for humans and that things like "how much memory are we using" is a wildly complex topic (lots of problems actually getting the correct number without over-allocation, memory allocator slack, OS-level page size, memory fragmentation, etc.).
But Fable was able to track down things like "LCD font hinting causes 90k allocations that are unnecessary", etc. etc. - which then also improve frame time, usually. Memory optimization + better perf pretty much go hand in hand (less allocations = more perf). I could track them down manually probably, but it would take way more time.
Having some basic understanding of data-oriented design, cache locality, memory tiering (L1/L2/L3/main RAM), does massively help with architecture decisions (e.g. Azul can use a single buffer for the entire DOM node list, in difference to normal browsers which do the more "object based" allocation model, which massively helped page breaking performance on html-to-pdf use cases[3]). Pure-functional style also helps (f(State) -> UI) because then it gets very easy to drill down exactly where things are slow and where caches are needed.
[1] https://azul.rs/ui/release/0.2.0#demos
[2] https://github.com/fschutt/azul/blob/master/e2e/css-anim-per...
[3] https://github.com/fschutt/azul/blob/master/layout/benches/f...
Note: UI Toolkit is still very WIP, docs + code are still slop, etc. - working on it. But I just wanted to add this.
But you need a really solid workflow, solid benchmarks which run quickly, and a lot of tokens -- plus a rigorous profiling workflow.
Here in hackernews we have a bubble of people who are very smart so they process information at high speed in general, and have lots of experience with computers, so they process computer-related information with extra speed. We're like race drivers, squeezing every single bit of performance from the car, while majority of use cases for a car is being stuck in traffic on your way to work. Those people simply don't care about performance above 150km/h because they'll never reach such speeds so putting expensive, high-performance engines is a waste of resources.
The point I'm trying to make is that if software is fast enough for average user, and most users are fucking slow, there's no business need to further optimize it. You might keep arguing "but but but back in my days apps were instant and now they suck" but users simply do not care. This is why companies put shitty, time-wasting animations even if nothing is being done in the background.
Otherwise, I would have left immediately.
I'm all for speedy, simple, plaintext websites, but it is a negligible amount of work in 2026 to throw some barebones CSS in and make it approachable.
Also, perhaps you should learn how to make use of the user stylesheet feature of your browser, or ask an AI to tell you how...
I think he might subscribe to the philosophy of everyone should style websites however they like and not be prescribed by the author.
He's had a usable website in the past. This one is explicitly made to be shitty. "Look at me, I'm such a geek" energy.
> Not everybody knows how to use sql or css or java or whatever thing it is that lets you pick the font.
lol at the idea that Dan Luu can't add css to his website by copypasting some links into his html. He could make the site readable in about 3 mins and never have to worry about it again.
<!-- Google Fonts -->
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300italic,700,700italic">
<!-- CSS Reset -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/normalize/8.0.1/normalize.css">
<!-- Milligram CSS -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/milligram/1.4.1/milligram.css">
> Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.