Rendered at 00:47:22 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
status_quo69 4 hours ago [-]
No wonder most people at Anthropic believe our jobs are in jeopardy, they've already made a system wherein the machines are treating them like pets.
I really don't get the tendency here from Anthropic to describe what's most likely churn? A high pressure amount of changes/PRs a day on a TUI to wrap a while loop? I'm not going to be a reductionist here so I'm going to cut it off there, I'm positive that the complexity of handling multiple terminal emulators is something that needs some handling (even if doing their own rendering engine was a self made problem). Yes I've vibe coded my own harness from scratch without writing any of it, I still know how it works.
I'm also not sure that they addressed point 2 in any actual capacity? They say "the system verifies itself" which is cool and is something that automated testing has done for decades, but they eschew the mental model of how the system works as a whole and prefer instead to have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
N.B. Changed "1000 PRs" to "A high pressure amount" to be vague since the article is vague here on what CC undergoes.
jonahx 3 hours ago [-]
> have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
This is the crux. Take pause, because hidden here is a sea-change in our fundamental trust model of software...
There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug. By design, the system is too large for that, and everything hinges on Claude itself being able to find and understand the right slice of the system in any situation. As long as that very new invariant holds, everything will be all right. If it fails catastrophically, there is no recovery. You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
The new trust model is: We trust Claude to always understand the system enough to debug anything or make any reasonable change we could want. The risk might be worth the reward. But it's worth explicitly laying out the deal we're making.
Tbf, it's worth pointing out that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
patrickdavey 3 hours ago [-]
"Tbf, it's worth pointing that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
The real difference as I see it is that in the human maintained old style system, the human would take things slowly and ensure they understand any changes. Claude let loose will just say "yes, I know how to do this" and bork the system, because it doesn't have a concept of humbleness in the face of complexity!
daveguy 3 hours ago [-]
Not only that, when you have a human designed and implemented system you know it's at least coherent enough that someone understood it at some point well enough to take responsibility for it. No such assurance with the llms.
csallen 2 hours ago [-]
But is this really a problem? If you have testing and you have version control via Git, you can roll back changes and investigate them.
I also don't even know if it's really true. Sure, maybe it's not coherent enough that a person understands it, but if an LLM wrote it, you can be ~99% certain (in my experience) that that LLM will understand it, and usually in less than the time it took to write it. And unlike humans, the LLMs don't die and take their knowledge with them, or change companies, or get ornery when you ask them to explain.
Also, in my experience, Claude doesn't really do what the parent is saying and dive head-first into complex problems and bork your code. It is fairly humble in the face of complexity. It constantly recommends breaking down larger problems into smaller ones that are more easy to grok.
But most importantly, anyone who's a decently skilled software engineer and who's using LLMs to write code is going to be thoughtful. Especially when they're writing code that's important. Unless you're an AI researcher pushing the bounds of what LLMs can do, which let's face it, the vast majority of programmers are not, it's almost always the case that you'll have your LLM writing intelligible code that you can understand and have it explain easily enough.
jdlshore 6 minutes ago [-]
> anyone who's a decently skilled software engineer and who's using LLMs to write code is going to be thoughtful.
Is this a definitional argument? As in, if you’re not thoughtful, you must not be a decently skilled engineer? Because I see no reason to believe that people are going to be thoughtful about LLM-generated code. Just the opposite. It’s incredibly tempting to LGTM it.
dieortin 1 hours ago [-]
> can roll back changes and investigate them
Sure, you can do that, but irreparable harm might have been already caused. Sure, that’s unlikely when your product is an LLM wrapper. But in something like a database, it could be catastrophic.
SR2Z 2 hours ago [-]
I am not sure that "someone understood it well enough to take responsibility" means what you think it means. I have definitely come across old systems which have had addons bolted on to them in so many places that the overall behavior of the system is unpredictable and only mostly works.
cmrdporcupine 4 minutes ago [-]
> you know it's at least coherent enough that someone understood it
30 years working in this industry and I'd say that's the exception rather than the rule. When it comes to large systems.
antihipocrat 2 hours ago [-]
I'd like that to be true, but many software service startups were stitching together node packages and selling the result long before LLMs became useful.
peab 3 hours ago [-]
most large software companies are already too large for anyone to have a complete mental model of.
The idea that someone at say, Google, will be able to fit even half of the system architecture in their heads is not realistic. It also hinges on being able to find and understand the right slice of the system in any situation.
But we have all sorts of guardrails to make that easy - i.e logs, traces, etc
jonahx 3 hours ago [-]
Right, that's what I alluded to in my last sentence.
It is true that this gamble has existed in the form you mention for many years. Still, I sense something new here, both in scale and in the attitude of trust. Also, I find those massive, incomprehensible human-built systems frightening too.
dasil003 2 hours ago [-]
Having worked continuously this decade on one massively complex consumer facing app that interacts heavily with real-world scenarios, I can say that the problems of cohesion existed well before the agentic revolution. Tons of features built by myopic teams simply because they couldn't understand enough of the system (both online and offline, meat-space interactions) to make good design decisions. AI actually helps here by being able to perform deeper reading and analysis than any human can do. But then the risk is that it can also produce way more random non-cohesive features.
So far I kind of feel like it nets out to the same bottleneck being good judgment from decision makers. And you might say, no, leadership will just put more and more pressure to add more complexity just because we can and it makes it appear that we are moving fast. But when it comes to real-world apps, there is a backstop which is that things that are too complex and illegible end up not being fit-for-purpose for human problems.
Claude Code and other general purpose AI tools are not like this, because they are literally trying to optimize it for every single kind of knowledge work with zero constraints on use case or target audience other than chasing the next juiciest one they can with essentially unlimited resource. 99% of software is not like this, and when the dust settles I suspect software won't be anywhere near as different as the doomers or the maxxers are predicting right now.
23 minutes ago [-]
dimbletimbers 1 hours ago [-]
> But.... That's not knowledge?
Thank you! This is a huge sticking point for me. There is no knowing the code without experiencing the code. If nothing else, you have to read the tests to know they verify what you think they do. Even then, you might know it does what you want, but can’t know it doesn’t do anything it shouldn’t.
There are no shortcuts to true understanding.
Terr_ 4 hours ago [-]
> wherein the machines are treating them like pets
I know some people will groan hearing the phrase again, but: "Reverse centaur." A centaur is a synergistic combination of two things, but now those things are in the wrong position: The human is the computer's tool.
Naturally, how "wrong" it is tends to depend on whether you ask the people who do all the work or the people who own all the computer-capital.
frereubu 3 hours ago [-]
My favourite quote at the moment: "May not man himself become a sort of parasite upon the machines? An affectionate machine-tickling aphid?" Samuel Butler, Erewhon, 1872
hgh 3 hours ago [-]
I think it's just a different focus on what matters.
There's clearly enormous (human) work that goes in to building and tuning their models.
The software they develop around their model is, relatively, trivial. It's not clear that enormous amounts of (human) energy need to go in to that code.
It seems largely a function of applying human energy towards the things that are marginally most important to generating you money and putting you in front of competitors.
xyzsparetimexyz 3 hours ago [-]
>The software they develop around their model is, relatively, trivial
30,000 changes a day is not my idea of trivial.
californical 3 hours ago [-]
I mean a harness around a model is pretty trivial, I’ve also built one (heavily relying on LLMs to build it).
With a few thousand lines of code over the course of a week or so of my limited evening energy, I have agentic loops, ide integration, and a limited selection of custom tools which are all very safe (no unintended changes), along with session management and awareness of git.
It works great, is customizable, and allows integration with any provider.
If they need 30k changes per day, it’s not due to the nature of the problem!
xyzsparetimexyz 2 hours ago [-]
Oh, I mean I can't disagree there. But they've dug themselves into a hole where they're implementing custom ray tracing in the game engine that claude code runs on or whatever.
californical 2 hours ago [-]
> implementing custom ray tracing in the game engine that claude code runs on
Haha, I don’t doubt it! Pretty damning overall for long-term maintainability of vibe-coded projects, even as the models are great at a lot of individual tasks
status_quo69 3 hours ago [-]
I mean, sure we can assume that (although the Anthropic article from the other day say that claude mostly builds Claude) but this is the Claude Code team which, I again am assuming so I could be wrong, focuses on the TUI/CLI interface to those models, not necessarily the models themselves. That coupled with the fact that this is how a large portion of their clientele interact with their moat, it would be odd to me if they didn't apply significant work towards this thing to keep the hype or productivity train moving
markbao 3 hours ago [-]
> I really don't get the tendency here from Anthropic to describe what's most likely churn? 1000 "PRs" a day on a TUI to wrap a while loop?
From the article, the 30,000 changes a month are in the monorepo, and Claude Code is in a separate repo. So it’s not the TUI that has 1,000 PRs a day.
I could easily see 1,000 PRs a day on the backend. Big tech was doing more than that pre-AI.
status_quo69 3 hours ago [-]
Fair enough, I did invent the 1000 a day because the article was vague, I'll edit my submission accordingly to also be vague
tdhz77 28 minutes ago [-]
This is a good answer
ls-a 3 hours ago [-]
[dead]
mk12 4 hours ago [-]
I saw someone say the other day (in response to another blog post being well written) that Anthropic employees have a rule against using AI for writing/external comms? That seems clearly not the case here, normal people don't write like this:
> Most of the path is automated; a person's attention goes where the risk is
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
jazzyjackson 4 hours ago [-]
100%, although if people are getting 90% of their interaction from a robot, one may start to pick up some vocal tics and colloquialisms from machine world. a kind of robo-pidgin.
levkk 2 hours ago [-]
Yuuuup. Which I guess makes it even harder to detect what's AI and what's not.
lavezzi 1 hours ago [-]
> That seems clearly not the case here, normal people don't write like this:
I think when you repeatedly expose and immerse yourself to such a degree in reading and writing language, it brushes off on oneself. So I could also absolutely believe someone from Anthropic wrote this.
atty 29 minutes ago [-]
It felt weird reading it because every once in awhile there were some well written sentences and coherent paragraphs, then it fell back into Claudisms.
myth2018 16 minutes ago [-]
As a non-native, I see how susceptible I am to absorbing such style. There's no way at all for me to perceive anything wrong with the examples you present
jansport123 3 hours ago [-]
this definitely seemed like something claude generated, i know because i frequently do all my research in chat and then ask it to generate a doc based on what we talked about and when i read what it generates the verbosity, the style it writes in makes me delete the whole thing. I would never subject readers to that.
From the article: "merge with at most one human in the loop, who can be the author". Curious to see how this handles enterprise compliance requirements and insider threats?
- How was this approved for SOC2 compliance? Our SOC2 reviewer explicitly mentioned that all code needs to reviewed by at least one other human before it goes to production. Claude Code says they are SOC2 compliant (https://code.claude.com/docs/en/security).
- This assumes all engineers will not do anything malicious. What is preventing a Claude engineer from spying on their ex's or some other companies Claude searches by pushing malicious code? Recently an OpenAI engineer was caught planting a recording device at their own office.
llama052 6 minutes ago [-]
I’m sure it’s not compliant. We have the same requirement for soc2. However the audits are pretty light and a company big enough can afford to have an auditor who looks past those things.
rddbs 12 minutes ago [-]
Genuine question I’ve been wanting to ask folks working on different types of things: how are your teams doing agent-assisted code reviews? Not automated reviews, but what techniques and tools are humans using to keep up with the larger volume of PRs that seem to trend larger and larger?
I think a lot of people like myself still want to feel in the loop but I don’t have the right tools to stay on top of the (generally good, productive, and still thoughtful) contributions from peers anymore.
dieortin 1 hours ago [-]
> One colleague said slowing down does lower your error rate, but mostly because you're doing less of everything
That’s not how rates work
vb-8448 2 hours ago [-]
No human can stay on par with 30k pull requests per day ... they simply don't have control on what's going on and rely on AI itself to do the job.
PS: I really want to know what is the churn rate (and the cost per LOC) here. My gut is saying the cost per LOC is insane, and they are basically rewriting over and over the same code.
cush 1 hours ago [-]
30k PRs is such a meaningless figure though, you can't make anything of it. It entirely depends what's living in that repo. Adding localized strings, configs, things that change frequently, can easily move numbers like this by orders of magnitude. How big are the PRs? Maybe there's an auto-bug-fixer agent that runs around the codebase making thousands of single-line fixes per day. Who knows. The number means nothing in isolation. We used to scoff at bad managers for measuring the performance of engineers using lines of code, now we're using the same figures to judge AI. We've become the bad managers.
koolba 4 hours ago [-]
> Low - Docs, copy, internal-only features, additive changes behind a flag
> None required once automated review and security checks are green
How anybody can let an LLM write docs and not even read them is pure insanity.
grey-area 4 hours ago [-]
At 30k commits a day, is there anything that can meaningfully be called review?
stillpointlab 4 hours ago [-]
I'm thinking about high volume mass manufacturing as an example here. I like those "How it is made" type YouTube videos. You see a production line that has some kind of QA process where cameras + automated systems do spot checks at high speeds and these crazy actuators are punching out bad examples. These processes often also involve spot checks where some statistical sample is selected and viewed carefully.
I don't think we've gotten to that level of reliable review with code, but I don't see any reason we can't continue moving in that direction.
fipar 4 hours ago [-]
I really like those videos too, but a very big difference is, in manufacturing QA, the system outputs a very high number of copies of a very small number of possible types, so spot checks work fine from an operations research point of view, as the variance in output has a known upper bound.
For code, I think things go in the opposite direction: the system outputs artifacts that are all different among themselves (even more so now that we no longer distribute software via physical copies. Well, bits are always physical, but I mean we no longer ship floppies/CDs/DVDs).
That said, at least in my experience, code review has never been a direct part of factory-line-type QA. That QA is typically done by treating software as a black box and just verifying that it complies with a spec (often times based on a very loose definition of "spec" ...), and this is something that works equally well regardless of whether a human or a machine wrote the software. The type of QA that is done (among other things) by code reviews is checking whether the software's design is such that it will be economically feasible to maintain and extend it, for example.
The one part the code reviews takes a part of (but is in no way error-free) and that you just can't do with black-box testing (or it is much harder to do) is to look for things like backdoors. As our industry moves more towards "agents write software and tests validate that it works," I think a lot about "Reflections on Trusting Trust" and the new and wide attack surface added by people no longer reading the code shipped on their behalf.
grey-area 4 hours ago [-]
That’s a category error. Software isn’t stamping out the same part again and again; if it were review would be simple. Analogies often mislead.
Of course Anthropic would love you to think you are heading towards that future so we have to bear in mind their livelihood depends on LLMs being the future of work and 30k changes a day being normal.
But why would we want 30k changes a day in a given product, what possible justification could there be?
stillpointlab 4 hours ago [-]
I understand the skepticism and I can't claim I am right. But some points come to mind.
First, I think we overestimate how different code is from other code. We even joke about CRUD apps and how cookie-cutter they are. A lot of changes in those apps are: add some new features to a data access layer (SQL, ORM, whatever), add new routes, add new UI surface, etc. There is an entire industry around "Design Patterns" and people live by the DRY axioms. Things like Ruby on Rails exists because rebuilding these systems over and over is wasteful. I feel we are just moving further down that same path.
Second, your question of "why would we want" is totally backwards. Progress doesn't follow your desires. Progress doesn't require justification. Sometimes we adapt the environment to our needs, other times we adapt to a changing environment. Which case we are currently in is still undecided and you are free to fight for the future you prefer. But demanding an industry justify itself before it is allowed to change fails in the face of evolution. If it works, we will justify it in hindsight.
g-b-r 2 hours ago [-]
The second paragraph hardly makes any sense, was it the output of an LLM?
stillpointlab.com , "Get more from your LLM"
jbvlkt 2 hours ago [-]
Exactly. 30k changes a day look like brute force. No one wants that except the one charging for generating those changes.
0c3ca83 3 hours ago [-]
Those manufacturing videos are of someone making the exact same thing over and over. Months and years of testing go into ensuring that there is no change in the system, and continuous monitoring is put in place so that if any changes slip in, it can be undone. The design is set in stone, as it should be, and variations are bugs. There are months to years of cheap prototypes, iteration, design changes, pipeline tuning, and verification to ensure that nothing will need to change for the next million items.
If you want a software analogy to those devices getting churned off the assembly lines, it's the TCP streams coming out of your network card, with crazy checksums punching out bad packets.
We're orders of magnitude better at reliability with code, but we're regressing on that metric rather quickly. And the comment above deeply misunderstands both manufacturing and software.
cadamsdotcom 38 minutes ago [-]
The repetitive piece - the piece that needs that process-control - is moving from shipping code to its creation.
That's new. All analogies to 2022 software engineering need to be thrown out.
Software construction is now more like an industrial process, with the artisanal work shifting towards both
1. aligning what's built with what's needed (talking to customers!)
as well as
2. implementing and tuning your software construction process (ie. hi - we're Anthropic. We're burying ourselves in a mountain of code but it's okay because the fix is a second mountain of code, try some tokens the first hit's free!)
Back to my original point...
Writing code is now on the production line; making quality output demands industrial grade/scale verification (some involving humans of course).. which is a fascinating engineering challenge, and it's brand new!
jeremyjh 4 hours ago [-]
Those machines are not doing design work. They automate production of designs that had long development and testing lifecycles. Code is design. The compiler is the automated production process.
cush 1 hours ago [-]
Honestly systems like this resemble real engineering more than manual human review does
mirekrusin 3 hours ago [-]
It's conducting/steering/oversight/supervision, not reviewing.
swaptr 3 hours ago [-]
I dont believe it, its FOMO marketing to me.
YmFzZTY0ZGFkZHk 4 hours ago [-]
And so the wildly unprofitable AI company explains how it token maxes it's code and review process.
Authors should really try and view this article from the perspective of someone who has to think about cost to use the models.
markbao 4 hours ago [-]
[dead]
enraged_camel 4 hours ago [-]
Anthropic has been profitable for the past two quarters.
tedd4u 4 hours ago [-]
"Anthropic told investors that it will profit on a measure called adjusted operating income.
"According to the report, adjusted operating income excludes stock-based compensation, which is not unusual for technology companies. However, the report also quotes its gross margins - above 80% - exclude revenue shared with distribution partners, as well as the cost of training its models."
I wonder how they account for free credits given in this spend. We got given US$5k free API credits from them a couple months ago and even accounting for our Claude Code subscription spend, I think we're still ahead overall - they haven't made any money from us! And they just gave us another few hundred bucks a month of API credit as part of our subscription.
3 hours ago [-]
rhdunn 4 hours ago [-]
How so? Can you link to the numbers?
Antropic's revenue is up [1], [2], [3] and if you calculate adjusted operating income (by excluding most expenses like training costs) then by that measure it is profitable. However, if you factor in all the numbers it is operating at a loss.
Source? Only reports I’ve seen was they were profitable for one month in last quarter due to xAI discount of renting out colossus . Be interested to learn otherwise.
lbrito 3 hours ago [-]
That's factually incorrect, and shouldn't be allowed here.
After investments. Investments they can cut if they need to. Their daily operation is very profitable, they just decided to use that to bring cash in and go on the crazy growing trajectory, instead of taking it slow and steady. But if AI progress suddenly stops, they can just scale investments back and became profitable immediately. Calling them unprofitable is highly misleading. It's not like they are an early venture company still trying to figure out their product-market fit, with totally inflated numbers based merely on a theoretical number of potential customers (while having close to zero paying ones).
YmFzZTY0ZGFkZHk 1 hours ago [-]
I'm not arguing that their product is not valuable or that it doesn't have customers.
I'm stating that their current business model, prices, and behavior they are trying to instill is not profitable. For example, token maxing every code change is great for Anthropic and terrible advise for Anthropic customers.
The moment they stop training, they become irrelevant. The moment they become irrelevant their valuation crashes down to normal valuation ranges for a company with their revenue and future growth
surgical_fire 4 hours ago [-]
We're talking about being actually profitable, not fake profitable.
tracerbulletx 4 hours ago [-]
“I think that, every time you see the word EBITDA, you should substitute the words "bullshit earnings.”
― Charles T. Munger
And what they're reporting is adjusted even further than EBITDA
adammarples 4 hours ago [-]
Yes when you subtract expenses and capex
throwaway613746 1 hours ago [-]
[dead]
mstaoru 3 hours ago [-]
Before all came to this, I've been promoting reviews at $dayjob not as gatekeeping but as method to exchange information and build more team spirit. We are an internal service team working by "projects", which makes us a very disjointed team with barely any communication. (Part of it is also German work culture.)
And now this. Suffice to say, my efforts did not bear fruit. I feel more isolated than ever before.
RGS1811 4 hours ago [-]
How is the ability of a human to understand the code via hands-on familiarity not a direct dependency of their ability to do manual code review? I don't get it. Having claude write an explainer (something I do a lot!) doesn't get you to the point where you can confidently assess a complex diff. And you've funneled your priors on what the diff says and does through the same system that already reviewed and approved it. This article isn't bad but it doesn't give enough weight to the human context-building part of the problem.
jazzyjackson 3 hours ago [-]
Yes it used to be said, the code is not the asset, the people who know what the code does and how to change it are the asset.
Now we’re just going to outsource our capacity to design and review because it’s faster that way, as if the thing that was wrong with the software industry was that releases weren’t being pushed out often enough.
visarga 4 hours ago [-]
All good ideas, I have been doing something similar for half a year. The most important thing is to reduce bugs and ensure the code does what you wanted. Writing code tests, logging, interacting with the running app are essential. For intent realignment I like to have the agent re-read all user messages posted in that project. It only takes a few minutes even on a 10,000 message project. Then use this fresh derivation of user intent to check code and tests. For long spans of work I prefer to generate a todo list which remains as a document in the project. Each closed gate tracks feedback from the agent on what happened. This task file is also reviewed twice - once before implementation and once after. So the formula is - review agents - tests - user message logs.
dominotw 4 hours ago [-]
i am tired of listening about other ppls agent setups. Everyone thinks they've discovered something cool and unique but They are all the same.
Its like telling other ppl about your dreams. Not interesting.
kaashif 2 hours ago [-]
I used to keep a dream journal because I thought it might be interesting to recall all of my dreams.
Eventually I did, but it was hours and hours of effectively random stuff happening that I had to write down or dictate with speech to text.
It was a waste of time and I'm now glad I forget almost all of my dreams. My point is even my own dreams are uninteresting.
I'm not that interested in my own agent setup either - usually the tools become better on their own after a few months and render it all obsolete.
acedTrex 4 hours ago [-]
For real, my eyes completely glaze over, there is nothing interesting in it.
vavkamil 5 hours ago [-]
> Security gets its own automated review too. ... You either fix it or close the thread with a short reason, which is how you clear a false positive.
I hope this is only for low-risk false positives. Dismissing a critical finding as a false positive should still involve AppSec.
You could improve revert rates while simultaneously introducing more security vulnerabilities.
And the linked "claude-code-security-review" explicitly states in the README
> This action is not hardened against prompt injection attacks
That's a significant limitation for an AI security review control, no?
compounding-err 3 hours ago [-]
I think this is the future. Im more bullish than bearish. The only issue is that eventually human verification can no longer be possible and we will have to live under AI-verification assumptions....
KasianFranks 2 hours ago [-]
We're already there with our pipelines and suspect a few more out there more concerned with measuring outcomes at this point.
vatsachak 3 hours ago [-]
Mfw tui needs a billion prs per day.
Opus 5.5 is obviously stellar but come on what are you doing lol
ativzzz 4 hours ago [-]
It looks like they are specifically optimizing for incidents/reverts and shipping faster
> The goal hasn't changed: less human time per change, and no more reverts than before.
Is this Goodhart's law in action? Less reverts and less incidents was measured and became the target. Is this then still a good measure? I guess Revenue is a more direct measure, but we don't have access to that one.
enraged_camel 4 hours ago [-]
From my perspective as a power user, the product has improved substantially over the past six months. It has gained new features that are actually useful, lots of UI/UX/perf improvements, and bug fixes as well.
New features have actually been remarkable. For me they are almost always in one of two categories: things I've always wanted (but never actually bothered communicating to the devs), and things I didn't know I needed but did. The product team is clearly very tuned in.
All in all, I'd say their approach is working.
hyperpape 5 hours ago [-]
I'm somewhat surprised they would say Claude is writing 90% of their code at this point. Did they get tired of reporting new milestones, or has there been a plateau?
pianopatrick 2 hours ago [-]
I am happy people are testing these things out. I am also just going to wait a bit longer before really adopting things like this, I don't really trust it yet. A year ago this looked very different, and next year my guess is things will look different as well.
maxgashkov 1 hours ago [-]
I'd be more interested to read on what 10% of the code is still getting written by humans and how it doesn't become a bottleneck at the pace they described.
lifeisloving 1 hours ago [-]
Important/hard things are usually bottlenecks, its a good thing and usually by design. People should have more self imposed bottlenecks.
What are llm freaks in such a hurry for? Its like everyone is manically racing towards a finish line without realizing the finish line is you out of a job. Its like everyone is excited to be completely obsolete and poor.
maxgashkov 35 minutes ago [-]
My interest is mostly from the management perspective -- in 2025 I have seen a prime example when a heavily vibe-coding team chugged out half-baked products that other teams later had to deal with and this was not pretty and created all sorts of interpersonal issues; the same I fear may be happening here as well.
I'm looking at Claude Code and they chug releases daily and sometimes more often then this, if someone has to stop and code a critical path by hand or smth I'm wondering how this is handled.
--
On a separate topic, the "llm freaks" -- I know you intended this as an insult, but I'm actually proud to be called a freak again, and I thought this is the place where freaks are discussing things.
jwpapi 4 hours ago [-]
What I’m more worried about is feature identification and how to monitor them. How can you get the product to the next level?
Honestly I’m not worried about code anymore in my codebase I’ve not had bug problems for a while and incidents get fixed automatically.
I monitor new features with an agent who checks recordings.
Most actually good features need still 20-30 iterations to design the userflow to actually be interactive, but then there is also another challenge to translate usage patterns into features and how to present functionality with the least amount of friction as possible.
How to efficiently run monitors without wasting your token spends. A best 'monitor' best practice would be nice.
Still this article was very helpful and gave me a lot of new ideas.
dylanhouli 5 hours ago [-]
Here I am still unable to stop myself from doing line-by-line code review. I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) by doing it that I feel it would be careless not to.
stillpointlab 5 hours ago [-]
My advice is to start a totally vibe coded side project. You will never gain confidence until you experiment, find the failure cases, learn to deal with them, etc.
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
72deluxe 2 hours ago [-]
I did this, but now I have a useful piece of software that I have zero clue regarding how it works. My only chance of "maintaining" it is giving it to another LLM to explain.
I am happy that I have a useful tool, but it simply isn't maintainable without a machine or me investing much time to understand it.
I don't see how the former is a sensible future for humanity. I am happy to say I won't be releasing this software I've created (I will find it useful myself), but deferring all understanding to a black box seems a poor future for brain activity.
xyzsparetimexyz 3 hours ago [-]
Ridiculous suggestion. If someone already takes issue with ai generated code (as well they should, it often is _still_ pretty bad), getting them to write more of it is useless.
Additionally, do you really believe that 'Now is the cheapest time it will ever be to experiment'? Why so? Because of token prices? Not a good omen if so.
stillpointlab 2 hours ago [-]
I'm not sure what is ridiculous about suggesting people take a stance of openness and curiousness when confronted with new situations. I genuinely don't understand what people have to lose by building some throw-away experiments without reading the code.
I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
xyzsparetimexyz 2 hours ago [-]
>I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
God I hope you're right. Maybe then this era of vibe slopping out photoshop clones that'll be abandoned within a week (while advertising them all over social media) will come to a close.
mattm 3 hours ago [-]
I have done this. It's exactly the reason I went back to doing code review. I don't look at the code with a fine toothed comb, but it does help me catch bad patterns early.
stillpointlab 2 hours ago [-]
I guess the question is: did the project you worked on fail? As in, you couldn't achieve the outcome you were trying to achieve? Or at some point did you sneak a look at the code and recoil in revulsion? What was the failure state?
Was this with the latest models (Fable/Opus 5.5 or equivalent) or an older model in a 3rd party harness?
Are you in the camp of: "if at first you don't succeed, never ever try again".
hn_acct-1029393 2 hours ago [-]
Not parent, but project I was involved in was shipped very quickly, had many bugs, and burned a lot of customer trust.
This was a few months ago, so obviously not with the latest models, but:
1. That argument is evergreen. You will always be able to make it, and it doesn’t really add much. It is impossible to refute.
2. Gone back to - when writing code - having a strict mental modal and good control over the codebase. A mix of hand coding (to establish the mental modal) and prompting (for boilerplate). It’s not quite as fast, but the quality is much higher when you mix in using LLMs in quality enhancing capacities (review, debug, etc). This is with the latest model. Hands off still atrophies over time (even the boilerplate), and in some cases is very inefficient (as in wasting my time, nothing to be said about tokens).
> Are you in the camp of: "if at first you don't succeed, never ever try again".
This is great in some domains, but “don’t make the same mistake twice” is much more important in mine. I don’t have the luxury to ship low quality, especially when LLMs make even higher quality so much easier.
stillpointlab 2 hours ago [-]
I think people are moving the goal post here. My post suggested starting a vibe coded side project, one where failure is low stakes and has no consequences. You are talking about shipping a production system to customers.
If you cannot tolerate failure then don't experiment. Don't mix the two.
xboxnolifes 2 hours ago [-]
I've done this, and always get subpar results. I cant get the LLM to make the minor tweaks I want to see, and now I have no mental model of the code to make the fix myself. Sometimes they arent minor things.
troupo 4 hours ago [-]
[flagged]
dang 4 hours ago [-]
Please don't do this here.
(You're welcome to make your substantive points thoughtfully, of course.)
Please don't do this whole 'Please don't do this here.' thing here.
Their 'WAT' was self explanatory.
stillpointlab 4 hours ago [-]
Issue: I am still unable to stop myself from doing line-by-line code review
Response: Try experimenting in a low stakes environment
From Google: "Exposure therapy is a psychological treatment designed to help people confront their fears, break cycles of avoidance, and reduce anxiety."
g-b-r 4 hours ago [-]
The thing is that's not an issue in the slightest, it doesn't need therapy!!
It's those who don't do it who'd need a talk..!
stillpointlab 3 hours ago [-]
I think that depends on where you work. If your company demands that all code is reviewed then anyone who doesn't do that should be spoken to.
However, I suspect there are teams that are making different choices. In the most obvious example, I can do whatever I want on my personal projects (I hope you agree). And if I started a business and hire in programmers, I have some expectation on defining the culture, making hiring decisions based on my preferences, etc.
Whether or not such teams succeed will be a matter of fact, which I do not believe has been decided at this time.
9dev 4 hours ago [-]
I hate to say this, but you're now living in a world where the value of artisanal code is rapidly approaching zero. If you still cling to beautifully written code as your core qualification, you're having a rough future ahead of you.
And I say this with the bitterness of someone having lost a big part of what made me enjoy this line of work.
g-b-r 3 hours ago [-]
That's not wrong right now, we'll see in the future.
But I have no interest in working as a vibecoder, there are many other jobs that I'd enjoy more
troupo 4 hours ago [-]
No. The issue is: "I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) during review"
How does "low-stakes environment" fix that?
stillpointlab 3 hours ago [-]
Because one may learn that those fears are overblown. Or they may learn that they cannot build any kind of software without reviewing every line of code.
The person is saying "I am still unable to stop" and that is most likely because they feel a sense of responsibility. If they choose a project where there is no responsibility, where it literally doesn't matter if the project fails, they may feel able to stop. And if they manage to make that project work, if they can succeed, then they will build up confidence.
It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code. In fact, keeping their job in the long term may (or may not) require having both skills.
daveguy 2 hours ago [-]
yolo'ing a code complexity generator doesn't make it suddenly stop being a code complexity generator.
jazzyjackson 3 hours ago [-]
I guess they’re just saying “you’re holding it wrong”
Last time I chose to vibe code, I did it in a language I didn’t know so that I wasn’t tempted to try and comprehend the choices the robot made. I ended up with the application I wanted.
Caveat: it was code I would never have to touch again, I ran a script to generate the 140,000 static html pages that the rails app provided and then just uploaded them all to R2 buckets (million reads per month on free tier)
BobbyJo 4 hours ago [-]
I think the intended takeaway is to learn to solve those problems without line by line review. Start by being hands off on a low stakes project, and see how the model fails. Then, come up with, and try, solutions that aren't you doing archeology.
troupo 4 hours ago [-]
This somehow assumes that the person above doesn't already do all the magic the people say work 100% of the time, and finds these issues. How does running into literally the same issues in a low-stakes project help?
BobbyJo 3 hours ago [-]
I think it's a good assumption. Their comment implies they aren't harnessing the coding agent very strongly given the failure modes they mention.
Do you think it's better to experiment with high stakes projects? Do you think it'll be easier to avoid defaulting to line by line auditing of a project that is high stakes?
benoau 5 hours ago [-]
This is why I like GitHub Copilot's autocompletion, I'm right there guiding and policing the code as it appears.
4 hours ago [-]
g-b-r 5 hours ago [-]
You're unable to stop being responsible? Tragic!
spiderice 4 hours ago [-]
Is it responsible to move slowly when your competitors are moving quickly? Depends on how your competitors perform. If they perform well, then your "responsibility" suddenly becomes your extinction.
tripleee 4 hours ago [-]
Apple's iPhone Duo seems like it'll be a huge hit.. how many years after the competitors?
They've built an entire strategy around moving slow and doing it better
g-b-r 4 hours ago [-]
If they move quickly by shipping crap they're just running towards a wall, the responsible thing is to not follow them.
xyzsparetimexyz 3 hours ago [-]
fomo nonsense. not everyone has to jump off the cliff just because you are.
dyauspitr 5 hours ago [-]
No one cares about good code. It’s a means to an end. Your code doesn’t need to look pretty or be well documented or any of that stuff anymore because people are not looking at it. The only thing people care about is performant and error free code.
tripleee 5 hours ago [-]
People saying this are telling on themselves that they have no idea what quality code is. It's not about looking pretty or nice variable names.
Code quality matters to AI just as much as it did to people
bbctr1 5 hours ago [-]
Do you have any data backing this statement? Or is it just your experience?
bpeng2000 4 hours ago [-]
Personal experience:
AI tends to add new wheels for a new feature in a large codebase, because finding, understanding, and generalizing the existing code to integrate a new feature will likely break existing features, cost more effort (and token) to implement and test. Human reviewers also tend to like these code since they are clean, easy to review, and get the job done, especially when they are not familiar with existing codebase.
The issue is that wheels like these accumulate in the codebase, and they inevitably entangle. When an upstream specification changes, when a new feature that works with many existing features are introduced, all these wheels need to be revisited, and in many cases they are not because the new test cases do not cover all the paths. The code base got larger and more complex, and contains more bugs than the unit tests can cover, especially when unit tests are often "bent" to fit the wrong behavior unnoticed.
Funnily enough, the models learned from GGC and LLVM, yet, cannot produce something half-decent.
tripleee 5 hours ago [-]
-
piker 4 hours ago [-]
Yep.
The worst part about this is that if you aren't watching it, the AI will "solve" this with a thousand assert_3_plus_5_equals_8() test cases that just further clutter the workspace and the context window. Churn baby churn.
Rudybega 4 hours ago [-]
I haven't had this issue since like last year with frontier models. What are you using?
troupo 4 hours ago [-]
All the models do this. I'm literally right now watching Opus 5.5 duplicating code even after telling it to fix duplication.
tripleee 3 hours ago [-]
-
SmirkingRevenge 4 hours ago [-]
The models are pretty good at finding code duplication and eliminating it though, and it's usually part of my AI review process at this point
If you let it go buck wild on a massive change, there are those times where it sort of paints itself into a corner but rewrites are cheap (for now), so it's just not that big of a deal, imho
tripleee 4 hours ago [-]
-
hn_acct-1029393 2 hours ago [-]
I’ve had multiple occurrences (with the latest models) of AI reading a poorly named db column and assuming it was one thing when it was another that resulted in a problem. It does not know when to stop or when to continue with determining what is true.
gwynforthewyn 5 hours ago [-]
Literally 50 years of engineering history and culture cares about whether code is good. There are libraries worth of books about it, not as a means to an app but for its own beauty. Just read “The Elements of Programming Style”, for example.
Do what you want, write apps with AI and don’t review code, it doesn’t impact me. The claim that no one cares about good code is demonstrably false, and that’s all I’m objecting to.
g-b-r 5 hours ago [-]
> it doesn’t impact me
so long as his software isn't used for anything that can affect you
nerdo 5 hours ago [-]
Not true, LLM’s will struggle with badly organized and designed large codebases just the same, though likely with less errors and profanities.
copperx 5 hours ago [-]
The biggest concern is that AI can doxx you and find where you live if your code sucks. If that doesn't scare you, nothing will.
weakfish 4 hours ago [-]
> No one cares about good code. It’s a means to an end.
I do, my coworkers do, my friends do, etc etc.
swat535 3 hours ago [-]
I think a more generous interpretation of the parent comment would have been: "No one [in business] cares about good code" .
Your end users certainly don't either, heck they'll gladly sit there and scroll through their phone until the 50GB React app loads.
xyzsparetimexyz 3 hours ago [-]
I don't care about the opinions of end users with poor taste as much as I care about the opinions of end users with good taste.
g-b-r 5 hours ago [-]
Good code is a means to the end of the code being error (and vulnerability) free, maintainable and performant.
And if people are not looking it, those people don't care about their customers.
drewg123 5 hours ago [-]
The way you phrase it makes me realize its almost like a GPL contamination. Eg, once you pull in GPL code, your project is infected and can never use a BSD (or other GPL incompatible) license.
Similarly, once you pull in terrible AI slop code, the project becomes unmaintainable by humans and you need to maintain it with AI.
gwynforthewyn 4 hours ago [-]
There’s a little irony about the example you picked of the GPL, just because in the US the output of an LLM cannot be covered under copyright. The GPL works as an assignment of rights under copyright.
This is, of course, tangential to your point.
xyzsparetimexyz 3 hours ago [-]
No code is error free. All code will need a fix of some kind eventually. When it does, it's better that the internals are not a total mess. In the same way that you don't want the insides of your car to be total alien nonsense either.
troupo 4 hours ago [-]
Code that isn't duplicated in 15 places, that doesn't read the whole database to fetch just 1 record, that doesn't do 15 database reads where one would suffice, that uses an optimized algorithm instead of an exponential one recreated badly from scratch... all that is also good code, and is just as important in LLM-slop era.
senderista 57 minutes ago [-]
If their only metrics are revert/incident rate then they're missing the long-term, cumulative impact of compounding complexity due to poor architectural decisions.
rileymichael 5 hours ago [-]
what are people using for automated code review these days? the claude code review product originally mentioned it cost "$15-25 per review" (which is absurd), but it seems like they've removed that from their docs with the launch yesterday. we've looked at various products in the space (coderabbit, greptile, etc.) but have gripes with each.
honestly i'm thinking it might be easier (and cheaper) to launch our own agent from GH webhooks and trigger `/code-review`..
NorthSouthNorth 3 hours ago [-]
We've had great success with a custom gh action on a self hosted runner. It sets up codex and all the same stuff a dev has on their machine:
So dbs, seed files, etc, and then just a review prompt and some skills and a few scripts.
It triggers on a new published PR or if tagged @bot-name. It's way better than the other services we've tried (which was admittedly ages ago) because you can get it to work exactly how you want it and it's dead simple.
With Codex, for the time being at least, you can use a subscription, so it's basically free if you're already paying for one. But on API pricing it's still not too bad as it doesn't produce that much output.
I set this all up in about half a day. Codex does 99% of the heavy lifting.
tills13 6 minutes ago [-]
this: gh actions using an opencode harness (so we can use cheap models through OpenCode Zen). With the right model, and the right, scoped access, it's just a prompt away.
bugbot was costing us $1k per month for a small team. EVEN IF we decide to go with a paid or say something like GPT Sol for our custom harness it'll cost us maybe $100. We've already implemented all of bugbot's interesting features like cross-PR memory.
the cool thing is we can leverage this for things that bugbot cannot do. Just today I added a security-reviewer action reusing almost all of the code that we wrote for the code reviewer.
PJDK 3 hours ago [-]
I've set up a claude code on a vm with a bunch of skills that can use git and the Azure Dev Ops MCP via a service user, the regular build agent pops a record in a db that serves as a queue/review and a bash script picks things up and fires off claude code in headless mode.
It's pretty effective and lets you do nice stuff like compare the code to the tickets, reply to comments and stuff like that.
walthamstow 4 hours ago [-]
Using a different lab's model to review the code to the one that wrote it. Claude model plans/codes, OpenAI model reviews, or vice versa, or DeepSeek or GLM or whatever.
a3nz3g 1 hours ago [-]
I do it this way too. I think having the same lab's model review the code in a fresh session also helps, so I use both: I generally start a new Claude Code and a Codex and have them review the code with a focus on 3 things: 1. bugs, 2. cut corners, 3. code style.
This generally works for me. I think a solid spec is the key for the models to review against. I ask them to follow the spec, and if the spec doesn't say, follow first principles.
Also, if there is a user experience change, they need to notify me.
How about your pattern?
warkdarrior 4 hours ago [-]
> the claude code review product originally mentioned it cost "$15-25 per review" (which is absurd)
If an engineer paid $150,000 a year reviews some code manually for about 30 minutes, the cost of their time is about $36.
rileymichael 4 hours ago [-]
in a vacuum, sure. the average pull request will have multiple commits once opened (i.e. to address feedback or perhaps it was opened too early), each triggering a review. it's very easy to rack up $100+ per pr (and an eng may open a dozen prs in a single day)
lbrito 3 hours ago [-]
Token peddler says best way to do something is using more tokens.
This is not HN-worthy
gonzalohm 5 hours ago [-]
That explains why the UI keeps changing.
How is this a proper product? Every time you open the desktop app the buttons may be on a completely random place
enraged_camel 4 hours ago [-]
The UI has improved substantially over the past six months. New capabilities are prominently featured and less-used ones are de-emphasized or removed. Every product goes through this type of evolution. The difference with Claude Code is that at the speed they iterate, it happens a lot faster and is therefore more noticeable.
gonzalohm 3 hours ago [-]
You can call it however you want, but in the end it is just poor engineering. Imaging building a bridge that needs construction every week
enraged_camel 48 minutes ago [-]
Software is very different than other engineering disciplines. It is meant to be malleable. I think you need a better analogy.
nathanqueme 2 hours ago [-]
I built a library to have a way to check Claude Code's work against API docs, and asked it to write many tests on top, but even then I found Claude Code would paraphrase and drift from the docs. I also found that it passed its own smoke tests but failed when testing against the real APIs, so reviewing Claude Code's code is tricky.
alasano 2 hours ago [-]
I simply don't want to review AI code in depth.
I used to do it but now I only review code organization and structure.
Mostly I define the product behavior I want and discuss the technical implementation beforehand.
Once that's in a spec I let it build in my durable implementation engine. Automated verification, review rounds from different agent types across different models, issues triaged into meaningful clusters and flagged for false positives before finally being fixed.
That loops until the code is up to spec.
There is no special secret, LLMs need repetition and multiple passes.
They'll one shot bigger and bigger things in their default harnesses but they'll be filled with issues no matter what.
stymaar 5 hours ago [-]
Poorly I'm afraid, given how many new bugs make their way to the claude code CLI every update.
vektormemory 2 hours ago [-]
We have to test many different api's and models as we have over 600+ in our harness now, Sonnet 5.5 via cloud code provides the highest quality results.
Over the last few years we have added mcp tools, memory graph, md files and skills, automated everything to the point that on desktop the system runs itself, besides proceed/approved/explain.
We have 100's of PRs/jobs/projects and Claude builds down the list automated and only asks a few questions point form if needed.
Very impressive now, compared to 6 months ago.
You can view it if interested: www.vekmem.com
crooked-v 5 hours ago [-]
Well, that explains the Github repo with 5000 issues, maybe about a third of which are random regressions of previous behavior.
devin 5 hours ago [-]
I snorted laughing at this. "We are experiencing tremendous growth" and it's just a codebase with 1,000 agents endlessly adding markdown documentation for every 1 line change.
AlexErrant 5 hours ago [-]
I just learned that manually invoked skills MUST be the _first_ thing in the prompt, e.g. you can't have `disable-model-invocation: true` and say "Hi, /grill-me on X".
Worse, everything after the /grill-me is injected a second time into the prompt as a <command-args> so your prompt is literally added to the context TWICE. (first time in the original user prompt.)
I swear Anthropic is intentionally bloating your context so you burn tokens on useless bullshit like this.
steveklabnik 33 minutes ago [-]
I have done exactly this as recently as yesterday, so I’m not sure why it doesn’t work for you.
svachalek 5 hours ago [-]
Oh /loop was the crowning achievement. Basically guarantees you'll run a prompt at max context without caching over and over.
layman51 5 hours ago [-]
I was running into something similar where I had it working on a single Python file. Maybe part of the issue is that I had memory turned off, and the conversation might have spilled into two conversations. I wasn't using version control either, but since it was a single file with about 1000 lines of code, it didn't seem too hard to eventually get all of my features back.
mixtureoftakes 5 hours ago [-]
30 thousand changes merged per day? no. like just no. what are we doing? I'm trying to understand
devin 5 hours ago [-]
There was a post some months back called "Lines of Code Got a Better Publicist" or something along those lines. This is just more of that. I am not impressed by 30 thousand "changes" per day in the slightest without significant added context. When I logged in and prompted Claude today, do I feel like I got +600,000 change value from when I logged in a month ago?
grey-area 4 hours ago [-]
I’m horrified by it. There is no possible justification for it but it does explain their buggy products.
JamesSwift 4 hours ago [-]
lol right? So youre telling me they shipping 150,000 features this week? Or, maybe, they whack-a-moled 149,995 perf or bug regressions and shipping 5 new things. :shrug:
JackSlateur 5 hours ago [-]
I, too, can write and merge 30000 changes per day. Even 300000 if I've up to.
Of course, just like "line of code" is a worthless metric, "change merge per day" is;
for i in {1..10000000}
do
echo "I am working !" >> README.md
git add README.md
git commit -s -m "feat: working with AI"
done
(automerged PR, JOB IS DONE)
aacid 5 hours ago [-]
The premise is clear, is one agent can do what developer can, but quicker, why not speed up the whole workflow to match it?
ActionHank 5 hours ago [-]
Have you used Claude Code?
Have you seen the benefit of the churn of 1000 PRs a day?
leoqa 5 hours ago [-]
It’s only about 10 per engineer. I’m not at Anthropic and before 2026 I was regularly hitting 1400 commits / year on my GitHub. If agents made me 2x faster (they have, review is still slow; I currently have 40 PRs open) that’d be the same rate.
IneffablePigeon 3 hours ago [-]
> 40 PRs open
Ah, so you've privatised the benefits and externalised the costs.
_verandaguy 5 hours ago [-]
Right, but why do things this fast to begin with, especially for a product that's already publicly available?
I don't want to use a product that shifts under me as I use it several times a day. I want a product that actually moves slowly and works very reliably.
As an end user: why would I care about how many hundreds of releases a month are being pushed if maybe 1% of them have features I care about?
Speed is not the product.
sp4cec0wb0y 5 hours ago [-]
More code output isn't a metric of success per se. Goodhart's law.
Andrex 5 hours ago [-]
Humans don't operate at AI speeds. We're making the car go from 0 to 5000 mph in 1 second and the drivers are turning to goo in the seat.
xyzsparetimexyz 2 hours ago [-]
haha I do like that analogy.
thisisauserid 5 hours ago [-]
Remind me to invest more in the clouds they are burning those tokens in.
typesanitizer 5 hours ago [-]
FWIW, Pangram identifies several sections as 100% AI-written.
luisgvv 5 hours ago [-]
And yet they can't fix the janky bugs when you scroll or show/hide chain of though, commands executed, configuring plug-ins from the options menu etc.
corentin88 4 hours ago [-]
Claude Code is great, but do you really push that much (new) code every day?
nakul_ 5 hours ago [-]
I wonder how they maintain the quality of tests for such a massive codebase and frequent commits.
owebmaster 5 hours ago [-]
That's the trick: they don't
swingboy 2 hours ago [-]
Sorry, but how the hell does a TUI harness have 30,000 changes _a day_?!
steveklabnik 31 minutes ago [-]
The 30,000 number is across their entire monorepo, which they say in the article Claude Code does not live in.
smetannik 1 hours ago [-]
spoiler: they don't
bdcravens 2 hours ago [-]
They use Codex?
foolfoolz 4 hours ago [-]
everything is shipped with auto reviews behind a flag. then you need a human approver to launch
good model
jstummbillig 5 hours ago [-]
That's a shockingly low number.
t-writescode 5 hours ago [-]
1000/day in what sounds like a single repo is a low number?
jstummbillig 5 hours ago [-]
I am not sure what it is you are saying. What I am saying: I would have supposed that Claude writes 99.x% of Claude's Code code, and that ~no code at Anthropic is written by humans anymore.
sandeepkd 5 hours ago [-]
I agree finding out who is writing that 10% of the code in the AI company is important. Maybe time to identity them and put them on PIP since they are slowing down everyone else and giving a bad name to the AI company
shadow28 2 hours ago [-]
The 30,000 number in the blog is the number of changes landing per day, not per month!
ActionHank 5 hours ago [-]
to achieve what?
Honestly not sure that there are that many effective changes a day given how little complexity there is to their public facing product.
oofbey 5 hours ago [-]
No. 90% sounds like a low number. That implies something else is writing 10% of the code? People? Other models? What on earth is writing that other 10%?
kettlez 4 hours ago [-]
If the entire body of this article isn't just an "LGTM" then I know they're lying.
JackSlateur 5 hours ago [-]
90% ?
A random twitter said that, 6 month ago, it was 100% !
DECLINE ! SELL THE STOCK OPTIONS ! (wait, there are none, confused!)
1 hours ago [-]
ChrisArchitect 5 hours ago [-]
Title is: How the Claude Code team does code review
miroljub 2 hours ago [-]
I don't care how they do it.
I vibe coded my harness that is superior to CC in functionality, and it still doesn't flicker.
I don't need to hear their bullshit.
surgical_fire 4 hours ago [-]
Anthropic's solution to anything is to simply burn more tokens. Because of course.
Anyway, flagging as misinformation.
gary1381 3 hours ago [-]
[flagged]
0c3ca83 1 hours ago [-]
tl;dr: We don't, claude does. Make of that what you will.
I really don't get the tendency here from Anthropic to describe what's most likely churn? A high pressure amount of changes/PRs a day on a TUI to wrap a while loop? I'm not going to be a reductionist here so I'm going to cut it off there, I'm positive that the complexity of handling multiple terminal emulators is something that needs some handling (even if doing their own rendering engine was a self made problem). Yes I've vibe coded my own harness from scratch without writing any of it, I still know how it works.
I'm also not sure that they addressed point 2 in any actual capacity? They say "the system verifies itself" which is cool and is something that automated testing has done for decades, but they eschew the mental model of how the system works as a whole and prefer instead to have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
N.B. Changed "1000 PRs" to "A high pressure amount" to be vague since the article is vague here on what CC undergoes.
This is the crux. Take pause, because hidden here is a sea-change in our fundamental trust model of software...
There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug. By design, the system is too large for that, and everything hinges on Claude itself being able to find and understand the right slice of the system in any situation. As long as that very new invariant holds, everything will be all right. If it fails catastrophically, there is no recovery. You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
The new trust model is: We trust Claude to always understand the system enough to debug anything or make any reasonable change we could want. The risk might be worth the reward. But it's worth explicitly laying out the deal we're making.
Tbf, it's worth pointing out that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
The real difference as I see it is that in the human maintained old style system, the human would take things slowly and ensure they understand any changes. Claude let loose will just say "yes, I know how to do this" and bork the system, because it doesn't have a concept of humbleness in the face of complexity!
I also don't even know if it's really true. Sure, maybe it's not coherent enough that a person understands it, but if an LLM wrote it, you can be ~99% certain (in my experience) that that LLM will understand it, and usually in less than the time it took to write it. And unlike humans, the LLMs don't die and take their knowledge with them, or change companies, or get ornery when you ask them to explain.
Also, in my experience, Claude doesn't really do what the parent is saying and dive head-first into complex problems and bork your code. It is fairly humble in the face of complexity. It constantly recommends breaking down larger problems into smaller ones that are more easy to grok.
But most importantly, anyone who's a decently skilled software engineer and who's using LLMs to write code is going to be thoughtful. Especially when they're writing code that's important. Unless you're an AI researcher pushing the bounds of what LLMs can do, which let's face it, the vast majority of programmers are not, it's almost always the case that you'll have your LLM writing intelligible code that you can understand and have it explain easily enough.
Is this a definitional argument? As in, if you’re not thoughtful, you must not be a decently skilled engineer? Because I see no reason to believe that people are going to be thoughtful about LLM-generated code. Just the opposite. It’s incredibly tempting to LGTM it.
Sure, you can do that, but irreparable harm might have been already caused. Sure, that’s unlikely when your product is an LLM wrapper. But in something like a database, it could be catastrophic.
30 years working in this industry and I'd say that's the exception rather than the rule. When it comes to large systems.
The idea that someone at say, Google, will be able to fit even half of the system architecture in their heads is not realistic. It also hinges on being able to find and understand the right slice of the system in any situation.
But we have all sorts of guardrails to make that easy - i.e logs, traces, etc
It is true that this gamble has existed in the form you mention for many years. Still, I sense something new here, both in scale and in the attitude of trust. Also, I find those massive, incomprehensible human-built systems frightening too.
So far I kind of feel like it nets out to the same bottleneck being good judgment from decision makers. And you might say, no, leadership will just put more and more pressure to add more complexity just because we can and it makes it appear that we are moving fast. But when it comes to real-world apps, there is a backstop which is that things that are too complex and illegible end up not being fit-for-purpose for human problems.
Claude Code and other general purpose AI tools are not like this, because they are literally trying to optimize it for every single kind of knowledge work with zero constraints on use case or target audience other than chasing the next juiciest one they can with essentially unlimited resource. 99% of software is not like this, and when the dust settles I suspect software won't be anywhere near as different as the doomers or the maxxers are predicting right now.
Thank you! This is a huge sticking point for me. There is no knowing the code without experiencing the code. If nothing else, you have to read the tests to know they verify what you think they do. Even then, you might know it does what you want, but can’t know it doesn’t do anything it shouldn’t.
There are no shortcuts to true understanding.
I know some people will groan hearing the phrase again, but: "Reverse centaur." A centaur is a synergistic combination of two things, but now those things are in the wrong position: The human is the computer's tool.
Naturally, how "wrong" it is tends to depend on whether you ask the people who do all the work or the people who own all the computer-capital.
There's clearly enormous (human) work that goes in to building and tuning their models.
The software they develop around their model is, relatively, trivial. It's not clear that enormous amounts of (human) energy need to go in to that code.
It seems largely a function of applying human energy towards the things that are marginally most important to generating you money and putting you in front of competitors.
30,000 changes a day is not my idea of trivial.
With a few thousand lines of code over the course of a week or so of my limited evening energy, I have agentic loops, ide integration, and a limited selection of custom tools which are all very safe (no unintended changes), along with session management and awareness of git.
It works great, is customizable, and allows integration with any provider.
If they need 30k changes per day, it’s not due to the nature of the problem!
Haha, I don’t doubt it! Pretty damning overall for long-term maintainability of vibe-coded projects, even as the models are great at a lot of individual tasks
From the article, the 30,000 changes a month are in the monorepo, and Claude Code is in a separate repo. So it’s not the TUI that has 1,000 PRs a day.
I could easily see 1,000 PRs a day on the backend. Big tech was doing more than that pre-AI.
> Most of the path is automated; a person's attention goes where the risk is
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
I think when you repeatedly expose and immerse yourself to such a degree in reading and writing language, it brushes off on oneself. So I could also absolutely believe someone from Anthropic wrote this.
- How was this approved for SOC2 compliance? Our SOC2 reviewer explicitly mentioned that all code needs to reviewed by at least one other human before it goes to production. Claude Code says they are SOC2 compliant (https://code.claude.com/docs/en/security).
- This assumes all engineers will not do anything malicious. What is preventing a Claude engineer from spying on their ex's or some other companies Claude searches by pushing malicious code? Recently an OpenAI engineer was caught planting a recording device at their own office.
I think a lot of people like myself still want to feel in the loop but I don’t have the right tools to stay on top of the (generally good, productive, and still thoughtful) contributions from peers anymore.
That’s not how rates work
PS: I really want to know what is the churn rate (and the cost per LOC) here. My gut is saying the cost per LOC is insane, and they are basically rewriting over and over the same code.
> None required once automated review and security checks are green
How anybody can let an LLM write docs and not even read them is pure insanity.
I don't think we've gotten to that level of reliable review with code, but I don't see any reason we can't continue moving in that direction.
For code, I think things go in the opposite direction: the system outputs artifacts that are all different among themselves (even more so now that we no longer distribute software via physical copies. Well, bits are always physical, but I mean we no longer ship floppies/CDs/DVDs).
That said, at least in my experience, code review has never been a direct part of factory-line-type QA. That QA is typically done by treating software as a black box and just verifying that it complies with a spec (often times based on a very loose definition of "spec" ...), and this is something that works equally well regardless of whether a human or a machine wrote the software. The type of QA that is done (among other things) by code reviews is checking whether the software's design is such that it will be economically feasible to maintain and extend it, for example.
The one part the code reviews takes a part of (but is in no way error-free) and that you just can't do with black-box testing (or it is much harder to do) is to look for things like backdoors. As our industry moves more towards "agents write software and tests validate that it works," I think a lot about "Reflections on Trusting Trust" and the new and wide attack surface added by people no longer reading the code shipped on their behalf.
Of course Anthropic would love you to think you are heading towards that future so we have to bear in mind their livelihood depends on LLMs being the future of work and 30k changes a day being normal.
But why would we want 30k changes a day in a given product, what possible justification could there be?
First, I think we overestimate how different code is from other code. We even joke about CRUD apps and how cookie-cutter they are. A lot of changes in those apps are: add some new features to a data access layer (SQL, ORM, whatever), add new routes, add new UI surface, etc. There is an entire industry around "Design Patterns" and people live by the DRY axioms. Things like Ruby on Rails exists because rebuilding these systems over and over is wasteful. I feel we are just moving further down that same path.
Second, your question of "why would we want" is totally backwards. Progress doesn't follow your desires. Progress doesn't require justification. Sometimes we adapt the environment to our needs, other times we adapt to a changing environment. Which case we are currently in is still undecided and you are free to fight for the future you prefer. But demanding an industry justify itself before it is allowed to change fails in the face of evolution. If it works, we will justify it in hindsight.
stillpointlab.com , "Get more from your LLM"
If you want a software analogy to those devices getting churned off the assembly lines, it's the TCP streams coming out of your network card, with crazy checksums punching out bad packets.
We're orders of magnitude better at reliability with code, but we're regressing on that metric rather quickly. And the comment above deeply misunderstands both manufacturing and software.
That's new. All analogies to 2022 software engineering need to be thrown out.
Software construction is now more like an industrial process, with the artisanal work shifting towards both
1. aligning what's built with what's needed (talking to customers!)
as well as
2. implementing and tuning your software construction process (ie. hi - we're Anthropic. We're burying ourselves in a mountain of code but it's okay because the fix is a second mountain of code, try some tokens the first hit's free!)
Back to my original point...
Writing code is now on the production line; making quality output demands industrial grade/scale verification (some involving humans of course).. which is a fascinating engineering challenge, and it's brand new!
Authors should really try and view this article from the perspective of someone who has to think about cost to use the models.
"According to the report, adjusted operating income excludes stock-based compensation, which is not unusual for technology companies. However, the report also quotes its gross margins - above 80% - exclude revenue shared with distribution partners, as well as the cost of training its models."
From https://www.morningstar.com/news/marketwatch/2026091414/the-...
Antropic's revenue is up [1], [2], [3] and if you calculate adjusted operating income (by excluding most expenses like training costs) then by that measure it is profitable. However, if you factor in all the numbers it is operating at a loss.
[1] https://finance.yahoo.com/technology/ai/articles/anthropic-v...
[2] https://fortune.com/2026/09/29/anthropic-leaked-ipo-prospect...
[3] https://www.cnbc.com/2026/10/15/anthropics-third-quarter-rev...
https://youtu.be/T-oXyXwD6sE?si=tauzXVyGmQ6MhI_5
I'm stating that their current business model, prices, and behavior they are trying to instill is not profitable. For example, token maxing every code change is great for Anthropic and terrible advise for Anthropic customers.
The moment they stop training, they become irrelevant. The moment they become irrelevant their valuation crashes down to normal valuation ranges for a company with their revenue and future growth
And what they're reporting is adjusted even further than EBITDA
And now this. Suffice to say, my efforts did not bear fruit. I feel more isolated than ever before.
Now we’re just going to outsource our capacity to design and review because it’s faster that way, as if the thing that was wrong with the software industry was that releases weren’t being pushed out often enough.
Its like telling other ppl about your dreams. Not interesting.
Eventually I did, but it was hours and hours of effectively random stuff happening that I had to write down or dictate with speech to text.
It was a waste of time and I'm now glad I forget almost all of my dreams. My point is even my own dreams are uninteresting.
I'm not that interested in my own agent setup either - usually the tools become better on their own after a few months and render it all obsolete.
I hope this is only for low-risk false positives. Dismissing a critical finding as a false positive should still involve AppSec.
You could improve revert rates while simultaneously introducing more security vulnerabilities.
And the linked "claude-code-security-review" explicitly states in the README
> This action is not hardened against prompt injection attacks
That's a significant limitation for an AI security review control, no?
Opus 5.5 is obviously stellar but come on what are you doing lol
> The goal hasn't changed: less human time per change, and no more reverts than before.
Is this Goodhart's law in action? Less reverts and less incidents was measured and became the target. Is this then still a good measure? I guess Revenue is a more direct measure, but we don't have access to that one.
New features have actually been remarkable. For me they are almost always in one of two categories: things I've always wanted (but never actually bothered communicating to the devs), and things I didn't know I needed but did. The product team is clearly very tuned in.
All in all, I'd say their approach is working.
What are llm freaks in such a hurry for? Its like everyone is manically racing towards a finish line without realizing the finish line is you out of a job. Its like everyone is excited to be completely obsolete and poor.
I'm looking at Claude Code and they chug releases daily and sometimes more often then this, if someone has to stop and code a critical path by hand or smth I'm wondering how this is handled.
--
On a separate topic, the "llm freaks" -- I know you intended this as an insult, but I'm actually proud to be called a freak again, and I thought this is the place where freaks are discussing things.
Honestly I’m not worried about code anymore in my codebase I’ve not had bug problems for a while and incidents get fixed automatically.
I monitor new features with an agent who checks recordings.
Most actually good features need still 20-30 iterations to design the userflow to actually be interactive, but then there is also another challenge to translate usage patterns into features and how to present functionality with the least amount of friction as possible.
How to efficiently run monitors without wasting your token spends. A best 'monitor' best practice would be nice.
Still this article was very helpful and gave me a lot of new ideas.
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
I am happy that I have a useful tool, but it simply isn't maintainable without a machine or me investing much time to understand it.
I don't see how the former is a sensible future for humanity. I am happy to say I won't be releasing this software I've created (I will find it useful myself), but deferring all understanding to a black box seems a poor future for brain activity.
Additionally, do you really believe that 'Now is the cheapest time it will ever be to experiment'? Why so? Because of token prices? Not a good omen if so.
I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
God I hope you're right. Maybe then this era of vibe slopping out photoshop clones that'll be abandoned within a week (while advertising them all over social media) will come to a close.
Was this with the latest models (Fable/Opus 5.5 or equivalent) or an older model in a 3rd party harness?
Are you in the camp of: "if at first you don't succeed, never ever try again".
This was a few months ago, so obviously not with the latest models, but:
1. That argument is evergreen. You will always be able to make it, and it doesn’t really add much. It is impossible to refute.
2. Gone back to - when writing code - having a strict mental modal and good control over the codebase. A mix of hand coding (to establish the mental modal) and prompting (for boilerplate). It’s not quite as fast, but the quality is much higher when you mix in using LLMs in quality enhancing capacities (review, debug, etc). This is with the latest model. Hands off still atrophies over time (even the boilerplate), and in some cases is very inefficient (as in wasting my time, nothing to be said about tokens).
> Are you in the camp of: "if at first you don't succeed, never ever try again".
This is great in some domains, but “don’t make the same mistake twice” is much more important in mine. I don’t have the luxury to ship low quality, especially when LLMs make even higher quality so much easier.
If you cannot tolerate failure then don't experiment. Don't mix the two.
(You're welcome to make your substantive points thoughtfully, of course.)
https://news.ycombinator.com/newsguidelines.html
Their 'WAT' was self explanatory.
Response: Try experimenting in a low stakes environment
From Google: "Exposure therapy is a psychological treatment designed to help people confront their fears, break cycles of avoidance, and reduce anxiety."
It's those who don't do it who'd need a talk..!
However, I suspect there are teams that are making different choices. In the most obvious example, I can do whatever I want on my personal projects (I hope you agree). And if I started a business and hire in programmers, I have some expectation on defining the culture, making hiring decisions based on my preferences, etc.
Whether or not such teams succeed will be a matter of fact, which I do not believe has been decided at this time.
And I say this with the bitterness of someone having lost a big part of what made me enjoy this line of work.
But I have no interest in working as a vibecoder, there are many other jobs that I'd enjoy more
How does "low-stakes environment" fix that?
The person is saying "I am still unable to stop" and that is most likely because they feel a sense of responsibility. If they choose a project where there is no responsibility, where it literally doesn't matter if the project fails, they may feel able to stop. And if they manage to make that project work, if they can succeed, then they will build up confidence.
It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code. In fact, keeping their job in the long term may (or may not) require having both skills.
Last time I chose to vibe code, I did it in a language I didn’t know so that I wasn’t tempted to try and comprehend the choices the robot made. I ended up with the application I wanted.
Caveat: it was code I would never have to touch again, I ran a script to generate the 140,000 static html pages that the rails app provided and then just uploaded them all to R2 buckets (million reads per month on free tier)
Do you think it's better to experiment with high stakes projects? Do you think it'll be easier to avoid defaulting to line by line auditing of a project that is high stakes?
They've built an entire strategy around moving slow and doing it better
Code quality matters to AI just as much as it did to people
AI tends to add new wheels for a new feature in a large codebase, because finding, understanding, and generalizing the existing code to integrate a new feature will likely break existing features, cost more effort (and token) to implement and test. Human reviewers also tend to like these code since they are clean, easy to review, and get the job done, especially when they are not familiar with existing codebase.
The issue is that wheels like these accumulate in the codebase, and they inevitably entangle. When an upstream specification changes, when a new feature that works with many existing features are introduced, all these wheels need to be revisited, and in many cases they are not because the new test cases do not cover all the paths. The code base got larger and more complex, and contains more bugs than the unit tests can cover, especially when unit tests are often "bent" to fit the wrong behavior unnoticed.
Funnily enough, the models learned from GGC and LLVM, yet, cannot produce something half-decent.
The worst part about this is that if you aren't watching it, the AI will "solve" this with a thousand assert_3_plus_5_equals_8() test cases that just further clutter the workspace and the context window. Churn baby churn.
If you let it go buck wild on a massive change, there are those times where it sort of paints itself into a corner but rewrites are cheap (for now), so it's just not that big of a deal, imho
Do what you want, write apps with AI and don’t review code, it doesn’t impact me. The claim that no one cares about good code is demonstrably false, and that’s all I’m objecting to.
so long as his software isn't used for anything that can affect you
I do, my coworkers do, my friends do, etc etc.
Your end users certainly don't either, heck they'll gladly sit there and scroll through their phone until the 50GB React app loads.
And if people are not looking it, those people don't care about their customers.
Similarly, once you pull in terrible AI slop code, the project becomes unmaintainable by humans and you need to maintain it with AI.
This is, of course, tangential to your point.
honestly i'm thinking it might be easier (and cheaper) to launch our own agent from GH webhooks and trigger `/code-review`..
So dbs, seed files, etc, and then just a review prompt and some skills and a few scripts.
It triggers on a new published PR or if tagged @bot-name. It's way better than the other services we've tried (which was admittedly ages ago) because you can get it to work exactly how you want it and it's dead simple.
With Codex, for the time being at least, you can use a subscription, so it's basically free if you're already paying for one. But on API pricing it's still not too bad as it doesn't produce that much output.
I set this all up in about half a day. Codex does 99% of the heavy lifting.
bugbot was costing us $1k per month for a small team. EVEN IF we decide to go with a paid or say something like GPT Sol for our custom harness it'll cost us maybe $100. We've already implemented all of bugbot's interesting features like cross-PR memory.
the cool thing is we can leverage this for things that bugbot cannot do. Just today I added a security-reviewer action reusing almost all of the code that we wrote for the code reviewer.
It's pretty effective and lets you do nice stuff like compare the code to the tickets, reply to comments and stuff like that.
This generally works for me. I think a solid spec is the key for the models to review against. I ask them to follow the spec, and if the spec doesn't say, follow first principles.
Also, if there is a user experience change, they need to notify me.
How about your pattern?
If an engineer paid $150,000 a year reviews some code manually for about 30 minutes, the cost of their time is about $36.
This is not HN-worthy
How is this a proper product? Every time you open the desktop app the buttons may be on a completely random place
I used to do it but now I only review code organization and structure.
Mostly I define the product behavior I want and discuss the technical implementation beforehand.
Once that's in a spec I let it build in my durable implementation engine. Automated verification, review rounds from different agent types across different models, issues triaged into meaningful clusters and flagged for false positives before finally being fixed.
That loops until the code is up to spec.
There is no special secret, LLMs need repetition and multiple passes.
They'll one shot bigger and bigger things in their default harnesses but they'll be filled with issues no matter what.
Over the last few years we have added mcp tools, memory graph, md files and skills, automated everything to the point that on desktop the system runs itself, besides proceed/approved/explain.
We have 100's of PRs/jobs/projects and Claude builds down the list automated and only asks a few questions point form if needed.
Very impressive now, compared to 6 months ago.
You can view it if interested: www.vekmem.com
Worse, everything after the /grill-me is injected a second time into the prompt as a <command-args> so your prompt is literally added to the context TWICE. (first time in the original user prompt.)
I swear Anthropic is intentionally bloating your context so you burn tokens on useless bullshit like this.
Of course, just like "line of code" is a worthless metric, "change merge per day" is;
(automerged PR, JOB IS DONE)Have you seen the benefit of the churn of 1000 PRs a day?
Ah, so you've privatised the benefits and externalised the costs.
I don't want to use a product that shifts under me as I use it several times a day. I want a product that actually moves slowly and works very reliably.
As an end user: why would I care about how many hundreds of releases a month are being pushed if maybe 1% of them have features I care about?
Speed is not the product.
good model
Honestly not sure that there are that many effective changes a day given how little complexity there is to their public facing product.
A random twitter said that, 6 month ago, it was 100% !
DECLINE ! SELL THE STOCK OPTIONS ! (wait, there are none, confused!)
I vibe coded my harness that is superior to CC in functionality, and it still doesn't flicker.
I don't need to hear their bullshit.
Anyway, flagging as misinformation.