- 0 Posts
- 2K Comments
11·8 hours agoHuman life is more valuable than money, property… but suicide is a reality of the human condition. It’s not something that has been meaningfully improved with modern psychology or medications. It’s worse among the poor but significantly prevalent across the wealth spectrum. Anything that interacts with people is going to interact with suicidal people.
Next time you are in a train station, count 100 people as they pass you - same effect. If you’re in a major station, look out across the big hall full of people, estimate how many hundreds of people you are looking at, and take in the reality of what 1 in 100 actually means.
LLMs are so sychophantic
that they would have articulated the reasoning of the systems engineers who would have been lobbying for better human factors, training, and visibility of the potential problems. If the systems engineers hadn’t been downsized that cycle, it’s a here today gone tomorrow discipline at many companies.
Frankly, I think the Star Wars trope of humans disrespecting droids was prophetic. We’re coming to a world where an LLM will spout off far more about a topic than you want to hear, and your attention just wanders off of them - rudely if they were an organic humanoid - without a further thought, until you want them to talk to the navicomputer or something else that they’re suited for.
Never tell me the odds.
There’s many good reasons why even Bachelor of Science degrees have such a load of core courses across the academic spectrum.
You might want to rethink that position: https://www.anthropic.com/research/self-replicating-ai
I mean, it is how ice cream became so popular… https://www.nationalgeographic.com/history/article/ice-cream-prohibition-era-craze
However, the end of Prohibition was a much better idea than the start of it.
I’m sorry, who or what exactly regulates knives? Knife manufacture? Knife ownership? Even knife sales… If you’re referring to ballistic knives or switchblades, sure - those are like guns, but the 99.999% of knives in the world? The ones anybody with a piece of metal can hammer out and sharpen for themselves?
sometimes held accountable for their actions
Very weak theater.
boy, does it get you from point A to point B quick.
Not so long ago, face to face interaction was the only way business was transacted, people were employed and paid.
In today’s world, if we could get past the baseless RTO mandates and let the 60% or whatever it is of modern workers work from home or wherever is convenient for them to work, get the commuters off the roads, restructure the food distribution systems to eliminate that need for cars, we really could make a significant improvement in the environmental impact of cars and trucks.
In the world of AI, I suspect a huge chunk of current AI usage, energy burned in data centers, etc. is “wasteful” - investigational, experimental, learning usage. If we can get past that kind of usage and eventually use it for what it’s good for and leave it alone for counterproductive wastes of time, it should also be a clear net-benefit for “getting stuff done” not only faster, but more correctly and valuably. Which software is more valuable: one coded by humans over the course of a year, with a half dozen undiscovered security vulnerabilities, or one coded by those same humans over the course of a month with those vulnerabilities exposed and fixed?
If you’re concerned about the environment, those humans commuting to work for an extra 11 months add up to far more environmental impact than the AI they used to accelerate their work. Good luck getting their employer to give them 10 months off with pay for delivering the same valuable product faster and better.
There’s an example used in regulatory circles: electricity would never be permitted in homes under today’s regulatory standards - we only accept it and its hazards because it is so incredibly common and useful in daily life and we’ve been accepting it since before we cared so deeply about safety.
So, have today’s goalposts moved so much that “AI” no longer passes modern society’s definitions of acceptable? I feel like such a technology would have been cheerfully embraced in the 1960s.
I read it
Did AI translate it for you first?
Wtf are you doing eating with a fork that you couldn’t be doing with your bare fingers?
Preach. There was a lot of fuss made when I was in school about “humans are the only animals who use tools” - of course by then all kinds of animals had been documented clearly using tools, but they were still teaching the kiddos (self included) that “humans are special.”
I have been making and using software tools for 45 years, a tool is something you use to make a job easier, faster, cleaner, safer, a tool might make the impossible possible, but that’s far from a requirement for “tool-ness.”
he also introduced at least one bug/vulnerability in the form of a race condition.
You mean Dirty COW? CVE-2016-5195 The race condition lived undetected in the source tree for nine years until security researcher Phil Oester discovered it in October 2016.
Unlike this race condition in the age of LLM agents which never made it into a stable release.
Why the are you getting more text than you can physically review at your job?
Regulatory required documented training, old conservative company.
How are you sure LLMs are not missing anything if nobody actually has the time to read through all the text?
You’re not. Neither do the humans tasked with the job do a 100% thorough review. Side by side, the LLM catches more flaws than the humans do.
produce proper specifications that developers would implement without wasting time on specifications containing garbage.
Sounds nice. We’ve got ~500 employees at our “site” (which is now distributed across 3+ states with remote workers), of those, about 8 do software development, so developers end up doing virtually all of the translation of customer “needs” to specifications / requirements. We also have a marketing department that changes their mind about twice as fast as the product development / delivery cycle.
If I leave a webpage with a calculator running unattended it won’t magically persuade a random visitor into commiting suicide.
That’s a touchy subject. A friend of ours recently had her son die of drowing in his bathtub, he was epileptic, had a seizure and fell in the tub. There was no suicide note, it was ruled an accident. He used ChatGPT to write his own obituary, as well as a book explaining his life from his perspective - each seizure was “taking away a little of who he was,” and the seizures were getting more frequent, none of the treatments or medications slowed them. It’s pretty clear, to me, that he was going to commit suicide and/or die soon with or without ChatGPT. After he passed, his mother had a ton of grief / guilt about not being closer with him in his life, but then… he actively pushed her away, even violently. It’s doubtful he would have written or expressed what he did as completely without ChatGPT assistance - how much was him, how much was LLM hallucination? Does it really matter? He wrote the book, and he put it where she would find it.
The next time you go driving, count the oncoming cars. When you get to 100, realize this: statistically, one - maybe two - of the drivers of those oncoming 100 cars will, eventually, end their life by suicide. And those are just the deaths officially classified as suicide. Engagement with ChatGPT undoubtedly works both ways, some cases accelerating the process or even causing them to take their lives when they wouldn’t have, and other cases slowing or stopping the pursuit of suicide altogether. It’s going to be hard to say if Chatbots or other world events drive the statistical changes starting a couple of years ago. The data isn’t clear yet, but the concerns have certainly been raised, and I would assume the supportiveness of Chatbots should improve as a result in the future.
There are all kinds of aspects to software development. Under-training pilots and then having software lead them to crash a plane full of passengers counts as a software development fail in my book. Also a human factors fail, marketing fail, a management fail, a quality systems fail, probably a regulatory oversight fail… it’s up there with ignoring the guy telling the Space Shuttle mission control not to launch when it has been under 32F the night before, which wasn’t a software fail at all, but was a broad cultural failure. Boeing’s MAX-10 situation just happened to have software in the crux of the matter.
I do software development. When I see something, I say something. I’m frequently ignored. Several times over the years, I’ve had the people that ignored me come back asking for help fixing the issue, after they realized that it was more important than they thought it was when they initially ignored me. Luckily, none of the people who died in those situations died (or were harmed in any significant way) because of the problem, but people dying “near the problem” brought the seriousness of it to a level where it finally got addressed.
Added to this is the prevalence of people thinking that LLMs can allow them to do anything, whether or not they understand it
1971, Willy Wonka and the Chocolate Factory depicted a computer which could (but was refusing to for comedic effect) predict the location of the remaining golden tickets. There were, and probably still are, a shocking number of people who believe that was a realistic scene.
I have an operations manager thinking he can make pages for our Intranet - where does it store the data? How is access controlled? So many basic questions that an actual programmer would know to think of.
Option 1: hold his hand and educate him on what he lacks
Option 2: let him flounder and make fun of his mistakes (the making fun part should be done anonymously if you value your job)
Option 3: he’s bright enough to ask the important questions to you or the LLM and get reasonable answers
Option 3 has a very realistic chance of working, if 1) you have documentation for how you do these things today - even informal documentation, anything the LLM can read and extract the relevant patterns from, 2) the LLM must have access to that documentation, and 3) your manager needs the patience and perseverence to work the discovery process until it’s viable and sustainable.
Odds are, your operations manager is going to fall short on the patience and perseverence dimensions and offload the boring parts to you, after all why did he become a manager in the first place?
maybe better to just ask a pilot to do it.
Yeah, except you can’t make nightly backups of the plane’s passenger manifest. If you’re not making nightly backups, you’re already in catchup territory LLM or no.
it’s benefit to us (and the world) is nowhere near worth the monetary and environmental cost.
I suggest doing a little research into what exactly the monetary and environmental cost of YOUR usage is. People engaged in low-value usages (too many to even begin to imagine, Rule 34 needs expansion to start to cover them) are burning a lot of un-necessary tokens.
What I have read, consistently from various sources, is that my $20/month Claude code subscription burns $11 to 12 in electricity to serve my responses. My house burns over $13 in electricity per day in the summer, mostly for the air conditioning.
If you are generating tests with an LLM, it’s not deterministic.
Once the tests are generated, they have become deterministic - unit test code is checked in alongside the repository, CI (continuous integration) systems can prevent the code from merging to main branch until all CI tests pass. No LLM is involved in the test execution process, and if all LLMs stop functioning tomorrow, the tests still work just like they did before.
read anything an LLM wrote that’s longer than a hundered words? It’s usually immediately obvious
About 250,000 lines of code so far, and no… I don’t read it all - my personal music player isn’t that important, I just care that it works. Other more important projects I have read every single line. Even in the music player I have read a few hundred lines and skimmed a couple of thousand. Yes, the style is obvious. It’s what you might call “too complete” attempting to cover lots of edge cases that a human would usually neglect, though as human code is maintained over the long haul, it tends to mature in that direction. And just because it’s very complete doesn’t mean it’s perfectly correct, and that’s a major point of conflict with me and LLM code, I’m constantly asking it to simplify because the complexity is easy to hide mistakes in. You know what happens when you ask it to simplify? It simplifies.
people will also do reviews with an LLM and this sort of stuff will just accumulate in the codebase with nobody to reject it.
That’s a management / maturity problem. As new as all this stuff is, it’s virtually inevitable there will be an adjustment process. Our company runs on processes and procedures, they average 50 pages each and there are thousands of them. I am given almost 100 of them to “train on” on a near annual basis - whenever they get revised. No way in hell can I do an accurate review of 5000 pages of intertwined procedures, not code, natural language work instructions. LLMs can. LLMs are. Simple questions to an LLM like: “review this procedure, is it internally self consistent” will find issues, real issues - usually minor - but undeniable when you read the review, the problem it describes is absolutely present in the document, page 14 says “X” and page 23 says “not X” and the context is basically identical - so which are we really supposed to do: “X” or “not X”? Well, that depends. Depends on what? Additional context and detail that the already too long 50 page procedure doesn’t address. We just did an internal audit which turned up thousands of these mostly little issues. Addressing them is going to be… interesting. We’ve been in business over 70 years, we know we’re not going to tackle it all in a single year, there will probably be a plan to work down the worst of the issues, then a second round for additional polish, of course corrective actions to ensure that we at least don’t make things worse along these metrics with future changes… I bet by 2035 we’ll have 90%+ of today’s real observations fully addressed.
LLMs literally kill people both directly and indirectly
As does nearly every activity of life, both in human society and the natural world.
a huge chunk of those arguments are ethical about the environmental and economic impact.
I have never seen an economic or environmental argument that puts objective numbers in context, how much more energy, water, etc. is AI using or projected to use? It’s all handwaving as if every single ChatGPT query is burning down a Macaw’s home tree in the rain forest and polluting a small town’s water supply, single handedly raising the average ocean temperature 5C.
makes it seems that anyone that does not abide by AI is a fool and an elitist
Back in the late 80s, there was a large contingent of the workforce who felt they were “too old to learn the computer.” Linus, and I, were getting our starts around then and we were the young upstarts using this “solitare playing machine” to do “a few little tricks, not sure if there’s actual value to justify the cost.” Through the 90s there was a good 20% or more of the workforce that probably would have been more effective in their jobs if they would have “skilled up” on “that computer stuff,” but they didn’t and we all got through.
Adopting AI is going to be easier than learning how to use Win95 when you’re coming from dictation, letter openers and interoffice memos delivered on paper, but I won’t be surprised or disappointed if a significant chunk of the workforce stays away from AI for a whole variety of reasons, and that’s fine. I also highly encourage those using AI to ensure that it’s actually improving their product, not just making something passable faster.
makes how you wrote your matter for some people.
Pedigree and provenance, whose child wrote this paper, who they studied under, where they grew up, all of that matters, a lot, to a lot of people, in a lot of fields. In computer code, particularly in computer code like the Linux kernel, it shouldn’t.
Boeing managed to kill plenty of people without a single bit of help from AI / LLMs in their MAX 10 software.
Used properly, LLMs could have articulated why their approach would lead to multiple crashes.
Used improperly, any tool is dangerous, from a chainsaw to a plastic spoon, the potential for injuries and deaths are unlimited.
Agreed, although I worry about this use case the most for “brain rot” - that security pen testers will quit thinking and just use LLM agents instead. The state of the art needs both, not just one or the other.
Coders with brain rot only have to make working code that fulfills all the requirements, security scans have a much higher burden, and they are also more directly linked to expensive problems when they fail in their task.