Rendered at 22:01:58 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
thinkling 5 hours ago [-]
I had the opposite take on LLMs using Kahnemann's framework.
Maybe I need to reread his book, but I think of System 1 as "fast, reflexive, instinctive" mental activity, and System 2 as "slow, deliberative, rational, verbal" thought.
So exactly inverse to this article, I characterize LLMs as 100% System 2, and would say that what's missing in AI is the more "embodied" System 1 intelligence.
My current thinking is that the experience of consciousness is closely tied to that embodied, emotional side of our nervous system, and is 100% lacking in LLMs. Anthropic people will say "the models have neural correlates of emotion", and that's fine, our cortex also has neural correlates for emotions and for much of what happens in the cerebellum and brain stem. That doesn't mean that that's where the primary function is implemented.
mrandish 3 hours ago [-]
> So exactly inverse to this article, I characterize LLMs as 100% System 2, and would say that what's missing in AI is the more "embodied" System 1 intelligence.
I've read Kahnemann's book and how I've seen System 1 / 2 typically used in the context LLMs often makes them seem like poor fits to what's being referenced. The concept was created to broadly group types of human reasoning but it never struck me as rigorously defined enough to be a technically precise distinction. Applying what was a mildly useful directional thumbnail referencing human thinking styles to LLMs, which can mimic human output but via completely different processes, quickly goes off the rails. As we get more detailed and precise, the System 1 / 2 distinction seems to increasingly diverge from how LLMs behave.
A quick example is how basic Flash-type chat models without 'extended thinking' or MoEs will keep suggesting different solutions instead of stopping after a few failures to step back, gather more context and diagnostic data and evaluate changing approaches or questioning the question. When I do that, the 'interrupt' is often a 'System 1-ish' intuitive feeling like "I suspect I'm barking up the wrong tree" then, once stopped, I'll switch to System 2 to analyze. But in LLMs that "interrupt" behavior typically arises from adding more System 2-ish thinking. N=1 but that's not how I usually 'get there' and is orthogonal to what the System 1 / 2 model suggests.
furyofantares 2 hours ago [-]
Yeah.
System 2 is slow because it is sequential. Every output depends on all
the prior outputs. Like a transformer. (Even if this comes about as the result of massively parallel processes, both in the brain and in the model.)
System 1 is more of a parallel thing, operating on the output as a whole rather than in sequence. Much more like diffusion.
svnt 4 hours ago [-]
Yeah the framing seems inverted. We have a faster system 2 with llms, and system 1 is what decision models are hoping to be, but so far they just seem to be system 2 models without language output.
The terms don't seem to translate well, possibly because they were the wrong framing to begin with. Possibly also because the billions of years of progressively complex evolutionary constraint that brought about animal cognition does not seem likely to be reproduced by running backprop on human language artifacts.
questionable_ju 3 hours ago [-]
>Maybe I need to reread his book, but I think of System 1 as "fast, reflexive, instinctive" mental activity, and System 2 as "slow, deliberative, rational, verbal" thought.
Current LLMs are the opposite of slow and deliberative, though? With the current transformer architecture, it seems like its function is more or less "look at this text and make a guess which word comes next", which is very much a fast and intuitive task. Any reasoning comes from CoT, which is more analogous to System 2 thinking. But that System 2 thinking can't function at all without relying on the representations from System 1 thinking (which are formed in pretraining).
So you could argue System 2 thinking is some part of the LLM, but it can't be 100%.
faishalmanzar 2 hours ago [-]
[flagged]
fmbb 5 hours ago [-]
I have never heard of ”System 1” and ”System 2” thinking before.
Wikipedia implies it’s just some pop-sci (psychology even, so extra pop) book from a few years ago. A book heavily criticized for relying on shoddy science. Why are the terms used to describe anything by anyone trying to sound serious or scientific?
amelius 5 hours ago [-]
> In 2002, Kahneman received the Nobel Memorial Prize in Economic Sciences, despite being a research psychologist, for his work in prospect theory. Kahneman stated he has never taken a single economics course – that everything that he knows of the subject he and Tversky learned from their collaborators Richard Thaler and Jack Knetsch.
__alexs 4 hours ago [-]
System 1 Vs 2 aren't even Kahneman's invention. They are from Stanovich and West.
cpgxiii 5 hours ago [-]
Because a truly massive portion of managers/leaders in AI-adjacent work are heavily buzzword-driven. "System 1/system 2","agentic <buzzword>", etc; someone in the industry starts using it and then everyone else picks it up, regardless of how much (or little) they understand.
Betelbuddy 5 hours ago [-]
Its all Kahneman...whose book and work has been discredited, due to the bad statistical methods, and he himself acknowledged it.
vovavili 4 hours ago [-]
Feel free to substitute these terms for "default mode network" and "execute control network" if you're looking for rigor.
rambambram 4 hours ago [-]
Are we making up words again?
viklove 4 hours ago [-]
All words are made up
2001zhaozhao 3 hours ago [-]
answer: you make a recurrent neuralese model
(btw don't do that)
dominotw 7 hours ago [-]
> One of Kahneman’s best known contributions to AIthought is the System 1 System 2 metaphor he brought tobear on human cognition in his book Thinking Fast andSlow.
I read this book published in 1974 called 'inner game of tennis' that has the same metaphor. Its one of the book i read every once in a while even though i dont play tennis anymore.
OJFord 6 hours ago [-]
I'm struggling to get through TFaS, really don't understand how it became such a 'PopPsych' (if that's a thing) classic.
It's like a collection of the obvious, the wrong, and autobiography of someone you haven't otherwise heard of.
vovavili 6 hours ago [-]
It'a probably obvious to you because of cultural osmosis.
mjburgess 6 hours ago [-]
Of that class of book, the game is to pretend to have read it. A little like quoting a play having read a book of quotes.
Betelbuddy 5 hours ago [-]
Kahneman book and work has been discredited, due to the bad statistical methods, and he himself acknowledged it.
raddan 2 hours ago [-]
I’d like to see some evidence of this. Not to mention—experiments that show a phenomenon is more complicated than originally thought is not evidence of shoddy work. It’s just science. We all stand on the shoulders of giants.
A good example is Kahneman and Tversky’s work on the conjunction fallacy. We now know that “stereotype threat” is not the only reason somebody might make a mistake of the kind they were studying. And over the decades, better experiments introduced better controls and more nuanced analysis. That said, people still make conjunction fallacies, but the effects are not as pronounced as originally believed. Is this bad science? Absolutely not! The finding is, in fact, pretty robust; we just know that there’s more at work.
So how do you count that? Is that a “failure to replicate”? Technically speaking, yes. But it is not bad science, and more importantly, it’s not a standard that we hold even “hard” scientists to. Nobody is saying that Newton has been discredited because of general relativity.
cameldrv 6 hours ago [-]
+1 on the inner game of tennis. It had a big influence on how I think about my own mind.
QuesnayJr 4 hours ago [-]
Kahneman credits it to someone else. His book just popularized it.
Maybe I need to reread his book, but I think of System 1 as "fast, reflexive, instinctive" mental activity, and System 2 as "slow, deliberative, rational, verbal" thought.
So exactly inverse to this article, I characterize LLMs as 100% System 2, and would say that what's missing in AI is the more "embodied" System 1 intelligence.
My current thinking is that the experience of consciousness is closely tied to that embodied, emotional side of our nervous system, and is 100% lacking in LLMs. Anthropic people will say "the models have neural correlates of emotion", and that's fine, our cortex also has neural correlates for emotions and for much of what happens in the cerebellum and brain stem. That doesn't mean that that's where the primary function is implemented.
I've read Kahnemann's book and how I've seen System 1 / 2 typically used in the context LLMs often makes them seem like poor fits to what's being referenced. The concept was created to broadly group types of human reasoning but it never struck me as rigorously defined enough to be a technically precise distinction. Applying what was a mildly useful directional thumbnail referencing human thinking styles to LLMs, which can mimic human output but via completely different processes, quickly goes off the rails. As we get more detailed and precise, the System 1 / 2 distinction seems to increasingly diverge from how LLMs behave.
A quick example is how basic Flash-type chat models without 'extended thinking' or MoEs will keep suggesting different solutions instead of stopping after a few failures to step back, gather more context and diagnostic data and evaluate changing approaches or questioning the question. When I do that, the 'interrupt' is often a 'System 1-ish' intuitive feeling like "I suspect I'm barking up the wrong tree" then, once stopped, I'll switch to System 2 to analyze. But in LLMs that "interrupt" behavior typically arises from adding more System 2-ish thinking. N=1 but that's not how I usually 'get there' and is orthogonal to what the System 1 / 2 model suggests.
System 2 is slow because it is sequential. Every output depends on all the prior outputs. Like a transformer. (Even if this comes about as the result of massively parallel processes, both in the brain and in the model.)
System 1 is more of a parallel thing, operating on the output as a whole rather than in sequence. Much more like diffusion.
The terms don't seem to translate well, possibly because they were the wrong framing to begin with. Possibly also because the billions of years of progressively complex evolutionary constraint that brought about animal cognition does not seem likely to be reproduced by running backprop on human language artifacts.
Current LLMs are the opposite of slow and deliberative, though? With the current transformer architecture, it seems like its function is more or less "look at this text and make a guess which word comes next", which is very much a fast and intuitive task. Any reasoning comes from CoT, which is more analogous to System 2 thinking. But that System 2 thinking can't function at all without relying on the representations from System 1 thinking (which are formed in pretraining).
So you could argue System 2 thinking is some part of the LLM, but it can't be 100%.
Wikipedia implies it’s just some pop-sci (psychology even, so extra pop) book from a few years ago. A book heavily criticized for relying on shoddy science. Why are the terms used to describe anything by anyone trying to sound serious or scientific?
(btw don't do that)
I read this book published in 1974 called 'inner game of tennis' that has the same metaphor. Its one of the book i read every once in a while even though i dont play tennis anymore.
It's like a collection of the obvious, the wrong, and autobiography of someone you haven't otherwise heard of.
A good example is Kahneman and Tversky’s work on the conjunction fallacy. We now know that “stereotype threat” is not the only reason somebody might make a mistake of the kind they were studying. And over the decades, better experiments introduced better controls and more nuanced analysis. That said, people still make conjunction fallacies, but the effects are not as pronounced as originally believed. Is this bad science? Absolutely not! The finding is, in fact, pretty robust; we just know that there’s more at work.
So how do you count that? Is that a “failure to replicate”? Technically speaking, yes. But it is not bad science, and more importantly, it’s not a standard that we hold even “hard” scientists to. Nobody is saying that Newton has been discredited because of general relativity.