I am not a specialist on the detail of these AI techniques, but the human process is self directed and would be guided by an attempt to optimise the reduction of uncertainty by choosing the best cognitive approaches (in expectation) at every step. It can be fully implemented by a computer in theory, but it it not how current computers work, nor how LLMs work.
I don't know the source of the data used in the tweet for human learning language and whether this specific point is correct. It is however possible that the points made both by Cummins and LeCun are right. Cummins only talks about textual data while LeCun refers to the whole data processed by a human, in particular visual data.
When I think about AI I think about Moravec's paradox, which is the idea that things we perceive as hard - like playing chess - are easy for AI. And vice-versa. They can't navigate 3d space effectively. Those robot soccer teams are dreadful!
This is related to some fundamental misperceptions about what humans are like. We may be 10 or 100 times as intelligent as the next-most-intelligent animal. But that is not exactly off the chart. We distinguish ourselves as intelligent, just like cheetahs distinguish themselves as fast. But a machine can outstrip a Cheetah and we should expect a robot to out-think a person (when it comes to abstract challenges), because a person is not an especially smart thing.
What humans do have that robots don't is a suite of other, more animal skills. Moving through space and commmunicating in groups. Our animal skills are extremely refined but we don't think much of them because we share many of these skills with other animals, so they've never seemed special. As they become our point of difference with the computers, they may come to seem more essential to who we are.
> What LLMs are missing is the ability to reason like humans, outside of the production of speech. They are unable to use the type of non-linguistic inferences made by Paul and Mary to “read the situation”.
If they miss the ability to read the situation, then it shouldn't be an issue to pick examples of interaction where LLMs actually fail. Because they certainly don't fail on these, prompted with "Explain in huge amount of detail how do you understand this interaction."
Excerpts below (not full output, because saying I want "huge" amount of detail was overkill):
[1]
> Contextual Understanding:
> Paul's Intent: Paul is likely suggesting an outing, which is a social activity implying that he wants to spend time with Mary or enjoy a shared experience.
>Mary’s Response: By stating she is tired, Mary is implicitly declining Paul’s suggestion without directly saying "no." Her reason for declining is given (her tiredness), which is socially polite and provides context to her refusal.
> Social Implications:
> Politeness Strategy: Mary uses an indirect refusal. Instead of a blunt "no," she provides a reason for her inability to accept the invitation, which helps to soften the refusal and maintain social harmony.
> Face-saving: By explaining her tiredness, Mary protects both her own face (by not appearing rude) and Paul’s face (by not outright rejecting him).
> Conclusion: This interaction between Paul and Mary, while brief, is rich with social, emotional, and contextual nuances. Paul’s invitation and Mary’s response demonstrate a polite, socially considerate exchange where both parties communicate effectively within the bounds of social norms and expectations. Understanding this interaction in detail helps highlight the subtle layers of human communication and the importance of context in interpreting meaning.
[2]
> Semantics: Paul is asking Mary if she is aware of a specific incident involving John.
> Semantics: Mary is expressing her general concern about glass windows and how dangerous they can be.
> Contextual Understanding:
> Paul’s Intent: Paul is sharing a piece of news, likely because it is surprising or amusing.
> Mary’s Response: Mary does not directly acknowledge the specific incident with John but instead generalizes the concern to all glass windows, indicating a broader anxiety.
> Social Implications:
> Politeness and Empathy: Mary’s response shows empathy and concern about the potential danger, rather than making light of John’s mishap. This shifts the conversation from a specific incident to a general safety issue.
> Conversational Dynamic: By generalizing the concern, Mary might be attempting to steer the conversation towards discussing safety or shared experiences regarding similar incidents.
To make sure it understood the "incident", I queried "what did Mary mean by treacherous?". Fragment of the output:
> 1. Deceptively Dangerous:
> Appearance: Glass windows are often very clean and transparent, making them difficult to see. This can lead to people accidentally walking into them, as they don't realize there is a barrier.
> Contextual Invisibility: In certain lighting conditions or environments, glass can blend in with the surroundings, creating an illusion of an open space.
> Injury Risk: Walking into a glass window can result in physical injuries such as bumps, bruises, cuts, or even more serious harm if the impact is severe.
> Surprise Element: The sudden realization that one has walked into a glass window can be shocking and disorienting, adding to the perceived danger.
> Worry and Concern: Mary’s use of "treacherous" implies that she is consistently aware and anxious about the potential for such accidents. It suggests a personal vigilance against what she sees as an ongoing risk.
> Past Experiences: She might have had personal experiences or know of incidents where glass windows caused accidents, reinforcing her perception of them as hazardous.
[3]
> Semantics: Paul is conveying an event where Sandy had to verify her identity at a bar.
> Semantics: Mary is expressing a sentiment that being asked for ID is fortunate for Sandy.
> Contextual Understanding:
> Paul’s Intent: Paul is sharing an observation or an event involving Sandy, which might be seen as noteworthy or surprising.
> Mary’s Response: Mary’s exclamation suggests that being asked for ID is seen as a compliment or a positive occurrence, implying youthfulness or desirability.
> Social Implications:
> Politeness and Compliment: Mary’s response can be interpreted as a compliment to Sandy, suggesting that she looks young enough to be carded.
> Conversational Dynamic: Mary’s response indicates approval or envy, framing the incident in a positive light.
> Mary’s State: Mary’s exclamation "Lucky her!" implies a positive emotional response. She might be reflecting on societal values that associate being asked for ID with looking youthful.
Then paste it in an LLM and ask "Please explain to me this picture posted in a Tweet" / "The tweet's text is "The peer review process." Please explain to me that tweet."
Fun article and great illustrations. How does chain of thought prompting and its iterative output differ from step-wise clear thinking?
I am not a specialist on the detail of these AI techniques, but the human process is self directed and would be guided by an attempt to optimise the reduction of uncertainty by choosing the best cognitive approaches (in expectation) at every step. It can be fully implemented by a computer in theory, but it it not how current computers work, nor how LLMs work.
this seems factually wrong https://x.com/mark_cummins/status/1788949893903511705 I would rather agree with Yann LeCun "In 4 years, a child has seen 50 times more data than the biggest LLMs." https://www.linkedin.com/posts/yann-lecun_ive-made-that-point-before-llm-1e13-activity-7156484065603280896-QH63/
I don't know the source of the data used in the tweet for human learning language and whether this specific point is correct. It is however possible that the points made both by Cummins and LeCun are right. Cummins only talks about textual data while LeCun refers to the whole data processed by a human, in particular visual data.
Interesting post, thank you.
When I think about AI I think about Moravec's paradox, which is the idea that things we perceive as hard - like playing chess - are easy for AI. And vice-versa. They can't navigate 3d space effectively. Those robot soccer teams are dreadful!
This is related to some fundamental misperceptions about what humans are like. We may be 10 or 100 times as intelligent as the next-most-intelligent animal. But that is not exactly off the chart. We distinguish ourselves as intelligent, just like cheetahs distinguish themselves as fast. But a machine can outstrip a Cheetah and we should expect a robot to out-think a person (when it comes to abstract challenges), because a person is not an especially smart thing.
What humans do have that robots don't is a suite of other, more animal skills. Moving through space and commmunicating in groups. Our animal skills are extremely refined but we don't think much of them because we share many of these skills with other animals, so they've never seemed special. As they become our point of difference with the computers, they may come to seem more essential to who we are.
> What LLMs are missing is the ability to reason like humans, outside of the production of speech. They are unable to use the type of non-linguistic inferences made by Paul and Mary to “read the situation”.
If they miss the ability to read the situation, then it shouldn't be an issue to pick examples of interaction where LLMs actually fail. Because they certainly don't fail on these, prompted with "Explain in huge amount of detail how do you understand this interaction."
Excerpts below (not full output, because saying I want "huge" amount of detail was overkill):
[1]
> Contextual Understanding:
> Paul's Intent: Paul is likely suggesting an outing, which is a social activity implying that he wants to spend time with Mary or enjoy a shared experience.
>Mary’s Response: By stating she is tired, Mary is implicitly declining Paul’s suggestion without directly saying "no." Her reason for declining is given (her tiredness), which is socially polite and provides context to her refusal.
> Social Implications:
> Politeness Strategy: Mary uses an indirect refusal. Instead of a blunt "no," she provides a reason for her inability to accept the invitation, which helps to soften the refusal and maintain social harmony.
> Face-saving: By explaining her tiredness, Mary protects both her own face (by not appearing rude) and Paul’s face (by not outright rejecting him).
> Conclusion: This interaction between Paul and Mary, while brief, is rich with social, emotional, and contextual nuances. Paul’s invitation and Mary’s response demonstrate a polite, socially considerate exchange where both parties communicate effectively within the bounds of social norms and expectations. Understanding this interaction in detail helps highlight the subtle layers of human communication and the importance of context in interpreting meaning.
[2]
> Semantics: Paul is asking Mary if she is aware of a specific incident involving John.
> Semantics: Mary is expressing her general concern about glass windows and how dangerous they can be.
> Contextual Understanding:
> Paul’s Intent: Paul is sharing a piece of news, likely because it is surprising or amusing.
> Mary’s Response: Mary does not directly acknowledge the specific incident with John but instead generalizes the concern to all glass windows, indicating a broader anxiety.
> Social Implications:
> Politeness and Empathy: Mary’s response shows empathy and concern about the potential danger, rather than making light of John’s mishap. This shifts the conversation from a specific incident to a general safety issue.
> Conversational Dynamic: By generalizing the concern, Mary might be attempting to steer the conversation towards discussing safety or shared experiences regarding similar incidents.
To make sure it understood the "incident", I queried "what did Mary mean by treacherous?". Fragment of the output:
> 1. Deceptively Dangerous:
> Appearance: Glass windows are often very clean and transparent, making them difficult to see. This can lead to people accidentally walking into them, as they don't realize there is a barrier.
> Contextual Invisibility: In certain lighting conditions or environments, glass can blend in with the surroundings, creating an illusion of an open space.
> Injury Risk: Walking into a glass window can result in physical injuries such as bumps, bruises, cuts, or even more serious harm if the impact is severe.
> Surprise Element: The sudden realization that one has walked into a glass window can be shocking and disorienting, adding to the perceived danger.
> Worry and Concern: Mary’s use of "treacherous" implies that she is consistently aware and anxious about the potential for such accidents. It suggests a personal vigilance against what she sees as an ongoing risk.
> Past Experiences: She might have had personal experiences or know of incidents where glass windows caused accidents, reinforcing her perception of them as hazardous.
[3]
> Semantics: Paul is conveying an event where Sandy had to verify her identity at a bar.
> Semantics: Mary is expressing a sentiment that being asked for ID is fortunate for Sandy.
> Contextual Understanding:
> Paul’s Intent: Paul is sharing an observation or an event involving Sandy, which might be seen as noteworthy or surprising.
> Mary’s Response: Mary’s exclamation suggests that being asked for ID is seen as a compliment or a positive occurrence, implying youthfulness or desirability.
> Social Implications:
> Politeness and Compliment: Mary’s response can be interpreted as a compliment to Sandy, suggesting that she looks young enough to be carded.
> Conversational Dynamic: Mary’s response indicates approval or envy, framing the incident in a positive light.
> Mary’s State: Mary’s exclamation "Lucky her!" implies a positive emotional response. She might be reflecting on societal values that associate being asked for ID with looking youthful.
Interesting. The footnote 11 describes the answers I got when requesting a simple description of the situation.
I almost posted another case with a sarcastic listing of reviewer names. See:
https://x.com/Toffeemen68/status/1789657995414249844
Then paste it in an LLM and ask "Please explain to me this picture posted in a Tweet" / "The tweet's text is "The peer review process." Please explain to me that tweet."
ChatGPT 4o misses it.
It is not suprising. See also Lecun's point about LLMs: https://x.com/page_eco/status/1795079001095991611