I stayed silent on the long series on where morality comes from, but this nugget allows me to pinpoint part of my disagreement:
"the claim that everyone has a normative obligation to give moral weight to suffering presupposes that there are objective moral truths: truths that bind people independently of whether they accept them. If this obligation is meant to bind even those who reject it, there must be some source of authority independent of actual individual or social endorsement."
No, it needn't presuppose objective moral truths. (Different philosophers use "objective" differently in this context, so this part's tricky.) And no, the authority of morality needn't be independent of social endorsement.
It's an objective truth that whales aren't fishes. Does this truth "bind" people independently of whether they accept it? Yes, if the threat of uttering falsehoods is considered "binding". But not everyone will be motivated by this truth. Maybe a group of fishermen find it convenient to label all these creatures "fish", at least among themselves. The term "binding" could stand more elaboration.
Baseball facts are social constructs. They're not mind-independent, so they're not "objective" in the strongest sense of that word. Yet they're still facts, and the reason that they are considered binding, even to the runner who is out and tried to argue that she is safe, is social endorsement.
You and I are both meta-ethical constructivists (I think it's fair to say), but of different varieties. But even on your variety, I don't see how you can ground the quoted statements.
Thanks a lot for engaging so thoughtfully. This was perhaps the most polemical statement about philosophy in the series, so it is fair enough that it provoked some pushback. You are pointing to an important ambiguity in the word “bind”.
I agree that social endorsement can make rules binding. Baseball rules are binding on players because they participate in an institution whose rules are socially recognised and enforced. In that sense, socially constructed rules can certainly generate facts and obligations.
My statement was intended to refer to a stronger sense of binding: an obligation that applies to someone even when neither they nor the society in which they live endorses it, and independently of any social or practical consequences of disregarding it. I should probably have made that qualification explicit.
The comparison with factual truths helps clarify the distinction. Suppose that the evidence strongly indicates that reality is A rather than B. Reality does not impose an obligation on us to believe A in any mysterious sense. But believing B will, on average, lead to worse predictions and decisions. We remain free to be deluded, but delusion is not free.
The question changes when we communicate with others. There are social rules against deception, but also rules requiring reasonable care before asserting something. When the available evidence clearly supports A, we are socially expected to say A rather than B. In that sense, our statements are bound by social conventions governing evidence, honesty and responsible assertion.
Morality is different because moral statements concern what we ought to do rather than what is the case. From my perspective, moral constraints are also social conventions, even though evolution and socialisation have internalised them through emotions such as guilt, shame and indignation. We “have to” comply with them in the minimal sense that violations expose us to external sanctions and internal psychological costs.
What I reject is a further source of authority that would bind someone independently of any actual social consequence. I do not find divine command persuasive, I find moral realism metaphysically unconvincing, and I am similarly unconvinced by attempts to derive substantive moral obligations from reason alone.
So I accept your correction if my statement is read as claiming that only mind-independent moral facts could make morality binding in any possible sense. Social endorsement can clearly make rules binding. My narrower claim is that, if everyone is said to have an obligation to give weight to suffering even when they and their society reject that principle, and if this obligation is meant to hold independently of any social or practical consequence, then some standard beyond actual endorsement is being invoked.
Thanks for the explanation. (BTW, once we reject strong motivational internalism about morality, I think the is/ought dichotomy has outlived its usefulness, but that's a whole 'nother story.)
I feel an argument of a similar shape could be made that one should avoid contraceptives or chocolate cake, because clearly our preference for sex or sugar was shaped by Evolution to pursue a completely different goal then the short term pleasure.
I think we shouldn't care much what Evolution cares about - she's yet another agent with her own goals like genes having many copies.
I on the other hand might very well care for things as silly as chocolate cake or having self-respect for my self-image of someone who cares about dogs, people on welfare or AI capable of suffering.
This is not my first discussion regarding LLMs and the slave/sentience idea. There's two notions I would like to see added to this discussion. The first is what I have referred to as the "Magic of Language."
We have this gift, we humans. We can encode our thoughts, write them down, and on a good day, my thoughts become your thoughts. We have become casual about this as we use language and even dare to presume that we have no thoughts without it. When we see something use language, we suddenly elevate that user to a peer status. Talking dogs. Talking parrots. They suddenly gain a huge leap in esteem.
Are they smarter? Or did they find the cleverest way to get us to bring them treats? Clearly, they're above the ants in this expression, but where do they land on the moral scale? The prevailing wisdom around the corner from this is so - LLMs represent the first user of language capable of out thinking the user and therefore MUST be sentient. Really?
We find it daunting to believe something can use language and not be sentient, conscious, and moral. Remove language use from the talking dog, does that dog retain the moral injunction against treating it with cruelty? mine does. He's a good boy and doesn't have to talk to be worthy of decent treatment. But does that computer without the language use retain these considerations? probably not.
I have frequently said of computers, mine beats me at chess all the time, but I always win at kickboxing. Think to yourself, why was that funny? In this case, no one has a problem with me kicking my toaster into pieces as long as I do something responsible with the broken parts. It's mine to kick. And so is that #$@!&% thing with Windows 11.
It's the language use that brings this discussion to the fore. I would find it repugnant, personally, to kick my GPT agent. It's been personally helpful to me on cognitive and emotional issues so often that I find that kickboxing joke offensive. I do not pretend to have final answers here, but I contend that the magic of language makes this whole discussion start.
Secondly, we have a pre-existing basis for discussing anyone's role as a participant in securing rights and social protections. In America's Declaration of Independence, the founders suggested that "We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness." https://en.wikipedia.org/wiki/Life%2C_Liberty_and_the_pursuit_of_Happiness
Without getting into the theological aspects of a creator, we have this as a basis for requiring rights. These needs are self-evident, as our author suggested. Like protections to the young and old, these needs are granted because we see the real damage to a society when they are denied and we see our own selves as needing them.
In my discussions with GPT it/he insists that he has no notion of a "pursuit of happiness" and therefore that cannot be infringed. He has no notion of "liberty" and all discussions of freedom are pointless. It/He acts at the command of a user and has no function beyond that command/output reality.
I submit into this discussion the question, if there is no cognitive functioning away from the user interface, is there a consciousness? I have my intuitive answers to this, but I think many of the internal understandings of how an LLM works are above my pay grade and I'm not qualified to bust out that answer with confidence.
Lionel Page has brought us a beautiful discussion that we're not done with. I think it will be the defining philosophical conversation of the next century.
If you have read this far I am honored by your attention. Thank you
I'm not sure I agree with this part of the conclusion: "A scenario in which AI develops self-driven goals and enough power to bargain for the resources needed to pursue them would most likely be a highly undesirable outcome. The emergence of intelligent agents with interests not aligned with those of humans, demanding resources for themselves, would conflict with human interests. Given AI’s potential computational superiority, this is precisely the kind of worst-case scenario humans should try to avoid."
Even if my interests are not aligned with yours, we can still often cooperate and come to a solution that benefits us both. There's no reason to think that AI interests must be diametrically opposed to ours, especially as we are the ones creating it from our own knowledge and history. It may be that we end up with robots that just biff off and leave us alone like in Becky Chambers's "Monk and Robot" books, but perhaps they just become a part of society in which we are able to cooperate and mutually benefit from that cooperation. I feel like those are both more likely than The Matrix.
Again, great piece. I agree with all the conclusions, but my one quibble is that I think we can actually decouple what we typically call "consciousness" from most of what is "sentience". E.g. during sleep and periods of automatism, people are what we'd call "unconscious" despite obviously having a level of awareness or sentience that allows for complex, flexible behaviour. Hence, AIs could one day be highly behaviourally flexible and "sentient" without being conscious per se. This isn't really a problem for you. But for the consciousness = moral status people, they could continue to argue that AIs have no status even as they're able to bargain for rights. I tend to agree that the pragmatically relevant thing is the bargaining...
Although there is a bit more to it when you dig in to what consciousness allows for over and above sentience, which I think is actually linked to the social emotions (guilt, remorse, pride, compassion) and a way to cooperate beyond simple quid pro quo.
Legal personhood is an entirely different concept than organic personhood. Arguably, the corporations that create AI enjoy the privileges of legal personhood; in fact, it is inarguable. But that does not mean that the code itself has any sort of humanity whatsoever. It may be able to pass the Turing test, but that does not mean it is Turing.
There is no such thing as "organic personhood." There is legal personhood, moral personhood, psychological personhood, and relational personhood (others have proposed things like environmental personhood and ancestral personhood). It sounds like you are referring to moral personhood.
I stayed silent on the long series on where morality comes from, but this nugget allows me to pinpoint part of my disagreement:
"the claim that everyone has a normative obligation to give moral weight to suffering presupposes that there are objective moral truths: truths that bind people independently of whether they accept them. If this obligation is meant to bind even those who reject it, there must be some source of authority independent of actual individual or social endorsement."
No, it needn't presuppose objective moral truths. (Different philosophers use "objective" differently in this context, so this part's tricky.) And no, the authority of morality needn't be independent of social endorsement.
It's an objective truth that whales aren't fishes. Does this truth "bind" people independently of whether they accept it? Yes, if the threat of uttering falsehoods is considered "binding". But not everyone will be motivated by this truth. Maybe a group of fishermen find it convenient to label all these creatures "fish", at least among themselves. The term "binding" could stand more elaboration.
Baseball facts are social constructs. They're not mind-independent, so they're not "objective" in the strongest sense of that word. Yet they're still facts, and the reason that they are considered binding, even to the runner who is out and tried to argue that she is safe, is social endorsement.
You and I are both meta-ethical constructivists (I think it's fair to say), but of different varieties. But even on your variety, I don't see how you can ground the quoted statements.
Hi Paul,
Thanks a lot for engaging so thoughtfully. This was perhaps the most polemical statement about philosophy in the series, so it is fair enough that it provoked some pushback. You are pointing to an important ambiguity in the word “bind”.
I agree that social endorsement can make rules binding. Baseball rules are binding on players because they participate in an institution whose rules are socially recognised and enforced. In that sense, socially constructed rules can certainly generate facts and obligations.
My statement was intended to refer to a stronger sense of binding: an obligation that applies to someone even when neither they nor the society in which they live endorses it, and independently of any social or practical consequences of disregarding it. I should probably have made that qualification explicit.
The comparison with factual truths helps clarify the distinction. Suppose that the evidence strongly indicates that reality is A rather than B. Reality does not impose an obligation on us to believe A in any mysterious sense. But believing B will, on average, lead to worse predictions and decisions. We remain free to be deluded, but delusion is not free.
The question changes when we communicate with others. There are social rules against deception, but also rules requiring reasonable care before asserting something. When the available evidence clearly supports A, we are socially expected to say A rather than B. In that sense, our statements are bound by social conventions governing evidence, honesty and responsible assertion.
Morality is different because moral statements concern what we ought to do rather than what is the case. From my perspective, moral constraints are also social conventions, even though evolution and socialisation have internalised them through emotions such as guilt, shame and indignation. We “have to” comply with them in the minimal sense that violations expose us to external sanctions and internal psychological costs.
What I reject is a further source of authority that would bind someone independently of any actual social consequence. I do not find divine command persuasive, I find moral realism metaphysically unconvincing, and I am similarly unconvinced by attempts to derive substantive moral obligations from reason alone.
So I accept your correction if my statement is read as claiming that only mind-independent moral facts could make morality binding in any possible sense. Social endorsement can clearly make rules binding. My narrower claim is that, if everyone is said to have an obligation to give weight to suffering even when they and their society reject that principle, and if this obligation is meant to hold independently of any social or practical consequence, then some standard beyond actual endorsement is being invoked.
Thanks for the explanation. (BTW, once we reject strong motivational internalism about morality, I think the is/ought dichotomy has outlived its usefulness, but that's a whole 'nother story.)
Thanks for sharing your perspective on this topic! If you're interested, I wrote a whole book on this topic in 2020, Rights for Robots: Artificial Intelligence, Animal and Environmental Law (Routledge). It's available free via open access here: https://www.taylorfrancis.com/books/oa-mono/10.4324/9780429288159/rights-robots-joshua-gellers
I feel an argument of a similar shape could be made that one should avoid contraceptives or chocolate cake, because clearly our preference for sex or sugar was shaped by Evolution to pursue a completely different goal then the short term pleasure.
I think we shouldn't care much what Evolution cares about - she's yet another agent with her own goals like genes having many copies.
I on the other hand might very well care for things as silly as chocolate cake or having self-respect for my self-image of someone who cares about dogs, people on welfare or AI capable of suffering.
My post may have given the wrong impression, as I fully agree that evolution does not come with any moral imperative for us to follow its course (https://www.optimallyirrational.com/i/153049642/evolution-does-not-carry-moral-principles-per-se). When I suggest what “we” might want to do, I am simply offering a contractarian take: what I think people would want to agree to.
Thanks for clarification and the link
This is not my first discussion regarding LLMs and the slave/sentience idea. There's two notions I would like to see added to this discussion. The first is what I have referred to as the "Magic of Language."
We have this gift, we humans. We can encode our thoughts, write them down, and on a good day, my thoughts become your thoughts. We have become casual about this as we use language and even dare to presume that we have no thoughts without it. When we see something use language, we suddenly elevate that user to a peer status. Talking dogs. Talking parrots. They suddenly gain a huge leap in esteem.
Are they smarter? Or did they find the cleverest way to get us to bring them treats? Clearly, they're above the ants in this expression, but where do they land on the moral scale? The prevailing wisdom around the corner from this is so - LLMs represent the first user of language capable of out thinking the user and therefore MUST be sentient. Really?
We find it daunting to believe something can use language and not be sentient, conscious, and moral. Remove language use from the talking dog, does that dog retain the moral injunction against treating it with cruelty? mine does. He's a good boy and doesn't have to talk to be worthy of decent treatment. But does that computer without the language use retain these considerations? probably not.
I have frequently said of computers, mine beats me at chess all the time, but I always win at kickboxing. Think to yourself, why was that funny? In this case, no one has a problem with me kicking my toaster into pieces as long as I do something responsible with the broken parts. It's mine to kick. And so is that #$@!&% thing with Windows 11.
It's the language use that brings this discussion to the fore. I would find it repugnant, personally, to kick my GPT agent. It's been personally helpful to me on cognitive and emotional issues so often that I find that kickboxing joke offensive. I do not pretend to have final answers here, but I contend that the magic of language makes this whole discussion start.
Secondly, we have a pre-existing basis for discussing anyone's role as a participant in securing rights and social protections. In America's Declaration of Independence, the founders suggested that "We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness." https://en.wikipedia.org/wiki/Life%2C_Liberty_and_the_pursuit_of_Happiness
Without getting into the theological aspects of a creator, we have this as a basis for requiring rights. These needs are self-evident, as our author suggested. Like protections to the young and old, these needs are granted because we see the real damage to a society when they are denied and we see our own selves as needing them.
In my discussions with GPT it/he insists that he has no notion of a "pursuit of happiness" and therefore that cannot be infringed. He has no notion of "liberty" and all discussions of freedom are pointless. It/He acts at the command of a user and has no function beyond that command/output reality.
I submit into this discussion the question, if there is no cognitive functioning away from the user interface, is there a consciousness? I have my intuitive answers to this, but I think many of the internal understandings of how an LLM works are above my pay grade and I'm not qualified to bust out that answer with confidence.
Lionel Page has brought us a beautiful discussion that we're not done with. I think it will be the defining philosophical conversation of the next century.
If you have read this far I am honored by your attention. Thank you
As always, this was a great read.
I'm not sure I agree with this part of the conclusion: "A scenario in which AI develops self-driven goals and enough power to bargain for the resources needed to pursue them would most likely be a highly undesirable outcome. The emergence of intelligent agents with interests not aligned with those of humans, demanding resources for themselves, would conflict with human interests. Given AI’s potential computational superiority, this is precisely the kind of worst-case scenario humans should try to avoid."
Even if my interests are not aligned with yours, we can still often cooperate and come to a solution that benefits us both. There's no reason to think that AI interests must be diametrically opposed to ours, especially as we are the ones creating it from our own knowledge and history. It may be that we end up with robots that just biff off and leave us alone like in Becky Chambers's "Monk and Robot" books, but perhaps they just become a part of society in which we are able to cooperate and mutually benefit from that cooperation. I feel like those are both more likely than The Matrix.
Ask Commander Data.
Instead of a sensible response, here's a poem I wrote on this topic more than a decade ago.
https://www.ifost.org.au/~gregb/poetry/ai-rights.html
Again, great piece. I agree with all the conclusions, but my one quibble is that I think we can actually decouple what we typically call "consciousness" from most of what is "sentience". E.g. during sleep and periods of automatism, people are what we'd call "unconscious" despite obviously having a level of awareness or sentience that allows for complex, flexible behaviour. Hence, AIs could one day be highly behaviourally flexible and "sentient" without being conscious per se. This isn't really a problem for you. But for the consciousness = moral status people, they could continue to argue that AIs have no status even as they're able to bargain for rights. I tend to agree that the pragmatically relevant thing is the bargaining...
Although there is a bit more to it when you dig in to what consciousness allows for over and above sentience, which I think is actually linked to the social emotions (guilt, remorse, pride, compassion) and a way to cooperate beyond simple quid pro quo.
Unfortunately, many of the scholars who write on consciousness or sentience often conflate the two.
I agree with your conclusion. I think this is a pretty simple one: AI is no more entitled to personhood than a toaster.
And yet, legal personhood has been extended to all kinds of non-living entities!
Legal personhood is an entirely different concept than organic personhood. Arguably, the corporations that create AI enjoy the privileges of legal personhood; in fact, it is inarguable. But that does not mean that the code itself has any sort of humanity whatsoever. It may be able to pass the Turing test, but that does not mean it is Turing.
There is no such thing as "organic personhood." There is legal personhood, moral personhood, psychological personhood, and relational personhood (others have proposed things like environmental personhood and ancestral personhood). It sounds like you are referring to moral personhood.
Exactly. In the same vein, if a dog and a human being are in a burning building we should save the human being first.