On two separate occasions in two days, my new Muse agent has flat out told me I was wrong about something. I did not necessarily expect that since Muse operates in a tough competitive environment. However, Lionel’s argument about the political/social effects of AI may lead more people to value having their priors corrected before they engage in argument, and in fact, their agent may correct their priors unasked. If so, this will be a good thing for each person and for society.
It's interesting to observe how this assessment rests on the assumption that the future of general usage AI is as a non-politically-coded technology (in both a social and programming sense).
LLMs have a neutral arbiter Leibniz quality currently, but I can also envisage a future where LLMs are designed with more Schopenhauer-oriented qualities. LLMs in China already have elements of that in the present.
It’d be interesting to see how far you could push this. How hard would it be to train an LLM to be a flat earther? It probably won’t have enough resources to establish that but I could see a creationist organization put money to building its own LLM.
I think it will allow us to significantly improve the quality of discussion and debate. An example is the issue of women’s wages relative to men’s. A simple AI query now explains the issue as both real and easily explained primarily by differences in choices, goals, values and careers.
A few other points.
I see no reason for someone to honestly and fairly only ask their AI to argue one side of the debate. Both sides should ask their AI to lay out the pros and cons, strengths and weaknesses before the discussion escalates. Doing so puts the respectable burden on the debaters to first convince their own AI before trying to persuade anyone else.
Second, I find that when I push AI to lay out a position it believes are best, that it lays out extremely sound and fairly well balanced positions on every conceivable topic from abortion, to homelessness, to law enforcement Iran, to health care. Does everyone else not ask their AI these things, or debate them when they disagree (for their own sake)? Assuming they do, are we all getting similar answers?
Finally, I am gobsmacked that nobody has yet taken the initiative to set up substack dedicated to debates using AI. This could be debates between models Claude vs Chat, or between people and models. I honestly think this would be the most interesting site possible. If I had the time, that is what I would build.
I agree that GenAI can have positive effects, e.g. by enabling quick verification of issues. I wonder what you think about the risk of LLMs flattening diversity across arguments, and flattening complexity within arguments? Basically thinking about the findings from this paper: https://arxiv.org/abs/2604.03136 (realize they study fiction, but I suspect that the current versions still have the same tendencies more generally).
By improving individual arguments, LLMs might make the overall debate less diverse. In the paper, the LLMs stories cluster around similar patterns, while human stories showed greater diversity. Importantly, this didn't just apply to the final message, but the way the argument was structured, what kind of evidence was relied upon etc. For example, human stories relied more on particular texts and situations - one could argue that arguments shouldn't do that, but it is an important element of human dialogue (as Mercier would also argue, I think).
Overall, LLMs could raise the floor of argumentative quality - but might reduce diversity?
I think the reduction in diversity of thought is a very relevant concern. One insight from cultural evolution is that it benefits from competition between different communities that innovate and whose ideas compete. I think it is reasonable to ask whether LLMs will flatten epistemic differences across communities and thereby limit innovation. The countervailing effect is that innovation might happen within the unified community, but if we get stuck in a bad place, there will be no other community to learn from about how to do better.
I did have an experience recently where someone was speaking word salad so much it came out as purposely abtruse. I asked ChatGPT and once it put things clearer, I actually did think he had some good points.
I'd add to your introduction the ways that debate, argumentation, and exploratory reasoning differ. Also, you might note how the thinking styles (the assumptions and methods) differ, say between Platonic-Socratic dialectics and Toulmin argumentation.
"Once people know that claims can easily be checked, they have stronger incentives to make claims that will survive scrutiny." I've noticed this to be the case. I make an assertion and it's critiqued by someone. I provide references to back my assertion. It's now very easy for the person to substantiate them via an AI overview.
"Suppose also that the theory is sufficiently complex that no human can fully assess it, while other AI models judge it to be correct. Should humans accept their verdict?" I doubt there is much in nature that can be "fully assessed." If tests are found that might confirm or deny the theory, then some probability of its merit can be calculated with Bayesian abductive reasoning.
I agree with the message of your essay. I too am positive about the future benefits of AI assistants (I want a Horizon focus), what Andy Clark sees as a new layer added to our extended minds.
Artificial intelligence boils down to mathematical intelligence - what Roger penrose calls a computational model. Mathematicians trust only in things which exist independently of time, ie AI will always run up against ATEMPORALITY. SO arguments solved by AI must do without TIME. Ridiculous.
Excellent piece; this seems like an important thing to think about. There's also some work showing AI use potentially has a depolarizing effect on political opinions (opposite effect of social media) [1], so it may even help nudge us to be less biased thinkers.
> Leibniz’s ideal of a perfect language that would determine what is right and wrong is likely outside the realm of what is feasible.
Should we give up on this prematurely? I don't fully understand bayesianism / bayesian networks, but from what I gather, they seem like the exact tool Leibniz had in mind. Especially if we could use AI to take real-world arguments and build the bayesian graph for each side, sift through the evidence to establish probabilities on variables and relationships, and then decide which argument is most coherent and consistent with available evidence.
You argue: “First, AI makes it much easier to verify arguments: factual assertions and interpretations of the evidence can be checked right away. Second, AI makes it much easier to craft good arguments. Taken together, I think these two effects are likely to improve the quality of the arguments we use in our discussions.” That’s interesting. What will likely be the real world consequences of that? In other words have you actually noticed who the American people elected (TWICE!) in spite of many good journalists fact checking (i.e., debunking) the constant stream of falsehoods dumped into the political debate? Really?
One of my favorite AI queries is:
“Assess the accuracy of this statement: <direct quote of what someone wrote on social media>.”
On two separate occasions in two days, my new Muse agent has flat out told me I was wrong about something. I did not necessarily expect that since Muse operates in a tough competitive environment. However, Lionel’s argument about the political/social effects of AI may lead more people to value having their priors corrected before they engage in argument, and in fact, their agent may correct their priors unasked. If so, this will be a good thing for each person and for society.
It's interesting to observe how this assessment rests on the assumption that the future of general usage AI is as a non-politically-coded technology (in both a social and programming sense).
LLMs have a neutral arbiter Leibniz quality currently, but I can also envisage a future where LLMs are designed with more Schopenhauer-oriented qualities. LLMs in China already have elements of that in the present.
It’d be interesting to see how far you could push this. How hard would it be to train an LLM to be a flat earther? It probably won’t have enough resources to establish that but I could see a creationist organization put money to building its own LLM.
I think it will allow us to significantly improve the quality of discussion and debate. An example is the issue of women’s wages relative to men’s. A simple AI query now explains the issue as both real and easily explained primarily by differences in choices, goals, values and careers.
A few other points.
I see no reason for someone to honestly and fairly only ask their AI to argue one side of the debate. Both sides should ask their AI to lay out the pros and cons, strengths and weaknesses before the discussion escalates. Doing so puts the respectable burden on the debaters to first convince their own AI before trying to persuade anyone else.
Second, I find that when I push AI to lay out a position it believes are best, that it lays out extremely sound and fairly well balanced positions on every conceivable topic from abortion, to homelessness, to law enforcement Iran, to health care. Does everyone else not ask their AI these things, or debate them when they disagree (for their own sake)? Assuming they do, are we all getting similar answers?
Finally, I am gobsmacked that nobody has yet taken the initiative to set up substack dedicated to debates using AI. This could be debates between models Claude vs Chat, or between people and models. I honestly think this would be the most interesting site possible. If I had the time, that is what I would build.
I agree that GenAI can have positive effects, e.g. by enabling quick verification of issues. I wonder what you think about the risk of LLMs flattening diversity across arguments, and flattening complexity within arguments? Basically thinking about the findings from this paper: https://arxiv.org/abs/2604.03136 (realize they study fiction, but I suspect that the current versions still have the same tendencies more generally).
By improving individual arguments, LLMs might make the overall debate less diverse. In the paper, the LLMs stories cluster around similar patterns, while human stories showed greater diversity. Importantly, this didn't just apply to the final message, but the way the argument was structured, what kind of evidence was relied upon etc. For example, human stories relied more on particular texts and situations - one could argue that arguments shouldn't do that, but it is an important element of human dialogue (as Mercier would also argue, I think).
Overall, LLMs could raise the floor of argumentative quality - but might reduce diversity?
I think the reduction in diversity of thought is a very relevant concern. One insight from cultural evolution is that it benefits from competition between different communities that innovate and whose ideas compete. I think it is reasonable to ask whether LLMs will flatten epistemic differences across communities and thereby limit innovation. The countervailing effect is that innovation might happen within the unified community, but if we get stuck in a bad place, there will be no other community to learn from about how to do better.
I did have an experience recently where someone was speaking word salad so much it came out as purposely abtruse. I asked ChatGPT and once it put things clearer, I actually did think he had some good points.
This is already happening.
I'd add to your introduction the ways that debate, argumentation, and exploratory reasoning differ. Also, you might note how the thinking styles (the assumptions and methods) differ, say between Platonic-Socratic dialectics and Toulmin argumentation.
"Once people know that claims can easily be checked, they have stronger incentives to make claims that will survive scrutiny." I've noticed this to be the case. I make an assertion and it's critiqued by someone. I provide references to back my assertion. It's now very easy for the person to substantiate them via an AI overview.
"Suppose also that the theory is sufficiently complex that no human can fully assess it, while other AI models judge it to be correct. Should humans accept their verdict?" I doubt there is much in nature that can be "fully assessed." If tests are found that might confirm or deny the theory, then some probability of its merit can be calculated with Bayesian abductive reasoning.
I agree with the message of your essay. I too am positive about the future benefits of AI assistants (I want a Horizon focus), what Andy Clark sees as a new layer added to our extended minds.
Artificial intelligence boils down to mathematical intelligence - what Roger penrose calls a computational model. Mathematicians trust only in things which exist independently of time, ie AI will always run up against ATEMPORALITY. SO arguments solved by AI must do without TIME. Ridiculous.
Excellent piece; this seems like an important thing to think about. There's also some work showing AI use potentially has a depolarizing effect on political opinions (opposite effect of social media) [1], so it may even help nudge us to be less biased thinkers.
> Leibniz’s ideal of a perfect language that would determine what is right and wrong is likely outside the realm of what is feasible.
Should we give up on this prematurely? I don't fully understand bayesianism / bayesian networks, but from what I gather, they seem like the exact tool Leibniz had in mind. Especially if we could use AI to take real-world arguments and build the bayesian graph for each side, sift through the evidence to establish probabilities on variables and relationships, and then decide which argument is most coherent and consistent with available evidence.
Or is that still in pipe dream territory?
--
[1]: https://www.tandfonline.com/doi/full/10.1080/08838151.2025.2519731?utm_source=chatgpt.com
^ not the best research but at least suggestive
Wikipedia is typically a better source than ChatGPT for questions like that of the faill of the Pakistani PM
You argue: “First, AI makes it much easier to verify arguments: factual assertions and interpretations of the evidence can be checked right away. Second, AI makes it much easier to craft good arguments. Taken together, I think these two effects are likely to improve the quality of the arguments we use in our discussions.” That’s interesting. What will likely be the real world consequences of that? In other words have you actually noticed who the American people elected (TWICE!) in spite of many good journalists fact checking (i.e., debunking) the constant stream of falsehoods dumped into the political debate? Really?