I agree with 90%+ of the essay and think the thesis is sound, but have a couple of critiques:
1) You wrote: "AI companies compete for consumers, who choose the models they find most useful and trustworthy ... Users’ trust is therefore a key asset for AI companies. Any intentional deception or obfuscation uncovered by users, journalists or competitors could greatly damage a company’s reputation and drive many users away."
-I think the commercial success of Newsmax, OANN and charlatan influencers reveal that for *many* consumers trust is not earned by telling the truth. Trust is earned by confirming your priors, making you feel good about your side and providing a plausible excuse for why the other side is wrong. The huge AI companies will have a clear economic incentive towards the truth (just like mass market ABC, CBS, NBC) but the more niche AI models will have a market for earning trust through confirming priors (just like smaller market cable TV Fox News). Among typical consumers, trust is typically a socially derived quality, not an epistemiologically derived quality.
2) You wrote: "AI companies would not, for instance, draw obvious benefits from pushing false claims on major social and political issues, such as whether a vaccine is effective ... It is therefore reasonable to expect that competition between AI companies gives firms strong incentives to provide citizens with models that give the best answers they can, rather than answers designed to protect the authority of others."
-Smaller bespoke AI agents flooding social media pushing products have clear obvious benefits for pushing false claims. "Buy this product, it is better" even if it is not can be widely profitable for a company running an ad campaign using AI bots. In this case it's not the user going to a LLM looking for a wingman, it is the company's ad team looking for a wingman to deceive potential customers. The "user" is not always an individual human, it can be a company's marketing team.
3) A great deal of people simply do not have the baseline level of knowledge about a topic or the baseline ability to evaluate things from a "scientific" lens. They will not know what questions to ask, what topics to ask about and will be unaware of the gaps in their own knowledge that the LLM could fix. The average American adult reads at a middle school reading level - and that's the AVERAGE. There are tens of millions who are far below that.
--------
TLDR - Good essay, mostly agree. But I think more pessimism is warranted about the abilities of the median human to use tools effectively, the median human's desire to find the truth and the dangers of bespoke smaller AI models flooding the zone with propaganda to make money.
For comparison, I tested Google's inbuilt AI on another gender-political issue where even informed people often misinterpret the data - the higher rate of male suicide. The AI response correctly observed that suicide *attempts* were actually more common among women, and that the reason that men nonetheless had higher suicide rates is because they chose more lethal methods, owing to higher male propensity for violence including violence against oneself.
I think it's important to distinguish the question of whether AI will reduce support for specific left wing perspectives that are often justified on the basis of 'its science' and whether it will reduce the influence of experts.
In general, I'd argue that expert influence is the ability of someone to counterfactualy move the debate based on what they say. The truth is that things like the wage gap *aren't* instances of academics forcing a view on the rest of society but of a left wing view being forced on academics and intellectuals. The academics may mostly not be struggling, they do tend to all be on the left, but this narrative isn't being dictated by the intellectual elite so much as the intellectual elite are being told by the rest of the left what is acceptable to say.
I hope AI means a reduction in that sort of thing but I'm not so confident. When I talk to people about it they either already tend to treat it as suspect (eg people who lean right) or happy to use motivated reasoning to reach that conclusion.
But if it does I think it will actually mean increased influence of intellectuals and experts because right now people are compensating for the fact that ideology shapes what the experts are allowed to say. The more you remove that, the more reason you have to actually change your view in response to an argument or experiment.
I agree with 90%+ of the essay and think the thesis is sound, but have a couple of critiques:
1) You wrote: "AI companies compete for consumers, who choose the models they find most useful and trustworthy ... Users’ trust is therefore a key asset for AI companies. Any intentional deception or obfuscation uncovered by users, journalists or competitors could greatly damage a company’s reputation and drive many users away."
-I think the commercial success of Newsmax, OANN and charlatan influencers reveal that for *many* consumers trust is not earned by telling the truth. Trust is earned by confirming your priors, making you feel good about your side and providing a plausible excuse for why the other side is wrong. The huge AI companies will have a clear economic incentive towards the truth (just like mass market ABC, CBS, NBC) but the more niche AI models will have a market for earning trust through confirming priors (just like smaller market cable TV Fox News). Among typical consumers, trust is typically a socially derived quality, not an epistemiologically derived quality.
2) You wrote: "AI companies would not, for instance, draw obvious benefits from pushing false claims on major social and political issues, such as whether a vaccine is effective ... It is therefore reasonable to expect that competition between AI companies gives firms strong incentives to provide citizens with models that give the best answers they can, rather than answers designed to protect the authority of others."
-Smaller bespoke AI agents flooding social media pushing products have clear obvious benefits for pushing false claims. "Buy this product, it is better" even if it is not can be widely profitable for a company running an ad campaign using AI bots. In this case it's not the user going to a LLM looking for a wingman, it is the company's ad team looking for a wingman to deceive potential customers. The "user" is not always an individual human, it can be a company's marketing team.
3) A great deal of people simply do not have the baseline level of knowledge about a topic or the baseline ability to evaluate things from a "scientific" lens. They will not know what questions to ask, what topics to ask about and will be unaware of the gaps in their own knowledge that the LLM could fix. The average American adult reads at a middle school reading level - and that's the AVERAGE. There are tens of millions who are far below that.
--------
TLDR - Good essay, mostly agree. But I think more pessimism is warranted about the abilities of the median human to use tools effectively, the median human's desire to find the truth and the dangers of bespoke smaller AI models flooding the zone with propaganda to make money.
For comparison, I tested Google's inbuilt AI on another gender-political issue where even informed people often misinterpret the data - the higher rate of male suicide. The AI response correctly observed that suicide *attempts* were actually more common among women, and that the reason that men nonetheless had higher suicide rates is because they chose more lethal methods, owing to higher male propensity for violence including violence against oneself.
You assume that Artificial Intelligence is epistemologically clean? LOL!
I think it's important to distinguish the question of whether AI will reduce support for specific left wing perspectives that are often justified on the basis of 'its science' and whether it will reduce the influence of experts.
In general, I'd argue that expert influence is the ability of someone to counterfactualy move the debate based on what they say. The truth is that things like the wage gap *aren't* instances of academics forcing a view on the rest of society but of a left wing view being forced on academics and intellectuals. The academics may mostly not be struggling, they do tend to all be on the left, but this narrative isn't being dictated by the intellectual elite so much as the intellectual elite are being told by the rest of the left what is acceptable to say.
I hope AI means a reduction in that sort of thing but I'm not so confident. When I talk to people about it they either already tend to treat it as suspect (eg people who lean right) or happy to use motivated reasoning to reach that conclusion.
But if it does I think it will actually mean increased influence of intellectuals and experts because right now people are compensating for the fact that ideology shapes what the experts are allowed to say. The more you remove that, the more reason you have to actually change your view in response to an argument or experiment.