4 Comments
User's avatar
Scenarica's avatar

Your two scenarios turn on a governance variable more precise than ‘AI capability’: who controls the agent, its data and the right to inspect its reasoning. The democratic case strengthens when citizens can audit power without creating a reciprocal state right to audit citizens. A useful threshold is whether oversight tools remain contestable, decentralised and legally firewalled from mass surveillance.

Nicolas Payen's avatar

Lionel, the Serenata de Amor example is a great reminder that AI-driven public scrutiny works best when the results are open for anyone to check, not just handed down by a platform.

Mikael P's avatar

One thing I am both optimistic and worry about is the possibility to customise news to people based on AI. We are already seeing it with social media feeds providing relevant content, though often that relevant content is very emotionally charged and can lead people down rabbitholes. If traditional media etc. start using AI to customise what people see, I worry that this could lead to more polarisation (or on the other hand, more people actually reading the news -> a more informed electorate)

Eric Borg's avatar

Yes, AI should be a friend to humanity in some ways, though a foe in others.

On the apparently soft authoritarianism of China, I’m of the opinion that its Social Credit System should already be changing the psychology of its people in a basic way. Coming AI assessments should merely amplify this. Citizens already know that government will automatically punish or reward them for certain choices that they make — their score will simply change on the basis of what they buy, where they go, the people they associate with, and so on. Past authoritarian regimes never had such a strong automatic tool at their disposal. Therefore citizens today can either play by such rules or leave the country — resistance is essentially futile. And just as importantly, people essentially want what government wants, or skills from which to promote Chinese wealth. Thus a symbiosis seems to exist here.

Consider how evolution handles wealth based collectives found in eusocial creatures like ants and bees. Why is there so little conflict between such individuals? Because evolution provides genetic acceptance for whatever roles exist. We humans however didn’t evolve to exist as parts of such massive societies. Instead civilizations were permitted by means of government rule, and often with tremendous conflict and waste. China may have found a way to maintain civility and productivity by means of “mind control”, or published expectations that determine punishing to rewarding implications given massive social monitoring. I suspect this will help China become wealthy enough to either become the universal government of humanity, or certain other countries will become powerful enough to remain independent by means of their own mind controlled citizens.

Next time I look forward to hearing about the somewhat different assessment that you’ve come to Lionel. For future essays however I hope you’ll also consider the possibility for the human to become extremely civil while remaining innovative by means of government based surveillance given its published expectations. I don’t like the thought of the end to liberal governing, though I also don’t currently see how this could potentially compete.