Many people are concerned that AI may be used to profile users so they may be better manipulated. LLMs can process vast amounts of information and, with chatbots and social media, can focus on very personal information. This could lead to ‘the algorithm’ being tuned for the individual to manage/manipulate them—standardising people to be compliant with the controllers’ system.

However, the other side is that while large-scale systems have traditionally had to rely on standardisation and one-size-fits-all (or none!) solutions, which users must then be made to comply with, LLMs and chatbots suggest that the practical need for such standardisation may no longer hold. AI can create bespoke, individually customised systems.

In principle it could go either way - but I think this is one of the most important moral challenge from AI and its widespread use.