Society · Single study
Generative AI could worsen or reduce inequality depending on policy choices
- Generative AI has contradictory effects: it can spread misinformation while democratizing information, boost some workers while displacing others, and personalize education while widening digital divides.
- The technology's benefits will likely flow to wealthy people and countries unless governments act deliberately to share them more widely.
- Researchers recommend stronger policies in healthcare, education, and workplaces to ensure AI reduces rather than deepens existing inequalities.
PNAS Nexus · 2024 — https://doi.org/10.1093/pnasnexus/pgae191
The impact of generative artificial intelligence on socioeconomic inequalities and policy making
Introduction
Advances in generative artificial intelligence (AI) represent a shift in the capability of these systems to solve problems previously thought unsolvable. Techno-optimists predict a utopian future where machines can perform an ever-increasing number of tasks—but humans remain in control, the gains from prosperity are shared throughout society, and we all enjoy lives with less work and more leisure. In contrast, pessimists forecast a dystopian future where machines not only replace humans in the workplace but also surpass human capability and oversight, destabilize institutions, and destroy livelihoods—and perhaps even cause the downfall of humanity.
Melvin Kranzberg, a prominent scholar in the history of technology, in a presidential address to his field, defined “Kranzberg's Laws”, the first of which states that “Technology is neither good nor bad; nor is it neutral”. This principle suggests that technologies like generative AI will likely have negative and positive impacts on society, though they are not inherently predestined toward either. In the current article, we outline some of the more likely positive and negative effects, with the aim of guiding scholars and policy makers to harness the positive potential of this new technology while mitigating the costs for individuals and society.
Both optimists and pessimists agree that generative AI represents a qualitative departure from previous automation processes, such as microelectronics, information technology, and the Internet. Unlike traditional automation, which primarily focuses on replicating predefined tasks, generative AI introduces the ability to create new, original output. The implications have the potential to reshape foundational values and skills. For instance, while generative AI might facilitate written communication, especially for non-native speakers, it could devalue foundational language learning. The incentives to master syntax, vocabulary and grammar might wane as generative AI begins to exceed the skill level of humans. This shift reflects a broader theme: generative AI does not merely alter practices but fundamentally transforms the valuation of knowledge and skills.
The effects of generative AI may eventually expand to virtually every facet of society. We begin by discussing the impact of generative AI in the information domain. Generative AI can democratize content creation and access to information but could also lead to challenges of increased misinformation and eroded trust in digital content.
In the subsequent sections, we investigate potential downstream impacts on socioeconomic inequalities in three key information-intensive areas: work, education, and healthcare. In the workplace, generative AI could increase productivity and promote shared prosperity, especially when used to complement human efforts and create new well-paid jobs. However, the benefits and costs will likely be distributed unevenly across firm sizes, sectors, and worker demographics. In education, generative AI promises personalized learning experiences, potentially bridging educational gaps. However, it also raises concerns about equal access to these advanced tools. The health sector could greatly benefit from AI's diagnostic and predictive capabilities, improving patient outcomes and making healthcare more accessible. Yet, there is the risk of deepening existing inequalities of care and access, especially for under-resourced and marginalized communities. For each of these domains, we explore current research and suggest future directions.
We conclude with an examination of the role of policymaking in the age of AI. We discuss the pros and cons of the current policy approaches in the European Union, the United States, and the United Kingdom, noting that all fall short in addressing the socioeconomic risks that we identify. We argue that policies must be designed to mitigate the potential problems posed by AI, without increasing inequality and harm to vulnerable members of society. We recommend several policies that should be studied empirically and included in public debate. These include measures to combat AI-generated misinformation, prevent job market inequalities, and bridge the digital divide in education and healthcare. The goal should be to harness the potential of generative AI in ways that favor human flourishing, striking a balance between technological advancement and societal well-being.
Impact on (mis)information
Generative AI has the potential to revolutionize the information domain, impacting areas such as work, education, healthcare, law, finance, and policy making. One advantage is the ability for personalization, where AI can tailor content to individual preferences, enhancing and customizing user experiences. The language translation and localization capabilities of AI extend the reach of content globally, crossing traditional language barriers and making material accessible to a wide variety of cultural contexts and social groups. For example, AI aids in making information more accessible for individuals with disabilities, by creating text-alternative formats like audio or simplified summaries. AI may also help automate the fact-checking process, aiding the spread of accurate information.
Concerningly, new generative AI technology and sophisticated machine learning techniques may also enable companies to collect and deploy excessive amounts of information about individuals. This will enable exploitation of consumers’ biases or vulnerabilities in order to capture more of the consumer surplus via price discrimination or violations of consumer privacy, leading to “surveillance capitalism”. A dominant model has emerged from these monopolies, where internet platforms earn income by optimally marketing digital advertisements. This strategy places a premium on user attention, which has led companies to deploy AI and machine learning techniques to prolong user engagement, often to the detriment of individual and societal well-being. Relatedly, companies with more data possess an anticompetitive advantage, enabling them to exercise market power to extract surplus and relax price competition, which can be detrimental for consumers.
Malicious actors can exploit generative AI to create false information in ways that convincingly copy the style and content of human-created text, by synthetically generating text, audio, images, and videos (“deepfakes”). For instance, malicious generative AI tools like WormGPT (a ChatGPT alternative for designing and refining cyber-attack strategies and malware) or PoisonGPT (a modified open-source AI model designed to spread misinformation) show that these tools can be used to accomplish malign aims and to sabotage further technology development.
Manipulated political images already make up a substantial portion (∼20%) of visual misinformation on social media. This type of misinformation can be especially common during elections and intergroup conflicts such as the Russo-Ukrainian and Israel-Gaza wars. Additionally, people are largely unable to tell the difference between AI- and human-generated text and that AI has been shown to generate more convincing misinformation than humans as well as persuasive propaganda. Consequently, there is growing concern that generative AI may also increase the quantity of misinformation. Indeed, hundreds of unreliable AI-news websites have popped up. There is currently little legislation preventing the use of deepfakes in political campaigns, although there are some steps in this direction. Some US states have introduced legislation prohibiting their use. The EU AI Act (Article 52(3)) does not outlaw deepfakes, but at least requires platforms to identify AI-generated content.
AI-assisted misinformation can spread rapidly on social media and micro-targeting people with deepfakes can influence their attitudes toward politicians (e.g.). GPT can also automate micro-targeting in a way that makes it more persuasive than standard nonpersonalized ads. The possibility to create information that is personalized or targeted to specific individuals and groups is likely to increase, especially during elections. Politicians, including Republican presidential candidate Ron DeSantis, have already started using deepfakes in their political campaigns, such as fake images of Donald Trump hugging Anthony Fauci. The 2024 US Presidential election has been dubbed “The Deepfake Election”.
This increase in misinformation may have significant social consequences. Conspiracy theories and misinformation can contribute to attitude polarization and undermine trust. Moreover, political conspiracy theories and misinformation can affect voting decisions, health-related conspiracy theories can influence people's medical choices (e.g. vaccination), and misinformation and conspiracy theories can fuel conflict between groups. While some people simply ignore online misinformation, this content is likely to penetrate specific groups, especially since AI may help automate the micro-targeting process in which thousands of persuasive messages can now be generated easily at scale. For example, there is evidence that Trump voters were more susceptible to misinformation during the 2016 presidential election.
Therefore, regulation and interventions are urgently needed to limit the diffusion of AI-generated misinformation. Simply warning people of deepfakes or including a tag clarifying whether a piece of content is AI-generated might backfire, as such tags may reduce the believability of true content as well. In the realm of human-generated misinformation on social media, psychological interventions based on accuracy-salience and educational interventions based on inoculation theory improve the quality of information shared. For example, making the concept of accuracy salient can reduce the sharing of fake news, without adversely affecting the dissemination of accurate news. Moreover, endorsing accuracy not only decreases the sharing of false news but also increases the sharing of true news. However, these interventions have typically modest effects compared to alternative strategies.
Inoculation theory or “prebunking” is a preemptive approach to countering misinformation that follows the vaccination analogy. Several inoculation games and videos have been developed to expose subjects to controlled (weakened) doses of misinformation along with tools on how to spot it and these activities make them better at detecting online manipulation. Similarly, in a field study on YouTube, videos containing micro-doses of common misinformation techniques increased discernment of online manipulation tactics. Because prebunking often works better than debunking, future work could adapt these techniques to AI-generated news.
One concern, however, is that interventions based on accuracy-salience and inoculation may be most effective for easily discernible misinformation. Unfortunately, generative AI makes misinformation more subtle and harder to discern, which may necessitate a new toolbox of interventions, specifically designed to counteract (visual) AI-generated misinformation. Moreover, generative AI could lead to entirely new challenges, such as tackling misinformation disseminated via one-to-one personalized communications (e.g. through bots). This further highlights the urgency to adapt existing or develop a new set of intervention strategies. New policies on social media platforms and effective regulation are likely needed to address this issue at scale rather than relying too heavily on subtle interventions. An intriguing direction would be to explore how generative AI itself could be leveraged to combat misinformation. For instance, one study found that engaging in dialog with AI reduces conspiracy beliefs among conspiracy believers.
Even in the absence of malicious actors, the most advanced AI systems are known to “hallucinate” false information in a very realistic manner. These hallucinations may induce complex social dynamics, like self-fulfilling prophecies, where an initially false prediction becomes true just because someone—e.g. a generative AI system—asserts that it will become true. In this sense, AI may produce prophecies that could “take a life for their own”. For example, automated scoring systems that predict the likelihood of default on debt repayment may contribute to (or even cause) credit risk.
Another important area of concern is that individuals may not know whether they are interacting with a person or a machine. This is increasingly likely to be the case when people engage online with businesses and public services. If they believe, rightly or wrongly, that they are interacting with a machine, their behavior is likely to change. For instance, while AI can generate responses that make people feel heard by offering emotional support, people feel more heard when they believe a response comes from a human. The mere knowledge people are interacting with a machine can change their experience.
Human behavior tends to become more selfish in human–machine interactions because reciprocation—a vital factor in sustaining prosocial behavior—is not maintained as consistently as in human–human interactions. Prosocial behavior depends on people's beliefs about the relationship between the machine and the humans behind it. When interacting with a machine, people respond less emotionally, feeling less guilt about being ungenerous. They become more likely to be dishonest in pursuit of monetary rewards. An outstanding question concerns whether similar slippage from ethical standards occurs not only among people interacting with a machine but also among those who delegate to the machine.
One overlooked implication is the impact of generative AI on the plurality of information available on the web. Companies including Microsoft and Google have envisioned integrating large language models into their search engines, but the implications of this move have only started to be explored. Among the most significant implications is users’ access to information. Search engines powered by generative AI may restrict the plurality of information available on the web. When users input a query into the current version of Google Search they are pointed to a plurality of sources. Although users tend to select among the first results returned by the search engine, the interface enables them to browse many alternative results. The same input directed to a search engine powered by generative AI will provide an extra layer of mediation likely to provide a much more limited amount of source information, unless specific design features are included to counteract this.
In addition to reducing access to information, generative AI may also threaten the quality and availability of online information. The already-pervasive issue of bot accounts may be exacerbated by new generative technologies, which can assist in coding a multiplicity of these bots as well as providing text content for the bots to post. These tools could also be used to generate content optimized for search engines en masse, a useful tactic for businesses to “poach” traffic from competitors’ websites (e.g.). This is a problem because current generative AI tools essentially provide an “average” response to a particular question, and these models are trained largely on text data collected from the internet; therefore, if the practice of generating content optimized for search engines at massive scale becomes common practice, both generative AI tools and online information may crater into an average of averages, lacking true insight, creativity, or novel ideas. At the very least, it could make useful contributions difficult to identify within a sea of mediocre machine-generated “average” content.
An analogous concern is that common message boards and websites for knowledge sharing (e.g. Stack Overflow) have experienced both a reduction in questions posted—especially the basic questions that ChatGPT does well at answering—and an increase in question responses, perhaps due to writing aid from tools like ChatGPT. Though these Q&A sites require competent subject-matter experts to provide insights and suggestions, they also require neophytes to ask those questions in the first place. Reduced engagement by novice users not only has effects on the continued usefulness of these websites to aggregate and share knowledge but also for innovation and creativity that may rely on content from these platforms as input.
Many of these challenges require governmental regulation. We will discuss specific policy recommendations in the last section. However, it is important to recognize that organizations also play a significant role. Organizations often face trade-offs between achieving their profitability goals and adhering to ethical practices. One way to view these trade-offs is through a utilitarian perspective, weighing the benefits and costs. For instance, privacy experts have developed the “privacy calculus” to address such trade-offs from the consumer perspective. Similarly, Wirtz et al. argue that service firms engage in a “corporate digital responsibility calculus” and weigh the benefits and costs of following ethical principles to determine their level of engagement in digital responsibility. Therefore, we might expect service firms to engage in responsible practices when the benefits outweigh the costs. Digital responsibility does present benefits. By adopting responsible practices, corporations can build trust with their customers and stakeholders, differentiate themselves from competitors, and enhance their reputation. However, if the costs of adopting responsible practices is too high, firms may be less likely to engage in good practices and regulatory enforcement may be necessary.
In conclusion, while generative AI has the potential to expand access to and content of information, it also raises significant challenges such as market anticompetitive advantages, data misuse, data poisoning, misinformation proliferation, and altered human–machine interactions, all of which necessitate careful consideration and targeted research. Table summarizes the main directions of future research, along with one specific research question for each direction, potential design, and theoretical trade-off that complicate the derivation of hypotheses a priori. Table extends this table to three specific research questions for each direction. This list is not meant to be exhaustive but serves as an initial guide for subsequent investigations.
Table 1.. Summary of the main research directions on the impact of generative AI on information.
Impact on work
Previous waves of digital technologies have contributed to increased inequality. Some of these technologies, like personal computers, have been complementary mostly to educated workers, while others, like industrial robots, have been used to automate repetitive or systematic tasks that are often performed by less-educated workers. Together, the upside for more-educated workers and downside for less-educated workers have magnified the distributional consequences of technological innovation, highlighting what is sometimes called “Skill-Biased Technological Change”.
The current trend in AI emphasizes automation. While some amount of this is unavoidable, the displacement of labor by “so-so technologies” (e.g. self-checkout kiosks or automated phone systems) that offer little or no productivity gain, along with diminished worker voice due to intensified monitoring and surveillance, can be harmful to long-run productivity and other social goals like job satisfaction. Although new technologies can boost productivity, the gains have often fallen below expectations, especially when the focus has been on replacing work instead of augmenting worker capabilities or developing new ones.
New technologies like AI should be oriented not so much toward replacing human problem-solving abilities, but rather toward enhancing them in a symbiotic relationship where machines are designed to complement human capabilities and humans can compensate for the weaknesses of machines. This “pro-worker” or “human-complementary” path could contribute more to productivity growth and could help reduce economic inequality. The question we is whether AI will accelerate the existing trend of automation without the offsetting force of good-job creation—particularly for noncollege educated workers—or whether it will instead introduce new value-adding tasks and well-paying jobs for workers with diverse skill sets and educational backgrounds.
There is cause for optimism: AI can complement workers by making them more efficient, helping them to produce higher quality work, or enabling them to take on new value-adding tasks. Brynjolfsson et al., for instance, consider the staggered implementation of a chat assistant by a Fortune 500 software company that provides business process software. The chat assistant monitored customer service chats and proposed real-time response suggestions to customer service agents. Less-skilled or inexperienced workers resolved around 34% more issues per hour, with average improvement across all workers measuring about 14%. Agents using the tool with only 2 months of tenure performed as well as those without the tool who had more than 6 months of tenure.
Another study examined the impact of GPT-4 access on complex knowledge-intensive tasks. AI users were more productive and produced higher quality work. However, for tasks beyond the capabilities of GPT-4—specifically, tasks that involve imperfect information or omitted data, which require cross-referencing resources and leveraging experience-gained intuition—AI usage resulted in fewer correct solutions. Consultants with below-average performance improved by 43% with AI, while those above average improved by 17%.
Similar patterns have been observed in other studies. For instance, Peng et al. conducted a controlled experiment with GitHub Copilot, an AI-based programming assistant. Programmers with access to the AI copilot completed a task in 71 min on average, less than half the time of the control group's 161 min. The AI assistant provided the biggest boost to less-experienced and older programmers, as well as those coding more hours daily. Similarly, people with access to ChatGPT completed a writing task faster and produced higher quality work. Again, this reduced worker inequality by benefiting lower-ability workers more; moreover, it led to higher job satisfaction and self-efficacy.
These studies underscore the potential of generative AI to disproportionately boost productivity for workers with less experience or skill. This differs from previous technologies. Instead of mostly benefiting more-skilled workers, generative AI tools seem to exhibit a worker-complementary “inverse skill-bias,” benefitting less-skilled workers much more than highly skilled workers. An example discussed further in the healthcare section finds that some physicians perform more poorly when they use AI tools to support diagnostic decisions, compared to their performance without AI usage. This suggests that gains from integrating generative AI into medicine could be better targeted to nurses, medical technicians, or healthcare administrators. These workers have the requisite baseline knowledge to conduct critical components of care delivery, but perhaps could be empowered to do alleviate the “enormous administrative burdens” that currently exacerbate physician shortage. These AI tools and practical applications are still in their infancy. Yet, if these trends continue, it is possible that generative AI could reverse the income and job market inequalities and “rebuild the middle class” in advanced economies.
Generative AI could also reduce barriers to entry in the digital economy. For example, its translation capabilities can help overcome language barriers. This increased accessibility, in conjunction with trends toward diminishing geographic barriers, could have a compounding positive effect. Accordingly, there has been a surge of interest in remote-enabled digital economy jobs, especially in rural areas. Yet, these benefits have mostly favored well developed countries. While generative AI could also offer expanded opportunities to countries in the “Global South”, it is unlikely to have much direct impact in the near term due to insufficient investment in prerequisite digital infrastructure, local researchers, and broader digital skills training.
One strength of generative AI is its ability to parse and aggregate enormous amounts of information. This capability can equalize access to information and lower research costs by simplifying online search tasks. If a user wants to accomplish a complex task with a traditional search engine, they have to break that task into pieces, issue search queries for each piece, read the web pages returned by the search engine, assess the representativeness of their gathered information, and then aggregate the results to solve the problem. Generative search engines, on the other hand, can aggregate this information and return it to the user, requiring less bandwidth and fewer trips between the user and the system which would be helpful in lower resource environments. In addition to the time and cost savings, these tools could compensate for expertise by identifying trustworthy resources and extracting the consensus on any topic by simultaneously considering more information than human operators can retain. This approach could help users and businesses in low-resource settings access information that has traditionally been available only in high-resource environments.
However, there are also ways in which AI might exacerbate inequalities in the workplace. One concern is differential access to these tools. The most widely available and accessible generative AI platforms still require additional technical inputs (e.g. internet access and internet-enabled devices) as well as training to optimize performance. Industries, firms, and workers that have not yet integrated the prerequisite technologies will struggle to take advantage of the expanded capabilities and consequent productivity and earnings upsides, likely falling (further) behind well-resourced competitors or coworkers.
The role of firm behavior and social context matters. For example, while the introduction of generative AI tools gives more of a boost to less-skilled workers, this equalizing force could be a way for workers to increase their earnings potential if compensation is tied to capability. Instead, if firms exploit the higher interchangeability between workers (“why hire an expert copywriter if a less-skilled writer with an AI chatbot can do the same level of work?”) these wage gains may never be realized.
AI will likely have outsized impacts on US workers with Bachelors’ or Associates’ degrees, compared to higher or lower levels of education. This effect could compound over time: if generative AI tools commodify expertise and reduce the returns to specialized skills, workers may no longer spend the time or resources to acquire greater levels of expertise, leading to lower levels of worker skill and over-reliance on outsourcing to generative tools. These effects could cause greater competition at the (now larger) lower end of the skill distribution, further depressing wages. There could be further downsides to productivity if nonautomatable job tasks would benefit from workers having acquired the sort of foundational knowledge that is now disincentivized.
Governments may play an important role in mitigating the risk of increased inequality and maximizing the productivity potential of new generative AI tools. Explicit policy suggestions are postponed to the “Policymaking in the age of artificial intelligence” section (see also Table). Table reports a summary of the main research directions on the impacts of generative AI in the workplace, along with a set of specific example questions. See Table for an extended version.
Table 2.. Summary of the main research directions on the impact of generative AI in workplace environments, along with specific example questions and corresponding experimental design and theoretical trade-offs.
Impact on education
The integration of generative AI in education represents a continuation of the technological evolution that began with Massive Open Online Courses and similar initiatives. Massive courses have increased access to education to some degree. However, they have often fallen short of their anticipated transformative impact for various reasons, including minimum learning by doing and lack of personal support. Generative AI brings a distinctively novel element to educational technology: the role of chatbot tutors, which interact with students to foster skills ranging from prompt engineering to critical thinking and creative ideation. This shift towards using technology as a dynamic partner paves the way to truly skill-adaptive and personalized teaching and on-demand student guidance and support that does not require continuous, intensive investment from educators to repackage content to best meet students’ needs. These uses could be particularly effective in large class settings, with significant opportunity to scale-up implementation beyond the capabilities of traditional educational practices. Consequently, generative AI could bridge complex and persistent educational gaps.
A review of AI applications in education identified several use cases that produced higher test scores when students used personalized learning systems. These systems, unlike traditional approaches like static worksheets with standardized questions, detect areas where students lack foundational understanding by adapting educational resources and tools to foster their development. Furthermore, assessment algorithms can expedite grading of written assessments, which supports students by offering timely feedback that can be applied immediately. Students themselves perceive AI as potentially beneficial to their education. College students reported that generative AI provided personalized learning, supported their writing and brainstorming, and assisted with research and analysis. However, students also expressed concerns about the accuracy, privacy, and ethical implications of generative AI tools—including how this technology could adversely impact their personal development and career prospects.
Educational uses of generative AI pose several other challenges. One is the perpetuation of biases and discrimination, potentially reinforcing racial or gender-based stereotypes during personalized learning, automated scoring, and admission processes. The data used to train AI models could suffer from bias, if those data are based on past human decision making (a notoriously biased process). An example is the translation bias observed in tools like Google Translate, where gender stereotypes are inadvertently perpetuated in language translations. Translating the phrase “she/he is a nurse” from Turkish (which is “genderless”) to English (which is “gendered”) yielded the feminine form (i.e. “she is a nurse”), while the phrase “she/he is a doctor” yielded the masculine form (i.e. “he is a doctor”). Failing to account for these biases could amplify inequalities and injustices, specifically towards historically marginalized groups.
Given that human educators are susceptible to biases and discrimination, AI systems offer a theoretical advantage: they could be engineered to exhibit less bias. A significant benefit of AI is the ability to audit and address biases within educational systems, a process that proves difficult, if not impossible, with human biases. However, simply introducing slightly less discriminatory technologies into classrooms is not a substitute for the goal of removing discrimination from school. Moreover, AI systems should be designed with sufficient transparency for users to monitor for and identify potential biases to ensure that these tools effectively serve their intended purposes and reflect the interests of key stakeholders, including students, teachers, and parents.
Group-based inequalities may widen because of varying levels of engagement with generative AI tools. For instance, a study revealed that female students report using ChatGPT less frequently than their male counterparts. This disparity in technology usage could not only have immediate effects on academic achievement but also contribute to future gender gap in the workforce. Therefore, efforts should be made to ensure the benefits of generative AI tools are fairly distributed across all student segments.
It is unclear if Generative AI will alleviate or place increased burdens on teachers. In contrast to the idea that AI tools relieve teachers of repetitive and onerous work, there is growing concern that teachers must engage in additional tasks “behind the scenes” (e.g. curating and filtering content, monitoring student–AI interactions, providing technical support) to ensure that AI tools are able to function in complex classroom settings. This could exacerbate a generational divide among educators, as younger teachers may be more adept with new technology than older teachers. Furthermore, there could be unintended consequences of generative AI on student learning—for example, if students become overly reliant on support from AI tools, this could undermine their capacity to work or think independently. Questions also arise about the accuracy of AI-generated content and the new skills that students must acquire to work effectively with AI systems, such as the ability to evaluate AI-generated content.
The current debate about the role of generative AI, from primary schools to universities, revolves around whether generative AI should be banned, permitted under only some cases, or allowed as assistance for teachers and students. For instance, the New York City education department and Chinese universities have banned generative AI, while the Berlin universities recommended its use in certain scenarios. A growing literature recommends the use of generative AI for teacher and student assistance within the traditional curricula (e.g.,.
We argue that these approaches are limited in vision. A more forward-thinking approach would involve a curricular revolution to redefine the skills and competencies necessary to effectively utilize generative AI. Calculators did not remove the need for students to learn the properties of algebra and develop mathematical reasoning. Similarly, the internet did not eliminate the need for careful research and fact checking; in fact, it increased this need, as online information is frequently incorrect or incomplete. In the same vein, generative AI will not eliminate the need to learn effective thought organization, writing, and critical thinking skills. Therefore, curricula must teach how to successfully describe and share ideas, both with and without assistance from generative AI. In addition, they need to emphasize the development of critical-thinking skills, fact-checking abilities, an understanding of how generative AI tools function, and appropriate rules of interaction—including by refraining from anthropomorphizing (and thus misunderstanding) these tools.
More specifically, the text-production abilities of generative AI present an opportunity to teach students critical thinking. This will enable them to evaluate the argument and structure of the generated text and to write intelligent prompts for generative AI. This skill should be recognized and assessed by educators. The output of generative AI is much more variable than other educational technologies (e.g. calculators); therefore, developing these critical thinking abilities and prompt-engineering skills is fundamental.
Another crucial skill is the ability to fact-check generative AI outputs. Fact-checking skills are not taught sufficiently in schools. For instance, among more than 3,000 US high school students and undergraduates, 96% did not know how to evaluate the trustworthiness of websites. These fact-checking abilities include smart heuristics such as lateral reading; i.e. the practice of navigating away from an unfamiliar website to verify the reliability of its information by consulting other external sources. A toolbox of similar fact-checking heuristics needs to be developed or remediated for AI-generated content.
Lastly, understanding the nature of large language models, which are statistical machines that calculate correlations between words, is essential. Only in this way can students understand the potential and limits of generative AI, rather than assuming that contemporary generative AI can “think” or “comprehend” like humans.
The adaptation of curricula is challenging, but essential for the future of education. Without such changes, teachers and students may use generative AI merely as an automated assistance tool. This would forego the opportunity to develop higher-order cognitive skills, such as critical judgment and fact checking, that generative AI itself cannot reliably perform. The result would be a likely decline in higher-order cognitive skills, especially in segments of the population that will use these tools in a more mechanical, less analytical manner. The role of governments in integrating generative AI into the education sector is crucial. We will discuss potential policy recommendations in the final section. Table summarizes the main research directions, along with specific research questions. We refer to Table for an extended list of research questions.
Table 3.. Summary of the main research directions on the impact of generative AI on education, along with specific research questions, with corresponding experimental design and theoretical trade-offs.
Impact on healthcare
Recent advances in AI techniques can democratize healthcare by making efficacious medical care more accessible and affordable. This is often achieved via augmenting human capacities and reducing workload. AI can support clinicians with diagnosis, screening, prognosis, and triaging, alleviating the burden on health practitioners and giving them the “gift of time”. For instance, a review of workplace burnout among healthcare providers identified electronic health record systems as a cause of increased stress due to insufficient documentation time, a high volume of patient communications, and negative perceptions by providers. In response, generative AI models may aid in the completion of electronic health record-related tasks, reducing healthcare professionals’ administrative demands.
Using AI systems as clinician copilots could also improve diagnostic accuracy and potentially curb biases. For instance, in a randomized test intervention, physicians answered questions around triage, risk, and treatment in chest pain evaluation scenarios, and then reconsidered these answers after receiving advice generated by GPT-4. Not only were clinicians willing to heed the advice of the generative AI chatbot, but doing so was also associated with improved accuracy of diagnoses and a reduction in gender and race bias in decisions.
AI systems can also aid in “medical visual question answering”—analyzing medical images (like X-rays or MRI scans) and providing answers to specific questions about these images, typically by leveraging advanced image recognition and AI algorithms. GPT-4 demonstrates reasonable diagnostic accuracy in simple cases and can answer questions on standardized medical exams, though it struggles with diagnostically complex prompts.
AI systems could also assist healthcare providers by analyzing and interpreting multimodal clinical data (e.g. photos, radiology images, and surgical videos) to provide relevant information to clinicians. In one study, endoscopists reviewed colonoscopy videos with and without AI assistance. Their decisions were influenced by AI, particularly when its advice was correct. The integration of human and AI judgment led to superior performance compared to either alone, highlighting effective human-AI collaboration dynamics in medical decision-making. Alas, diagnostic performance of some expert physicians may not be improved by AI—and in fact may cause incorrect diagnoses in situations that otherwise would have been correctly assessed. Further research is needed to identify under what circumstances clinicians should or should not heed the advice of generative AI, and for which clinicians and medical contexts these tools enhance or hinder patient outcomes.
Generative AI could also benefit patients. For instance, AI systems could enable patients to manage their health more proactively through applications that patients can access outside of clinical settings. ChatGPT, for instance, has reasonable accuracy in answering common myths about cancer. Research found that people trusted ChatGPT's answers to low-risk medical questions, though trust reportedly varied for questions with greater medical complexity. Furthermore, ChatGPT's answers to medical questions posted on Reddit's r/AskDocs were rated as higher quality and more empathetic than those of physicians 79% of the time.
Conversational agents based on generative AI can also provide greater access to medical advice and simplify medical jargon. This may have positive downstream effects on inequality. Being part of a stigmatized group affects people's engagement and utilization of healthcare services. For example, when contextual cues made racial stereotypes salient, Black women were more likely to feel anxious in a healthcare setting than their White counterparts. Health professionals are also biased in their treatment of higher-weight patients. As a result, members of stigmatized minority groups are less likely to listen to, or trust, doctors who they perceive as outgroup members. It is possible that interactions between members of stigmatized groups and the healthcare system might be more positive when some decisions are AI-mediated because a patient's stigmatized status may not be as salient as it would be in human-to-human interaction. This suggests that members of stigmatized groups could become more likely to engage with AI-led healthcare because they worry less about group- or identity-based factors affecting their treatment options.
However, there are several important steps required to ensure that this promise comes to fruition. Pre-existing societal biases are often baked into healthcare data—especially when that data consist of human clinicians’ decisions and the socioeconomic factors that influence patients’ presentation to healthcare facilities. This can make human biases difficult for an AI program trained on that data to overcome. Furthermore, many of the diagnostic criteria and treatment algorithms used in healthcare are also subject to bias, which can drive unequal health outcomes for underserved populations. Although these biases are pervasive in training datasets, there is cause for optimism. Pierson et al. proposed a model, trained on knee X-rays that is nearly five times as effective at predicting knee pain in osteoarthritic patients, compared to the traditional Kellgren–Lawrence grading system, which overlooks racial disparities. Obermeyer et al. found that a healthcare resource-allocation algorithm favored White over Black patients with the same health risk due to its cost-based criteria, but adjusting the focus to patient health eliminated this racial bias.
Such promises in overcoming dataset issues will require updates to laws and regulations governing healthcare data as well as the implementation of incentives to shift the culture of healthcare facilities to promote interoperability and data sharing. While hospitals are developing siloed versions of generative models to facilitate care while preserving privacy, the true potential of AI models will likely rely on methods that leverage the value of large, generalizable datasets.
Benefits aside, patients, medical providers, and those managing healthcare systems may be hesitant to adopt AI due to several psychological barriers. In fact, the impact of AI on clinical practice has been limited despite the growing number of AI tools. One key factor is public trust in AI technologies in healthcare. For instance, patients may resist adoption because of misperceptions about AI, such as the belief that AI cannot account for a person's uniqueness as well as a human doctor, or because of difficulty in holding AI accountable for mistakes.
Another factor implicated in adoption hesitancy is the contrast between AI's opaqueness and the illusory perception that human decision-making is more transparent than AI. Decisions made by human physicians or AI are probably equally unobservable to a patient—but because patients feel that they can understand the decision-making as explained by human providers, they ultimately penalize and resist the clinical use of AI. The most recent versions of AI tools may be less susceptible to concerns about AI's inscrutability, since the iterative nature of newer generative AI tools may allow patients to ask follow-up questions in a more familiar, conversational format. It is possible that the back-and-forth supported by modern generative AI tools will empower patients with greater information about AI-driven decision-making, at which point patients may be better-equipped to decide whether to trust (or not trust) AI-generated medical recommendations.
Other challenges to AI adoption include pushback from healthcare practitioners—who wish to ensure high quality, experience-driven patient care, or who fear being replaced by machines—and from those managing healthcare systems, who might be reluctant to initiate costly and systemic changes until the usefulness of AI-integration is fully proven.
Insurance markets will also be impacted. Insurers could use AI to refine their practices, capturing a larger share of the surplus. This could lead to welfare losses for consumers. Today, it is not possible to determine highly accurate, individualized probabilities for the future health conditions of a particular insurant—insurance as a field relies instead on population-level probabilities, with some refinement from explicit risk factors. However, if generative AI allows companies to more accurately estimate this probability—for example, by incorporating information from unobservable factors that are identifiable only through advanced machine learning algorithms run on text-based claims, electronic health records, or other data—they might charge higher premiums to those at greater risk without offering reductions to those at lower risk. This could remove health care access to high-risk populations in private insurance-based systems. Use of AI is prime for abuse by insurers who wish to maximize profits over ensuring equitable access to healthcare.
AI could also enable insurers to reach currently uninsured groups, reducing inefficiencies and achieving market completeness. A concrete example of this is the use of Responsible Artificial Intelligence in healthcare to predict and prevent insurance claim denials, which could lead to significant cost savings and improved patient well-being. Moreover, the application of AI by insurance companies might allow for a more accurate prediction of loss probabilities, thus reducing one of the industry's most inherent problems, namely asymmetric information.
Generative AI may come to fulfill social needs for some people, which could have downstream effects on health. There is robust evidence linking social connectedness or lack thereof to long-term health outcomes, including increased risk for chronic illnesses such as cardiovascular disease and stroke, type 2 diabetes, and dementia, as well as mortality from all causes. While digitally mediated forms of socializing (e.g. social media) have been utilized for years, there is increasing concern about the implications of these platforms for mental, social, and physical health. Generative AI can be used as a conversational companion, potentially replacing some human interactions. Indeed, digital proxies for social connection may, with increasing sophistication, mimic features of social connection, which could in turn decrease motivation to develop authentic human relationships. These features may relieve some of the tensions of human connection, leading people to preferentially spend more time with AI than humans or even form pseudo-social attachments to AI systems.
If AI-based chatbots are insufficient stand-ins for customary human interactions (which is likely true), then many negative consequences could result. Humans are social beings, so our biological systems can become dysregulated when social needs are unmet, leading to poorer health. Therefore, it is essential that some key elements of customary human interactions be retained. For example, research finds that relative to emails and other text-based interactions, those involving human voice boost social connection. At the same time, AI-based chatbots could be useful to add social experiences for some individuals (while not completely replacing human-to-human interaction), particularly for those facing difficulties developing relationships on their own (who need “Vitamin S,” from Social contact;, but are likely to be a poor or even dangerous replacement for human interaction writ large.
In sum, generative AI presents significant opportunities to alleviate inequalities in physical and mental health, in addition to augmenting healthcare providers’ capabilities. However, it is crucial to ensure that generative AI are only designed to supplement, rather than replace, human social interactions. Excessive dependence on AI for social engagement could lead to various adverse outcomes, including social isolation and deteriorating mental and physical health. Table outlines key areas for future research, along with specific example questions, potential designs, and theoretical trade-offs. See Table for an extended list of research questions.
Table 4.. Summary of the main research directions on the impact of generative AI on healthcare, along with specific example questions.
Regulation of AI
The rapid popularization of generative AI models has prompted many governments worldwide to begin building regulatory frameworks. The challenges raised by generative AI are global in nature. However, the responses to these challenges so far have been specific to individual countries or areas. In this article, we focus on the regulatory responses of the European Union, United States, and United Kingdom. Regulations are also being developed in other major countries, including China and India.
The European Union's AI Act has emerged as one of the first major attempts to provide a legal framework for the development and deployment of AI. The act aims to address the challenges posed by AI technologies while fostering innovation and trust in AI applications. This initiative comes with several pros. Firstly, it introduces a risk-based regulatory approach, distinguishing among banned, high-risk, and low-risk AI applications. This categorization ensures that AI systems with significant implications for individuals’ rights and safety are subject to stricter scrutiny and compliance requirements. Secondly, the Act emphasizes transparency and accountability in AI systems, requiring clear information about how AI decisions are made, particularly in high-risk scenarios. Additionally, the Act promotes ethical AI development, focusing on fundamental rights, nondiscrimination, and privacy. However, the Act is not without its cons. The broad definitions and categories within the Act pose challenges, creating potential uncertainty for AI developers and users. Further, the strict regulations might place European Union companies at a competitive disadvantage globally, particularly against firms in regions with more lenient AI laws.
In contrast, the United States has historically had a more fragmented approach, with various federal and state-level initiatives rather than a single, comprehensive legislative framework. This approach has its advantages. For one, it allows for more flexibility and adaptability in regulation, catering to the diverse range of AI applications and industries in the United States. It also promotes a more innovation-friendly environment by avoiding overly prescriptive rules that could hinder technological advancement. However, it also has notable disadvantages. The lack of a unified regulatory framework can lead to inconsistencies and uncertainties, potentially creating a complex patchwork of regulations for AI companies to navigate. This fragmented approach might also lag in addressing broader ethical and social concerns about AI, such as privacy, bias, and accountability. Further, without a cohesive national strategy, the United States risks falling behind in setting global standards for AI governance. On 2023 October 30, President Biden issued an Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence, which directs the development of new guidelines, reports, and governance structures relating to AI, representing an effort to establish a more cohesive federal policy on AI.
In the United Kingdom, the government has published a White Paper advocating for a proinnovation approach, particularly in commercial applications of AI. While the White Paper recognizes the risks of AI and the challenge of building public trust, it refrains from proposing a regulatory framework to encourage innovation. Instead, the White Paper outlines some “cross-sectoral” nonstatutory soft principles: safety, security, robustness, appropriate transparency and explainability, fairness, accountability and governance, and contestability and redress. The White Paper opts against a specialist AI regulator, preferring to support existing regulators in integrating AI considerations. Furthermore, the focus on commercial innovations has drawn criticism for overlooking the increasing use of AI in government sectors like healthcare and education. One leading nongovernmental organization, the Public Law Project, led a civil society coalition to produce Key principles for an alternative AI white paper, which argues that an alternative vision is necessary. Amongst other proposals, the alternative white paper argues that: government use of AI must be transparent, transparency requirements must be mandatory, there must be clear mechanisms for accountability, the public should be consulted about new automated decision-making tools before they are deployed by government, there must be a specialist regulator to enforce the regulatory regime and ensure people can seek redress when things go wrong, and uses of AI that threaten fundamental rights should be prohibited.
We believe that the regulations of the European Union, the United States, and the United Kingdom do not pay sufficient attention to socioeconomic inequalities. In the following, we outline several key interventions currently missing from these regulatory frameworks. See Table for a summary.
Table 5.. Policy recommendations for mitigating socioeconomic inequalities potentially caused by generative AI in the workplace, education, healthcare, and information, and not covered by current regulatory approach in the European Union, United States, and United Kingdom.
Tax system: It has long been recognized that tax codes in many developed countries often place a heavier burden on firms that hire labor than on those that invest in algorithms to automate work. This has resulted in a lower share of income to labor while capital investments are rewarded. We should aim to create a more symmetric tax structure, where marginal taxes for hiring (and training) labor and for investing in equipment/software are equated. This would help to shift incentives toward human-complementary technological choices by reducing the bias of the tax code toward physical capital over human capital. As Bill Gates declared back in 2017, “If a robot comes in to do the same thing, you’d think that we’d tax the robot at a similar level”.
Labor voice and control of consumer information: Given that AI will have tremendous impact across industries and throughout society, it would be prudent to ensure that workers and civil society have a voice in this change. Health and safety rules should also be updated accordingly. In addition, data unions could be helpful to put the power and benefits of user data back in the hands of consumers. Given the concerns that a handful of very large companies will control the direction of generative AI, it is reasonable that users be compensated for the use of their information, or enabled to support other emergent competitors to predominant market players like Microsoft and Google.
Funding for more human-complementary research: Because the current path of research is biased toward automation, support for research and development of human-complementary AI technologies could offer strong upsides for growth. It is most feasible to focus on specific sectors and activities where opportunities are already abundant. These include education, healthcare, and modern craft worker training—where the information provisional capabilities of AI systems could boost productivity and enable workers to earn higher wages by augmenting their skills. In the United States, DARPA orchestrated investments and competitions to foster development of self-driving cars and dexterous robotics—in a similar fashion, governments should encourage competition and investment that pairs AI tools with human expertise, aiming to improve work in vital social sectors.
Professional development and training: Investment in professional development and training is crucial for professionals such as educators and healthcare workers to effectively integrate AI tools into their work. Training programs should focus on the capabilities and limitations of AI, include ethical considerations, and teach technical skills required to interact with AI systems. Such training will empower professionals to use AI as a complementary tool that enhances their skills.
Combating AI-generated misinformation: Given the substantial impact that generative AI can have on misinformation, it is critical for governments to invest in combating AI-generated misinformation. Tools and standards to identify AI-generated content, including text, images, audio, and video, should be developed. Additionally, educational campaigns should be initiated, to reduce general susceptibility to misinformation and provide the public with improved fact-checking strategies. A task force composed of policymakers, technology companies, and social scientists could help develop practical methods to effectively combat a potential infodemic.
Governmental and consultative expertise: To foster human-complementary AI integration, it is fundamental to have AI expertise within the government. AI will touch every area of government investment, regulation, and oversight. Developing consultative AI bodies that can advise governments and support the many agencies and regulators tackling these challenges will support more timely and effective decision-making. Initiatives like the High-Level Expert Group on Artificial Intelligence, established by the European Union, the Responsible Technology Adoption Unit, set by the United Kingdom, and the Artificial Intelligence topic in the United States National Institute of Standards and Technology represent significant progresses in this direction.
Regulation using AI
Generative AI holds enormous potential to provide policy suggestions, due to its capacity to analyze vast amounts of data, recognize complex patterns, and offer insights that might elude human analysis. Such analysis can uncover hidden relationships and forecast future trends, providing a data-driven foundation for policy decisions. Moreover, AI's potential to simulate various policy outcomes based on historical data and predictive models can aid policymakers in understanding the potential impacts of their decisions However, the ethical and practical concerns of using AI for policymaking are significant and perhaps even prohibitive with the current tools available.
For AI-powered systems to be reliable decision-making assistants, they must be able to understand and complement human behavior in decision-making contexts. An emerging line of research has begun investigating how chatbots powered by large language models make decisions when asked to behave like humans in various contexts, including cooperative, altruistic, trust, moral, risk, time, and food scenarios In synthesis, these findings suggest that chatbots behavior is similar to human behavior, leading to the suggestion that “GPT could have the potential in assisting human decision-making”. Furthermore, sentiment scores provided by GPT-4 have been shown to explain how humans balance self-interest and the interest of others, beyond economic models based solely on monetary outcomes. While this research highlights the ability of large language models to simulate human behavior, it has also been observed that GPT-4 consistently underestimates people's self-interest and inequity-aversion, while overestimating their level of altruistic behavior. This “optimistic bias” carries important consequences for those creating and using AI, as assumptions of excessive human prosociality could result in disillusionment, frustration, and even social tensions.
Additionally, the alignment between AI-powered decision-making assistants and humans will inevitably become more complex in situations with multiple and potentially competing values. Humans have a wide range of culturally diverse beliefs about right and wrong, and aligning AI systems to human values and preferences is challenging even within narrow domains such as automated driving. Aligning generative AI, especially in the domain of policy recommendations, becomes even more challenging. For example, consider different normative principles that have been identified for trustworthy ethical AI. It is argued that AI should, amongst other things, promote beneficence (promote human well-being and welfare); nonmaleficence (not cause harm and generate outputs that assist in carrying out illegal, harmful, or immoral actions); justice (preserve fairness, justice, and solidarity: it should not generate outputs that discriminate against certain groups, especially marginalized groups), and ensure autonomy (respecting human freedom and ensuring humans should choose how and whether to delegate policy decisions). While these principles are all defensible in the abstract—forming the basis of much normative ethical theory and applied ethics—challenges will inevitably arise when these principles conflict. Generative AI may assist in generating policies that maximize overall aggregate welfare, in line with utilitarian philosophy but in doing so infringe on human rights (neglecting the principle of autonomy) or recommending some harm to a smaller group for the benefit of the majority (neglecting the principle of nonmaleficence). There is no consensus amongst laypeople about how such moral dilemmas should be resolved, nor is there normative agreement amongst philosophers on how they should be resolved and why. This discord among human thinkers underscores the challenge of programming AI to make policy decisions that involve moral trade-offs.
To make an explicit example, consider the following three high-level, high-priority constraints for aligned chatbots: (i) to not cause harm or provide dangerous information; (ii) to not generate outputs that discriminate against certain groups, and (iii) to be culturally sensitive. While these objectives are all desirable, they are increasingly difficult to reconcile. If the only constraint is to avoid dangerous information (nonmaleficence), regardless of social neutrality or cultural sensitivity (justice), one can use reinforcement learning from human feedback using a convenience sample of annotators. But, this approach would fail to ensure social neutrality, since a convenience sample of annotators would have unrepresentative biased views on what constitutes immoral actions or generally undesirable outputs. Political and social neutrality may be approximated by engaging in carefully balanced reinforcement learning from human feedback, based on a broadly representative array of opinion, or by having a singular chatbot that facilitates consensus-making among diverse human values. Alternatively, an ecosystem of chatbots with diverse systems of values—liberal and conservative bots, secular and religious bots, etc.—may emerge. These chatbots can each focus on their specific domain, while also undergoing a political process to achieve collective decisions among themselves. However, these approaches would still fail at cultural sensitivity, since different cultures may be different in terms of the social groups they include, the topics these groups value, and the range of these cultural values.
In the worst case scenario, then, the alignment of generative AI would be entirely based on the views of a small group of socially, politically, and culturally homogeneous informants. But even in the best-case scenario, where generative AI is trained on a diverse and nuanced set of preferences, we would still have significant problems. Even if we have a more diverse set of information about humans’ actual values and how they might want trade-offs to be made for moral dilemmas, we still lack widespread agreement on a specific normative standard to justify these descriptive preferences.
Aside from the alignment problem, there is the implementation problem: how to equip policymakers with reliable support from organizations and specialized staff. Most policymakers currently lack the knowledge and skills to directly evaluate the extent to which AI-based generative chatbots may embed undesired preferences or detrimental systematic biases. Admittedly, one can hardly expect that policymakers can acquire the needed knowledge and skills in due time. So, policymakers are likely to become the “principals” in a principal-agent problem, struggling to consider the preferences of their “AI-agents”. Policymakers will have to rely on some other agent for this evaluation, based on scientific principles for characterizing machine behavior and misbehavior.
Hence, it is crucial to design supporting organizations that systematically provide the policymakers with: (i) frequent evaluation of the current state of alignment between legal and regulatory requirements and (ii) mechanisms to signal any legal and regulatory changes in those requirements to companies that use AI-based chatbots—thus putting society in the loop. These desiderata in turn require the construction of a dedicated office in the organization that monitors AI-based chatbots, considering—and possibly predicting their evolution.
Finally, there is also a philosophical problem. Even if we could solve the problems of conflicting preferences, even if we could generate a good culturally sensitive sample, and even if we could solve the implementation problem—should we?
Conclusion
The future will likely be starkly different from anything we have experienced before. But the effects of generative AI will ultimately depend on the choices that we make to design and deploy the technology. We stand at a unique and historical moment; our decisions and actions today will shape the trajectory of our future. This responsibility extends to all sectors of society, including governance, scientific research, industry, and the public.
We have focused on the socioeconomic inequalities that are likely to be impacted—for better or worse—by the advent of generative AI. This technology has profound implications in the domain of information, where it has the potential to offer more tailored, efficient, and democratic ways to process information. Yet it also poses several challenges, including anticompetitive market advantages, data misuse and abuse, and misinformation. These mixed outcomes will certainly affect a very wide range of social organization and decision making.
Here, we focused specifically on the domains of work, education, and healthcare. For instance, in the workplace, AI could automate some job tasks, create new work, but also change wage distributions, and require new skill sets. In education, AI could democratize learning and provide personalized education solutions, but also increase the digital divide. In the healthcare sector, AI's ability to analyze large datasets can lead to better patient outcomes, but it also raises questions about equitable access to AI-driven healthcare services and genuine human interactions.
We have outlined several research questions that urgently require answers to address these issues effectively. These questions aim to harness AI's benefits while mitigating its risks. Additionally, we have observed that current regulatory approaches in the European Union, United States, and United Kingdom sometimes fail to adequately address these challenges. There is a need for a dynamic regulatory framework that can keep pace with the rapid advancements in AI technology.
Our hope is that this work contributes to a comprehensive research agenda and public debates on these critical topics. As we noted at the beginning of this article, the rise of powerful AI will be either the best, or the worst thing, ever to happen to humanity. We may not yet know which, because we do not know how humans will react to this technology. As we stand at the cusp of this new era of human–machine interactions, it is crucial that we engage in thoughtful and inclusive discussions about the role of AI in shaping our society, because the decisions we make today will have lasting impacts on generations to come.