Quantifying AI Risk: A Novel Approach to Understand and Prioritize AI Safety Concerns
Abstract
This article explores a new project proposing thorough methodologies for quantifying and prioritizing AI risks, addressing the growing need for robust risk assessment in the rapidly evolving field of artificial intelligence. Drawing from his unique professional experiences and education, the author proposes a framework combining actuarial science, AI expertise, and product-building insights to create a comprehensive AI risk database. This approach aims to provide more explicit, more actionable information for policymakers, businesses, and the public by crowd-sourcing risk estimates from experts. The article demonstrates this method using the example of AI-powered computational propaganda, estimating its potential impact on democratic processes and individual productivity. This project seeks to move beyond theoretical discussions and enable more effective AI governance and risk mitigation strategies by offering a way to compare and prioritize various AI risks.
AI Risk
So, it turns out that AI Risk is a thing.
And something several people care about.
Software has already eaten a large part of the world, and AI is eating software. Understanding and managing AI risk has become paramount.
But how do we measure something as complex and seemingly intangible as AI risk?
As part of what I learned in the AI Safety Fundamentals course and reflecting on my professional journey (actuarial science, risk industry, entrepreneurship, machine learning product management, academic research), I have decided to start exploring how to quantify AI risks, providing valuable insights for policymakers, businesses, and concerned citizens alike.
I collaborated on an AI risk think tank with risk management industry veterans and venture builders a year ago. Our goal? To define novel problem spaces, investigate solutions, and scope new ventures. While our solid venture proposals didn't gain immediate traction (a typical scenario in the slow-moving risk management industry), the landscape has since exploded with initiatives driven by research institutes, nonprofits, and AI scaleups.
Regulators have caught up, too, as evidenced by the flurry of AI-related laws and bills passed in recent years:
Despite this progress, confusion still reigns. Many existing AI risk repositories suffer from limitations:1
Copying chunks of text without proper context
Classifying multiple risks under a single category
Relying on theoretical risks without real-life examples
Several repeated entries crawling from the news
But they are a good starting point, and we shall definitely thank and keep contributing to such initiatives. As part of this initial phase of analyzing repositories, I built a simple Streamlit app to navigate and check items classified in the MIT Risk Repository. You can try it here, find the data to upload here and watch a quick video on how to use it here.
Quantifying the “Unquantifiable”
Drawing inspiration from the book “How to Measure Anything”2 and leveraging my unique background in actuarial science, AI, product management, and ethics, I've developed an initial, naive methodology for quantifying AI risks. The methodology entails deconstructing complex phenomena into quantifiable elements or their proxies and then establishing cogent mathematical relationships among these constituents. Risk uncertainty is subsequently characterized through probabilistic distributions, whose parameters are derived from a synthesis of scholarly literature, publicly available datasets, and calibrated expert judgments. The culmination of this process involves comprehensive scenario simulations, yielding quantitative insights into the overarching risk landscape.
Let's apply this to a complex risk: AI-powered computational propaganda undermining democratic processes.
Don’t get me wrong. This is a long, challenging, and resource-intensive process. But here’s where I am using the methodology I learned in building ventures and shipping products: we start from an MVP (which stands for “minimum viable product”), i.e., a prototype good enough to achieve a specific goal. My goals for now are to showcase the art of the possible with such analyses, explain why it matters, and attract people to work together on the broader project (sign up here if you’re one of them!).
Breaking Down the Risk
Computational propaganda, manipulating public opinion through social media platforms, algorithms, and automation, has become a significant concern in political communication (Woolley & Howard, 2018). This phenomenon involves using social bots, troll armies, and other automated systems to spread misleading information and influence collective attitudes (Pote, 2024).
As a first MVP, we can analyze this risk by considering:
Frequency of exposure to AI propaganda campaigns
Effectiveness of campaigns in swaying opinions
Impact on individual productivity and societal costs3
Running the Numbers
I have engaged in extensive research, conversations with experts, probabilities parametrization, and simulating scenarios.
The impact of computational propaganda on public opinion and political processes has been studied through various approaches, including platform neutrality, detection of manipulation agents, and analysis of network dynamics (Santini et al., 2018). While machine learning-based frameworks have been developed for bot detection, challenges remain in identifying coordinated botnet activities and incorporating image features (Pote, 2024). The rise of computational propaganda has necessitated a revision of existing conceptual frameworks in propaganda studies and highlighted the need for further research into its effects across different social media platforms.
Assuming that what I found in the literature4 and estimated thanks to experts’ calibrated opinions are reasonable, here's what I found:
As a European citizen, you face a risk of AI-driven propaganda decreasing your annual productivity (GDP per capita) by over $4,000. Exposure to such events runs up to four times a year.
On average, we "pay" a $580 risk premium annually, and we are 90% confident that this varies between $0 and $3,000.
In worst-case scenarios, degraded democratic institutions could cost us up to $20,000 in productivity per year.
In this analysis, I must acknowledge the inherent complexities and intangible factors at play. Some experts advocate disconnecting GDP metrics from measures like the Democracy Index, citing questionable correlations and a lack of established causality. Moreover, certain principles, such as democratic governance, may be valued intrinsically within specific ethical frameworks, irrespective of cost-benefit analyses.
It's crucial to recognize the diversity of global perspectives on governance. The Western-centric view of democracy as an unequivocal ideal may not resonate universally. Experiences across cultures reveal that different governance systems can yield positive outcomes for their populations. Even within democratic nations, the lived experience of freedom can vary significantly.
This global diversity underscores the importance of calibrating our assessments and broadening our understanding of governance models. It reminds us that concepts like freedom and effective governance are multifaceted and can manifest differently across cultures and historical contexts. As we engage in such analyses, we shall remain open to diverse viewpoints and avoid oversimplifying complex sociopolitical realities.
Finally, let me clarify that, as part of open-sourcing the AI risk database with quantifications, I plan to outsource as well a web app to play with these quantities and run different scenarios.
Further Work
By conducting this exercise and speaking with several great people who gave me feedback, I know there are several areas for enhancing the risk assessment model for AI-powered computational propaganda undermining democratic processes. Here are some:
Geographical and political factors:
Country-specific analyses
Differentiation between types of political events (e.g., elections vs. referendums)
Explore correlations with other socioeconomic indicators and the Democracy Index
Expand other voter behavior metrics
Exposure to AI-driven campaigns
Engagement rates with propaganda
Proportion of undecided voters
Fine-tune event-specific thresholds, e.g., determine tipping points for influencing elections or referendums
Assess consequences for various societal groups
Temporal considerations:
AI development trajectory and its impact on influence effectiveness
Changes in the frequency of algorithmic campaigns over time
Consider demographic and spatial factors, like adjusting influence estimates based on population segments and geographical distribution.
Incorporate Bayesian updating to refine estimates over time
Explore Extreme Value Theory (EVT) for modeling rare, high-impact events
So What? Practical Applications of Risk Quantification
The quantification of AI risk opens up new avenues for analysis and decision-making. Let's explore the practical implications and potential applications of this approach.
By assigning numerical values to AI risks, we can begin to assess their acceptability. For instance, my first naive analysis suggests that in worst-case scenarios, the degradation of democratic institutions due to AI-powered propaganda could result in a productivity loss of up to $20,000 per capita annually. On average, over a long-term horizon, we estimate GDP per capita fluctuations between $0 and $7,500 in 50% of scenarios. This raises important questions: What level of risk is acceptable to society? How do we balance potential economic impacts against less tangible effects on democratic processes?
This quantitative approach enables us to compare and rank different AI risks. For example:
We estimate that the risk of AI-powered propaganda affecting democratic processes occurs on average four times yearly.
In contrast, the risk of AI systems being compromised for malicious purposes in customer service contexts may occur ten times yearly.
Such comparisons allow for more informed prioritization of risk mitigation efforts.
Imagine you can also compare quantities like these for many other AI risks.
Some factors, like democracy's intrinsic value, may not be easily quantifiable in economic terms. In such cases, we can compare risks by focusing on other measurable aspects, such as frequency of occurrence.
In summary, by continuously refining this framework, we can develop a more nuanced and accurate understanding of AI risks, enabling more effective governance and mitigation strategies:
Prioritization. By quantifying various AI risks, we can rank and compare them objectively.
Policy guidance. Policymakers can use these insights to craft more effective regulations.
Business strategy. Companies can make informed decisions about AI adoption and risk mitigation.
Public understanding. Citizens can better interpret AI-related news and form educated opinions.
Crowdsourcing AI Risk Intelligence
I'm open-sourcing this AI risk database and inviting contributions from experts and concerned citizens alike. Together, we can:
Incorporate new data and insights as they emerge
Source experts who can give calibrated ranges
Defining risks
Refine risk models for different countries and scenarios
By collaborating, we can reduce uncertainty and build a world where we confidently harness AI's benefits while managing risks.
New observations and insight
It is notorious that uncertainty drops significantly with a handful of new observations. Let me give you an example drawn from the great book of Hubbard. Without reading further, think of your 90% confidence interval (CI) for the average jelly bean's weight in grams. You need to state a lower bound and an upper bound. You’re 90% confident that the exact weight will fall within those two boundaries. According to your experience and knowledge, you may come up with a narrow or wide range. It doesn’t matter.
If I give you some information, like one cubic centimeter of water (imagine a thimble full of water) is 1 gram, can you narrow your CI down?
When I did the exercise myself for the first time, my initial estimate was between 0.5 grams and 20 grams. What was yours?
Taking it further, directly from the book:
“Now consider the following four questions. Answer each one before you go to the next point.
Suppose I told you the weight of the first jelly bean I sampled was 1.4 grams. Does that change your 90% CI? If so, what is your updated 90% CI? Write down your new range before proceeding.
Now I reveal that the next sample weighed 1.5 grams. Does that change your 90% CI again? If so, what is your CI now? Write down this new range.
Now I give you the results of the next three randomly sampled jelly bean weights, for a total of 5 so far: 1.4, 1.6, and 1.1. Does that change your 90% CI even further? If so, what is your 90% CI now? Again, write down this new range.
Finally, I give you the results of the next three randomly sampled weights of jelly beans, for a total of eight samples so far: 15, 0.9, 1.7. Again, does that change your 90% CI? If so, what is it now? Write down this final range.
Your range usually should have gotten at least a little narrower each time you were given more data.”
I expect your narrowing down was very steep based on the first question. For instance, when I did the exercise, my CI went down from 0.5–20 grams to 0.5–3 grams: a steep reduction in uncertainty! By the time I reached question 4., my estimate was between 1.1 and 1.8 grams, which is not so far from the experimental values of 1.21 to 1.57 grams, with the population average at 1.45 grams.
Calibrated experts
You would be surprised to learn how close a bunch of calibrated experts can give confidence interval estimations that are very close to the true ones! Experts already have a good grasp of their areas of expertise, and an exercise of 1-2 hours helps them to calibrate their estimates. What does it mean to calibrate? Experts or not, we all tend to over or under-estimate things. An exercise that aids our thinking of economic incentives of giving the best estimate and completing a few estimations dramatically improves our calibration to avoid too much over or underestimation.
So, the plan would be to invite experts (5 are enough to reduce uncertainty considerably) for each quantity we want to measure that does not have enough coverage from more extensive studies or data sources. For instance, in my experience of estimating the chance of AI-driven propaganda to influence an election outcome successfully, I only found a few studies and highly uncertain outcomes. So, experts in marketing, political campaigns, and social and behavioral sciences may contribute significantly to quantifying our uncertainty on the chances that a well-targeted, personalized campaign will change someone’s mind.
Defining risks
Help define risks. For instance, I defined the first one I am working on, starting from the MIT Riks Repository and singling out the elements that make one event unique.
Refine risk models
“All models are wrong. Some are useful.” – George Box
All models are wrong, but they help us reason about uncertainty and possibly reduce it. Or, at least, they let us know how broad that uncertainty is.
The landscape of AI risk modeling is as diverse as it is complex, with each facet potentially demanding its unique analytical approach. We stand at the frontier of a new paradigm in risk assessment, where conventional methods may prove insufficient. This is where your expertise becomes invaluable. What novel frameworks might we employ to capture the nuances of various AI risks? How can we push the boundaries of current methodologies to grasp these emerging challenges better?
As a few collaborators and I will refine our approach in the coming weeks, we invite you to join this intellectual endeavor. Your insights on risk classification, component modeling, and innovative assessment techniques could be the key to unlocking a more comprehensive understanding of AI risk. We encourage you to reach out and contribute to this evolving dialogue, helping to shape the future of AI risk analysis.
Embracing Rational Optimism
Risk is both a downside and an upside. While I usually get eye-rolling gazes when I speak about AI risks and safety, this is not for igniting fears. Rather the opposite, it’s about the opportunities solving these problems open up! With a clearer understanding of the risks we face, we can make informed decisions, develop targeted solutions, and shape a future where AI serves humanity's best interests.
Together, we can transform AI risk from an abstract threat into a manageable challenge, paving the way for responsible innovation and a brighter technological future.
[Edit April 2025: This project has long been deprecated because highly qualified teams are working on this and progressing well.]
https://www.aiaaic.org/; https://oecd.ai/en/incidents; https://airisk.mit.edu/.
Hubbard, Douglas W. How to measure anything: Finding the value of intangibles in business. John Wiley & Sons, 2014.
Many analyses try to establish a relationship between democracy and GDP. A recent one is Pelke, Lars. "Reanalysing the link between democracy and economic development." International Area Studies Review 26, no. 4 (2023): 361-383. I also downloaded data from this source: https://ourworldindata.org/grapher/gdp-per-capita-vs-electoral-democracy-index to base my numerical assumptions.
On top of the sources already cited for quantification analysis, I’d add Gorrell, Genevieve, Mehmet E. Bakir, Ian Roberts, Mark A. Greenwood, Benedetta Iavarone, and Kalina Bontcheva. "Partisanship, propaganda and post-truth politics: Quantifying impact in online debate." arXiv preprint arXiv:1902.01752 (2019).