As the competition between massive artificial intelligence companies heats up, the repository Hugging Face has emerged as a rare point of neutral ground. Built as a GitHub-style clearinghouse for open-source data sets and models, the site has become a vital resource for anyone working in AI. Without the regulatory baggage of a giant like Meta or Google, Hugging Face has also become a voice of reason in the policy world, advising regulators around the world on the unique promise and risk of AI, while leading its own technical work on bias assessment and watermarking.
The company’s head of global policy, Irene Solaiman, is at the center of that work. A former public policy manager at OpenAI, Solaiman was the first person to test ChatGPT for social-impact bias. Now, her team at Hugging Face is advising regulators from the U.S. to the European Union on how best to approach the nascent AI industry, and how to navigate thorny issues of bias, consent, and existential risk along the way.
Rest of World spoke with Solaiman about the promise and risk of the new generation of large language models — and how to build them with the rest of the world in mind.
This interview has been edited for length and clarity.
I want to start off with the major AI scandal of the month, which is the lawsuit between Scarlett Johansson and OpenAI. It seems like an alarming case for AI policy professionals. Is there anything that surprised you about the case?
On the legal side, the biggest implication I see is establishing precedents for future action. There’s this question of how much Scarlett Johansson has a right to likeness as a voice actor. But no case explicitly like this has existed, so we’re going to find out as things shake out what we can expect in the future.
At the same time, the whole case brings to light the emotional connections people are having with AI. There’s a lot of AI girlfriends out there. Particularly girlfriends and not boyfriends. I’m not a psychologist but I do think that has serious implications for how we engage with AI in a loneliness epidemic.
You’ve said before that you actually had your voice cloned without your consent, as part of a product demo that backfired. Do you think there’s a bigger problem with how the tech world thinks about consent?
Part of it is just the tools: We’re way behind where we need to be in being able to sort through these massive amounts of training data. And that comes out when you think about consent, but also accuracy.
So when I had my voice cloned, the platform does say that you need to get consent of the data subject. But the enforcement is just a little checkbox that says, “I received consent.” And sometimes it’s ambiguous who owns the video or is in a position to consent. In my case, the training data was from a public video of a talk I’d given — actually, a talk about the importance of getting consent from data subjects.
Bias has emerged as one of the trickiest problems in AI development. On one side, we’ve reported on the stereotyping effect in image-generating models. On the other, Google ended up publicly apologizing for Gemini’s image generator after it introduced a range of genders and ethnicities into queries where that diversity didn’t make sense. Is there a way to handle this that makes sense? Or are companies just going to keep stumbling into political problems here?
So, one important part is moving beyond the models to the larger system. Image generators like Gemini are, by default, incredibly visual. And while I can’t say specifically how Gemini was built, I think it’s unlikely that people are directly prompting the model. There’s layers of systems that come with how people interface with a consumer-facing product. It’s really hard to find where biases are introduced. And probably the answer is, at every point. But this is part of why people are investing in evaluations, interpretability, and red-teaming.
When you look at the whole system, it also includes the data set. I’ve said for a while that we need to glamorize data-set research a whole lot more. It’s not my area, but Dr. Abeba Birhane has done some of the best work around looking at multimodal data sets.
Because you’re drawing from data that human beings created, a lot of it comes down to amplifying existing social norms. So, what proportion is representative? Are we overindexing on one specific population and its history and its infrastructure? In the end, you can never be fully unbiased because perspectives will differ, especially across the world, but even within one country, one city, or one family. What I view as unbiased might look very different from somebody with different political beliefs, from a different upbringing.
This is where AI is facing some similar parallels to social media. I remember discussing these issues with social media platforms 10 years ago. How do we treat existing norms? We don’t want to do social engineering, but if you’re only amplifying the existing norms, isn’t that still a kind of social engineering? Most of these problems are not specific to AI, but the way that we build, measure, and mitigate them look very different in an AI context.
The Gemini case was focused on image, but I imagine it’s even more complicated in text, which is where even more AI development is happening.
Right, the way that we react to consume, measure, and mitigate biases and stereotypes is quite different in image than what we would do in text. I did the first social-impact bias stereotype testing on OpenAI systems way back in the day. I did the first non-Latin character testing on OpenAI and GPT systems. I did it in Bangla [Bengali] because it was the only other language that I knew that didn’t use Latin characters. Which is to say, representation matters.
“There’s so much infrastructure that leads to more data being available on specific languages, even down to the keyboards that the internet was built for.”
There’s also the question of low-resource languages. We’ve seen models really struggle with basic operations in languages like Bengali and Tamil, simply because there isn’t enough online text to train on. How do you think about that kind of bias?
This is particularly close to my heart. I learned my heritage language of Bangla as an adult, and it is Sanskrit-derived. And I learned how difficult it is to start to understand the script. There’s 56 characters and they shift and it doesn’t translate well to a keyboard. The internet was originally created in the Western world and a lot of it was created for Latin characters, specifically English. There’s so much infrastructure that leads to more data being available on specific languages, even down to the keyboards that the internet was built for.
Something that I learned this year is that it’s actually more expensive by token to train on and generate non-English languages. You’re paying a higher price to process each unit of data, especially non-Latin character languages. But that’s getting a lot better as costs come down. OpenAI with their latest GPT-4o launch has shown huge reductions in the cost of tokenization for many Indic languages. Cohere Command R announced halving the cost of tokenization for many languages as well. Oftentimes people don’t report or even measure the financial cost. But, you know, money is a big deal.
We’ve also seen certain communities actively withhold training data. We reported specifically on a group of writers in Singapore who refused to make their work available, out of a concern that the resulting model would be used against them. Do you think actions like these are effective, or do they just widen the gap between high-resource and low-resource languages?
I think this is one of the hardest questions. I adore Singapore. I met my fiancé there. I think Singlish is a beautifully rich language, and it makes sense that the authors should have a right to opt out, because that is their work. But it gets a lot fuzzier when we get to the question of who actually represents the language.
We’ve seen this play out in some Indigenous communities. For example, a couple of years ago, a representative of the Maori community spoke out against training models on the language and selling it back to the Maori people. But shortly after, another group and the Maori community trained their own language model and preferred to have full ownership. So I think part of the question here is, who owns the language and who is able to benefit from it?
India is another interesting case. The Indian government is actually working on funding more Indic-language data sets for AI through their Bhashini program. And Hugging Face is working with them on an open-source Hinglish-language model. We’re doing more evaluations, so we launched an evaluation leaderboard in Arabic and Korean. And I think the signaling can help push people towards not just training, but measuring performance across languages.
