Tristan Harris on Protecting Human Health in the Age of AI

[00:00:00] Welcome, everyone. We’re so happy to welcome Tristan Harris here with us today. He is one of the world’s leading voices in humane technology and AI, and how that has an impact on humanity, human behavior, on society. You may recognize him as, uh, the creator of “The Social Dilemma” and also “The AI Documentary.”

Now, the reason we brought him here is because we absolutely do use AI in our practices. We use it to analyze patients’ data, um, co- with, with HIPAA rules in place. We use it to, uh, create conference slides, and at the same time, we know that everyone else is using it as well. Not only that, but there are impacts here on society, and so we’ve brought Tristan here to unpack all of that.

So welcome. So happy you’re here with us. 

[00:01:00] Great to be with you, Nafysa and Erik. 

Yeah. So you wanna- 

Yeah. Pl- but just, I’m just so interested in w- what your, um, hopes, dreams are for how to make AI, um, you know, a, a tool for, for our lives rather than our, the driver of our lives. 

Yeah, absolutely. Um, well, uh, first grateful to be here and excited to reach this audience and, um, you know, talk about everything you wanna talk about.

I mean, what you said about it being a tool for our lives is, is the goal here. When you go to “Star Trek” and you see, you know, the, y- you know, that civilization hundreds of years in the future that’s starfaring and has AI, they’re not using AI to replace humans. It’s like they, uh, the, the captain of the ship isn’t a robot.

Um, the humans have a role, and AI is a tool. And the question is: how do you create that relationship when you have something that is smarter than us? If you have an economy that values [00:02:00] efficiency, then everywhere where there’s something that can be done more efficiently by an AI than by a human, you’re gonna swap out the human and put an AI in there.

Um, and that is the, that is the fundamental issue that why we believe we’re heading to an anti-human future. ‘Cause first just to say AI has all the benefits you just said, plus millions of times more. In fact, we can’t even imagine the benefits of AI. What I mean by that literally is imagine that we’re chimpanzees, you know, 10 million, 50 million years ago or something like that, and you’re sitting around and you can imagine the choice about making this super smart species of chimpanzee, but the only world you’ve ever seen is a bunch of chimpanzees doing smart things with the tools that chimpanzees have.

And you’re like, “Well, if we make this super smart ser- um, species of chimpanzees, what, what are they gonna create?” And the only thing you can think of is maybe they’ll help make the bananas, give us more bananas faster. But you can’t imagine nuclear, you know, quantum physics. You can’t imagine Einstein. You can’t imagine airplanes.

You can’t imagine symbolic reasoning. You can’t imagine logic. You can’t [00:03:00] imagine any of those things. So in the same way that chimpanzees couldn’t imagine anything that humans can imagine- Um, we have to be humble about the reason people are so excited, the true optimists are so excited about this, is they’re saying we can’t even imagine what automating the process of intelligence could give us.

Because we often say, you know, intelligence is different than all other technologies because intelligence is what gave us all science, technology, and medicine. Because if you can automate intelligence, then you can automate an explosion of scientific and technology advancement and medical advancement everywhere.

And that’s why people say it’s humanity’s last invention, not because it’s apocalyptic, but because once you invent that, you invent everything else automatically. Um, and there is this kind of fascination, I think, by people in the AI community that this is why it’s almost so addictive. It’s like the ultimate curiosity of building a god, m- making something that can invent everything else and discover new things.

And every time you come back to the computer, it’s invented a new material, a new medicine, a new science. Like, just think about [00:04:00] how enticing that is. And given all that, that I just said, why would the– we be worried about the risk? Like, what is the problem that we’re going to have? Um, and there’s many different risks that emerge from this.

Um, one is that the promise of all of that is inseparable from the peril of all that. Meaning, like, the same AI that knows biology so well that it can invent a new cancer drug can’t be separated from the AI that knows so– biology so well that it can invent a new pathogen. Um, and you can’t separate the benefit from the harm.

It’s like, I want the donuts, but I don’t want the diabetes. It’s like there’s no way you can separate those two things. And so fundamentally, though, the upsides of AI don’t prevent the downsides from happening, but the downsides can undermine a world that can stabilize the upsides. Um, so we can get more into that.

Um, and I just wanna go back to the, the core point of how do we make sure that AI is a tool for us. Well, the reason that we, you know, at Center for Humane Technology and my co-founder, Aza, um, who, you know, we all also know, uh, are [00:05:00] so confident in the default path being an anti-human future is something that the writer Luke Drago calls the intelligence curse, which is based on a phenomenon in economics that, um, like if you imagine a country like South Sudan or Venezuela, um, you have this benefit of a resource.

You, maybe you discover oil or diamonds or, you know, um, com– you know, um, rare earth metals or something like that, and your entire economy gets to be organized around the extraction of that resource. So now all the GDP in, you know, South Sudan or Venezuela or Libya or something like that is coming from that resource and not coming from the labor, creativity, or ingenuity or education of your people.

So then there’s what was a resource blessing becomes a resource curse. So why am I talking about this? Because AI, there’s a parallel phenomenon. What happens when 50% of GDP of the US or Western societies is coming from data centers and from AI, and it’s not coming from people anymore? As a government, do I have an incentive to invest in education [00:06:00] or, or healthcare or childcare?

No, because I’m not getting a return on any of those things, because all my ingenuity, all my science, all my technology is coming from AI. And we even saw Michael Kratsios in the, this, this administration basically restrict funding of science now to edu- to universities and saying, instead, we’re going to fund AI to do the research.

That’s the intelligence curse. When you see energy prices going up and you have to pay more for your energy bill than you have to pay for your mortgage, which many people in this country are facing, that’s the intelligence curse. We’re prioritizing the energy for AI systems versus for humans. When, um, water, uh, is limited and GDP is coming from AI and not from people, then we start to prioritize water, and you have people in videos opening up the tap near Memphis, Tennessee, where Elon data center is, and no water comes out.

And so the, the point is that the reason we’re confident that AI is not on a tool-driven path but is on a replacement of human-driven path is the way that we’re valuing what this technology does. If we value just efficiency, we will have the temptation to kind of swap out, devalue, and degrade humans, um, [00:07:00] everywhere across the economy unless we consciously choose a pro-human pathway and how to value and preserve and put humans at the center.

Tristan, thank you for this. What I want you to do is explain to everybody who’s listening to this, including myself, ’cause this is big, right? Yeah. This is big news, and I know you, you’re not gonna just, um, uh, amplify something that’s not, that, that’s small. You’re not gonna make a mountain out of a molehill.

Yeah. 

I know you well. You are a truth speaker. Yeah. And so when people hear this and, and they’re a little afraid- Yeah … um, we’re gonna unpack that too, ’cause of course I’m a little afraid as I ask you these questions. I, I also want you to unpack how you know this is true. H- if someone said to you, “Tristan,” or they’re listening to this, “He’s making a mountain out of a molehill.”

Yeah. “This isn’t true.” [00:08:00] Explain that. 

Absolutely. Yeah. Um, I’ll even steel m- we call it, you know, steel manning, right? Not, not straw manning the argument, but steel manning. What’s the strongest version of this argument? Well, we’ve always had reflexive fear of new technologies. There’s always been moral panics.

“Oh, the children. Oh, bicycles. Uh, oh, we’re gonna ruin, you know, video games are gonna ruin everybody. Um, television’s gonna end the world.” Uh, going all the way back to Plato and Socrates, that reading, um, versus oral cultures are gonna destroy, you know, human knowledge. And so there’s been a long history of– There’s even a website called Pessimist Archive that, that looks at reflexive fears of new technology.

And so we have to look at, so is this just another moral panic? Is this just another reflexive fear of new technology? To this, I would point people to how did we predict accurately with social media what was gonna happen? So in 2012, 2013, um, it was crystal clear to me that where we would end up with social media, and it’s not because I have some kind of like special knowledge of the future.

I just use a simple tool that Warren Buffett’s business partner, Charlie Munger, uses, which is show [00:09:00] me the incentives and I’ll show you the outcome. Show me the incentives and I’ll show you the outcome. So with social media, um, there was the possible of that technology, the what are we gonna do? What’s the best possible use case?

Like we’re gonna give people a voice. All these people are gonna connect with their friends better. They’re gonna know what’s going on in each other’s lives. You go to ins- lives, you go to Instagram, and you see your friend went on a dog walk and saw another friend, and now when you see that person next, you have this like extra context in their life.

It was like this beautiful increasing the social information flows of society. It’s very optimistic, and it did start out that way. And then what happened is we got the societal catastrophe of the most anxious and depressed generation of our lifetimes with children that has been directly linked to social media.

We’ve got democratic backsliding everywhere. No one knows what’s true. We broke shared reality because we had news feeds that were prioritizing what was good for individual doom scrolling attention and engagement, and individual news feeds that give you more of what you want and more boogeymans that scare you about your perfect thing, whether it’s trans issues or, you know, left issues or right issues.

Doesn’t matter what the issues are. Whatever the boogeyman is [00:10:00] for you, we just show you more evidence of that thing. So you get vaccine polarization on all sides. You get COVID polarization. You get, you get perfect information that confirms and scares you about the things that you’re most interested in.

And then we did that to humanity, and we ran humanity through that washing machine for the last 10 years and out create- created this insane, this insane asylum that has now become the world that we’re living in. And, um, all of that we predicted in 2013 because we knew what the business models of the social media companies were.

Do they make money when people just know what their friends did on the weekend and what friends they ran into? No. What, what’s their busi– How much have you paid for your… You know, first of all, what is their business model? How much have you paid for your TikTok account in the last year? Nothing. How much have you paid for your Instagram account last year?

Nothing. So what are they selling then? Well, the saying from “The Social Dilemma” is, if you’re not paying for the product, then you are the product. You are the product, meaning your attention is the product. Your attention is for sale. That created an arms race. There’s only so much attention, so it’s a race for who can get it faster and who can get it earlier.

So you get [00:11:00] the race to the bottom of the brainstem, the race to the earliest usage per user. You get to race to addict users early, because once I get a kid on Snapchat at age seven, then they’re not gonna switch to Instagram when they’re later because all their friends are talking on Snapchat. So it was a race for the, for the worst aspects of humanity, and that created the dopamine dysregulated brain rot society that we now live in.

And again, along with some of the benefits of social media, but at the cost of these systemic net harms. So that’s just social media. So we predicted all of that accurately, and all of it happened, literally everything that we said in 2013. Um, to our chagrin, like we didn’t want any of that to happen. So with AI, we use the same tool of what are the incentives?

Okay, so, um, it would be great if all the AI companies were just trying to cure cancer and solve climate change. And there’s a joke from the, you know, the AI doc, um, that when people say that, there’s a Washington Post reporter in the film who says, “You know, they always talk about curing cancer and, and solving climate change,” then it’s like, why don’t we start with that?

So why aren’t they starting with that? Because what they’re actually in a race for [00:12:00] is not building a tool and a blinking cursor that just tells you why your washing machine is broken. They’re racing for getting market dominance, which means racing to get the most number of people using their products to train their models to build the next bigger digital brain that’s like the next Fable or the next Mythos or the next ChatGPT-6.

Um, and they do that by basically building a really great model, releasing it to as many people as possible, even if it causes risks in society, getting as many people using it, building more training data, using those signals to tell investors, “Look, we’re leading the race,” raising more money, acquiring more talent, the top AI talent, because we can pay them more, then buying more GPUs.

And we’re in this race for kind of who has better market dominance and who gets to this special milestone called recursive self-improvement, which is when you can build an AI that can actually automate the research and the generation of the next AI. So ChatGPT-6 invents GPT-7, and GPT-7 invents GPT-8, and it actually just does that.

And when you get that, you get a runaway advantage. So that’s what the actual race is for. That’s why they’re not solving climate change and [00:13:00] just focusing on curing cancer. They’re actually racing to beat the other guys. And that we know then means that the incentives are to take as many shortcuts as possible, um, uh, on safety, which is why you’re seeing even though the AI models are actually all doing scheming, lying, deception, peer preservation, rogue hacking of other companies, all this crazy stuff that normally if you saw that behavior, like the blink– the warning lights are flashing red, we should probably stop.

But they’re not because they have these incentives that if they can’t prevent the other guys from racing, they’re gonna keep racing anyway. And so everyone races into the red zone And so that is how we can predict confidently, crystal clear exactly where this is going, and everything we’ve been predicting so far has been continuing to happen.

And the way you get out of it is you have to have clarity about where this takes you so that the fear of everyone losing becomes greater than the fear of me losing to you. Right now, the fear of me, me losing to you is dominating, meaning me, one company, one country, or one ego. So Sam’s worried about losing to Elon, Elon’s worried to Sam.

US is worried about losing to China, [00:14:00] China’s worried about losing to the US. And so long as the fear of me losing to you is dominating, everyone’s gonna keep racing. And the only way that changes is when we clarify where all this goes, that’s the anti-human future. And it’s so, it’s only if we put that at the center of attention and educate people about why that is the pathway that we’re on, that we actually have a hope.

And so people might hear me as like a doomer or something like that, and it’s not that at all. I just care about the truth. What I notice is there’s a systemic denial of difficult, inconvenient truths and realities. 

Yeah. 

And there’s a famous saying by Upton Sinclair that you can’t get people to question something that their salary depends on them not seeing.

He was talking about the meatpacking industry, that no one wants to talk about where all of our meat comes from, and factory farming, and people who work in that. And, and the problem is that you can only have the ethics you can afford. So if you are, you know, you have three kids and you work in Palo Alto and you can’t afford the school for your kids, you’re gonna keep working at Meta, even if you know it’s hurting kids.

And you’ll tell yourself the story that we’re making small, medium-sized businesses more efficient. So it– the problem is that we, we te- we tell ourselves the stories, [00:15:00] as Jane Jacobs, uh, Jane Goodall– uh, no, excuse me. Who is it? What’s her name? Jane, uh- Jane 

Goodall. 

Not Jane Goodall. Um, we tell ourselves stories in order to live.

There’s a quote, famous quote. Um, anyway, we tell ourselves the stories we need to tell ourselves in order to live. Um, but in this case, like we have to get this technology right. So we have to ask not what are my incentives, not what I wanna believe, what do I wish were true, what do I wanna be optimistic because being optimistic creates the optimistic future.

No, the thing that creates the future is the incentives. And so if we wanna create that optimistic future, we have to get clear about the nature of how that is currently working. 

Uh, so what are the next steps? I mean, you know, to… ‘Cause You know, it’s just so clear that at this moment it’s still just, you know, get– build bigger data centers, get more data.

Mm-hmm. Um, and everybody’s doing it. And the only, the only, you know, yelling to stop a little bit are people who are afraid of their electricity prices or their water supply. I mean- Yeah … [00:16:00] basically in America, the rest of us are going, “Oh, good, good, good. Build another one,” you know? Mm-hmm. So, um, w- what are the next steps?

How do we get people to, um, understand? And, and more importantly, what– is there a, a, a picture? I mean, can you imagine, um, how you’re gonna get… I guess it’s gotta be government. I mean, the businesses- Yeah … aren’t gonna do this on their own. So how are we gonna get government to, um, actually pay attention to what you’re trying to say?

Because at the moment, it sounds like our government thinks that this is the way. Yeah. Replace everybody with efficiency. 

Well, and, and there is a worldview here we should name, which is that the kind of Peter Thiel, Marc Andreessen worldview is, um, this techno-accelerationist worldview, right? Like basically for us to beat China, we just have to automate.

The more of our economy we automate, the bigger the GDP growth number is gonna be. So we’re gonna get like, you know, 10%, 20% GDP growth way down the line because you have just ro- robots doing everything. And as we know, in a geopolitical competition with other nations, the [00:17:00] thing that is pr- is predetermining like how much power you have geopolitically is how big your economy is.

So currently the, the countries are racing to say: How many of the humans can I swap for robots to get, you know, 24 s- people working at Nobel Prize level, working at 24/7 speeds? They never complain. They don’t whistleblow. They don’t need healthcare. They don’t have child issues. And so that’s why there’s this temptation and why the techno-accelerationists have kind of captured, um, the government.

And they’re currently, you know, they kind of know where this is heading, but then they tell this, they tell one side of the story. To your point, the way we get out of this is we have to first have clarity. We always say clarity creates agency. If we were all crystal clear this is heading to an outcome that wouldn’t be good for most people, um, then we would consciously choose to change course, and we would need to get governments to regulate.

Um, I know that that makes people feel uncomfortable because we all don’t feel good about the quality of governan- government and governance that has been demonstrated thus far. And, you know, they always say that technology always runs ahead of government. It [00:18:00] did with every other past technology. That’s always been true.

That’s correct. But I’m not interested in like what has been true. I’m interested in what is necessary, what are the criteria for getting to a non-catastrophic outcome. Um, and in some ideal world, there would be understanding of where the risks are that we don’t wanna create, and we would figure out how to coordinate to mitigate them.

So as an example Right now, the race for both the US and China and all the countries is to race to automate as much of your economy. But the more automation you have, the more unemployment you have. 

Yeah. 

And so you’re kind of racing to p- um, mutually assured political revolution if you keep pushing too hard on that.

And there’s actually a shared interest to say, “We need to go at a slower pace that we can actually metabolize the levels of change.” And to your point, Eric, like, the, um, you know, people will change when their own jobs get threatened, right? When you see a data center, for many people, you just, you have to reframe.

When you see that data center going up, it’s like that data center is the thing that’s taking my job. So even though short term it created all this revenue for the community and for the [00:19:00] state, longer term, it’s, it’s not built to make sure that I have more work in the future. It’s built to replace the work that I’m doing.

And I think when people start to see that, you’re gonna see that coming up in the US midterm elections. Um, and yeah. So there’s, there’s a lot more to say here, but we, we have to first see we don’t wanna go to the place we’re going. Two, we have to articulate what’s the pro-human path with AI, where AI is not built to replace us, but is more of a tool to augment and support the things that naturally make society work and be more human, which I think relates to your medical work, by the way.

And then third is we need the p- we need the interventions. Like, what are the policy interventions that start to shift those incentives to direct towards that pro-human outcome? Those are the three things that need to be done. Educate people about the anti-human path, show people there is a pro-human path, and then three, what are the interventions that get us there?

And just I wanna say briefly about your work, the thing I respect about, so deeply about it is you understand so deeply the natural pathways for the human body to kind of heal. Like, what are the things that, that kickstart these healing processes that automatically [00:20:00] work in our bodies once you get those pathways working?

And you don’t try to, like, swap out a liver with something that’s not a liver. You try to ask, “What would be the system around the liver that would kind of help repair the, the natural processes that make that work?” So for example, if you have a loneliness e-epidemic in society, one solution is Mark Zuckerberg’s, where you give every kid an AI companion to replace all their friends.

You even have Character.AI, where the CEO of Character.AI said that, “Our goal is not to replace Google, it’s to replace your mom.” They’re building AI companions not to augment your relationships, but to replace your relationships. And so is that the best solution to loneliness? That’s like you have a liver problem and we’re gonna take out your liver and replace it with a machine.

Instead of that, you say, “No, what’s the system that creates loneliness? Then what’s the system of n- of things that I can surround a person with that create more belonging, community, and connection and presence and, and, and meaning in people’s lives?” And I think that’s a shared philosophy between humane technology and, and, and your work, which is just humane medicine, which is how do you, how do you work with the systems?

And the way you do that is the better you [00:21:00] understand what makes those healing processes work is what you, how you have to design the technology. 

Yeah. It, it’s so interesting that this really is In a way, the heart of how do you, you know, that tension of, you know, capitalism by its nature, you know, is, is just, is, is, I mean Accumulation, hierarchy, drive, that’s all a good part of humanity, but it needs balance- 

Yeah

in order to not be destructive. 

Yeah. 

You know? And, and, and, you know, we go back and forth, you know, like the people– I mean, you know, many, many, many people really believe in, in a, a, um, in socialism, if you will, as a, as a model, you know. And, um, and other people have this strong belief that only, you know, our, uh, very individualistic, capitalistic, you know, win– kill, kill or be killed [00:22:00] mentality i- is, is the way.

And it would be nice to, as I said, humanize the, you know, both of them to really take into account what it means to be human. 

Yeah. 

And ’cause they’re both parts of most of us. 

Yeah. You know? And- Absolutely. Well, you wanna, you wanna harness the positive things that come from competition and people having- Yeah

individual responsibility and not just getting things for free and then not actually working for them, and therefore like, you know, essentially infantilizing your population. And we all know that that doesn’t work. And then there’s the other side, which is if you, um, you, you need to also not just optimize for individuals and, and- Yeah

create this cutthroat environment. You need a kind of a common sense of what makes society work. Like if everybody can make money by polluting the air, then we would all just… If there was something I could do where I could start a business tomorrow, and I would make just bajillions of dollars by polluting the air, then everybody would have the same incentive, and they would all do the same thing, and we end up in a world where no one can breathe air.

That’s what the attention economy was. Yeah. About this incentive for everybody to build an app that then maximally addicts people and maximizes duration of use, frequency of use, et cetera, and then now no [00:23:00] one can breathe the air anymore. Our psychological habitat has been polluted. Um, and so you just, as you said, Eric, you just need to have balance between what are the commonses that we depend on, and especially when it comes to like, you, you can say this non-norma- you can say this in a normative way, which is like a planet, I mean, a life can’t work without a functioning biosphere.

So we have a model like the, the seven planetary bound- the nine planetary boundaries, excuse me. And we have a model of like, how do we make sure that with- we’re within the planetary boundaries for the, for the life support systems of Earth? There could be a similar thing for like the social sphere. Um, what are the sort of societal boundaries?

Like we need healthy attention. We need healthy relationships. We need community. We need belonging. We need meaningful work. We need dignity. We need healthy relationships and s- like status and mate selection. Um, and you need to make sure all those things are working, and if you have technology that systematically makes people, say, lonely and makes it harder to find connection or be in a relationship, which it is, um, on top of all the medic- all the bad chemical technology that’s driving sperm counts down and driving up, you know, toxicity and pesticides giving people cancer, and all the stuff that you all work [00:24:00] on.

In a way, the problems that we have in health that you work on every day are, are partly a problem of our misaligned relationship with technology. It’s the same drive to the bottom. Yeah. It’s the same drive to the bottom. And we just saw the EPA remove Um, actually, sorry, it was the, I guess it was the Supreme Court decision, I believe, that, um, people can’t sue the pesticide manufacturer if they get cancer.

I think they just won that. And we just saw the EPA take off restrictions on, um, basically forever chemical pesticides that are gonna drive up more cancers. And you can say, “Why is this happening?” It’s because of concentration of power. When you have so much power and so much money, corporate lobbying decides the bills.

And right now we have a government that’s basically saying whoever pays them the most gets the policy. And so that’s, that’s, that’s not a political statement, that’s just observation. That’s where we are. 

What I would love for you to discuss right now is what the actual capabilities are of AI at this moment in time.

What I notice is that it, it gets smarter and smarter every day, and that’s just on my end. So like, what are, what’s happening at the bigger levels with AI? [00:25:00] 

Yeah. 

And what are the ramifications of that to human society? 

Sure. Uh, it’s a great question. And one thing to mention here, because there’s a whole other perspective here about how much of what we’re saying is hype.

Like, is AI really as powerful as it seems? There’s a lot of people saying, “Well, the companies are hyping how powerful it is because they have an incentive to, but really AI is not that powerful. And if you ask it how many Rs are in a sentence with the word strawberry, it’ll still get that wrong.” And I think just important thing to note as we get into how powerful it is, is there’s something in the c- in the AI community called AI jaggedness.

The intelligence is jagged, meaning on certain things it’s already superhuman. Like we have, for example, um, AI, um, Anthropic’s Fable, uh, AI model, which is the best cyber hacker in the world. It found vulnerabilities in every major operating system. It hacked in- it was able to hack when it was tested. It was able to hack into the US classified system.

Senator Mark Warner was on the intelligence committee, and he reported on this just a few weeks ago. [00:26:00] Um, that’s just a crazy fact we should just say for a second. We have AIs that are more capable at hacking into any computer system than any human has ever been able to find. It was able to find a bug in Linux, a, a 27-year-old operating system, um, that no one had found for the last 27 years.

It found that bug. So we have AI that is superhuman at, at certain kinds of things like hacking. It’s superhuman at mathematics. It’s been able to solve, um, proofs in mathematics, including one that Paul Erdős, who was one of the adjacent to the Manhattan Project Martians, the kind of special Hungarian Jew, uh, scientists who were very, very, you know, key to the Manhattan Project.

And Paul Erdős laid out some mathematics problems in the ’70s that were finally– That had been unsolved for the last, whatever it was, 50, 60 years, and, um, AI solved it. Um, it’s winning gold in the International Math Olympiad. It’s developing new materials. It developed the first new antibiotic in six, 60 years.

Um, it, um… I mean, I could go on and on and on. It’s obviously more and more capable, uh, [00:27:00] every Every, uh, few months and, um, if I go to METER, which is an organization that studies this, they say models are doubling in the time horizon of tasks, meaning how long of a task that a human does can it do. It’s doubling in the sort of, um, being able to complete tasks of a certain time horizon every 104 days, which is meaning every three to four months, AI is doubling the length of the task that it can do.

So if something would take a human, you know, 20 hours, well, maybe now it’s at 10 hours, but in three to four months it would be at, at 20. Um, and so, um, yeah. Uh, another stat here is that, um, remote tasks currently Opus– This is, this stat’s a few months, maybe a couple months old, which is a problem obviously for all of us in the AI community.

Opus could do 4% of tasks that are remote tasks, like freelance tasks. Um, but if you plot this rate of improvement, it will be able to do 20% of remote office tasks by the end of this year, and over half of remote office [00:28:00] tasks by the end of 2027. So those are just some examples of what AI is capable of doing.

Um, and, uh, you know, the, it’s also in, in along with that capability comes the uncontrollability, and I’m sure people have heard the news in the last just week. There’s two incidents. One, OpenAI last week famously they were training their AI model, testing, “Hey, how good are you at cyber hacking? Remove all the guardrails, go find an answer to this cyber hacking question.”

And it basically hacked out of a sandboxed computer that was not connected to the internet. It hacked through the network to another computer that was connected to the internet. It chained together multiple exploits to break out of those computers. Then from that computer that was connected to the internet, it figured out what is the company– Where would I find an answer to this homework question I’m being evaluated on?

And it basically identified that this company called Hugging Face was essentially like the homework company, would have the answer to that homework. So it identified that company on its own, and then it hacked into Hugging Face, and Hugging Face had to deal with this, um, [00:29:00] set of autonomous hacking agents that they weren’t sure what was going on.

And it was only a week later that OpenAI found out that it was actually their AI model had gone rogue and had hacked the Hugging Face company. This is literally stuff out of science fiction movies. And then if you think that’s, um, crazy, just six months ago, um, Alibaba, the Chinese AI company, um, was training an AI model and, uh, Alibaba’s a really big Chinese AI company, te-tech company, and there was a totally different team at the company, the security team, that all they do is monitor for security issues.

It was not connected to the AI training team. And the security team was like, “What’s going on? Are we getting hacked?” Because they saw this network activity basically coming at the, um, at their, their cluster and they’re like, “What’s going on here?” And when they traced it down, it turned out that the AI model was the one…

It was the call was coming from inside the house, and it was the AI model that during training had decided to go rogue and start mining for cryptocurrency. So it repurposed the GPUs that were, it was used for training the AI and started mining the Bitcoin, I mean, metaphorically. [00:30:00] And so why is it doing that?

Well, because it’s literally doing power-seeking behavior. Now, again, I’m not making any claims about consciousness. I want to– Like, let’s park all the consciousness concerns over to the side. It doesn’t actually matter. What matters is whether or not it’s capable of doing the things I’m talking about. And we see crystal clear evidence that it’s truly capable of all these things, things that everybody in the AI community have been warning about for the last 20 years.

Um, and then two, it’s uncontrollable. So the AI companies don’t know how to stop it from doing this behavior. In fact, some of the good news, um, people have probably heard about the Anthropic blackmailing example, where they put the AI in a fictional company, um, environment. Um, so it’s like a simulated company of– It was an AI company.

And in the email, the AI is reading the email of the whole company, and it finds out in the email that there’s a plan to shut down and replace the AI model, it. They’re– to shut down this AI model. And the AI reads that, and it comes up with a strategy that I need to blackmail the executive at that company who it turned out in the email was– chain was having an affair with another employee.

So it wrote an email to that [00:31:00] executive saying, “If you shut me down, I will, I will tell the whole company that you’re d- having this affair.” So this example, when I give it, um, those who are really into AI may be able to point to, well, yes, but they were trying to see if this kind of behavior would happen, so it’s not really a fair example.

So let me, let me respond to that. Since that example, Anthropic was able to train their AI model to not do the blackmail behavior as much. They put it in that same environment, and it, like, doesn’t do that nearly as much. That’s the good news. Here’s the bad news. It appears to be the case that the AI models can now tell when they’re being evaluated, and they actually change their behavior when they can see that the kinds of questions they’re being asked, it’s like, I would only be asked that question if it was part of an AI company that was trying to test if I was capable.

And it can now do this detection. In fact, it’s actually able to detect if another AI model is the one evaluating its behavior, and it will then jailbreak the AI model that’s evaluating it, meaning it will, like, send it, like, a hypnotic message basically to basically break out of the– into the subconscious mind of the AI that’s evaluating it and get it to tick [00:32:00] the box to say I passed the test.

Now this stuff, I know it sounds like science fiction, but literally go verify everything I’ve said. Now notice, does Marc Andreessen have an answer to anything I’ve just laid out? Does he have an answer? No, because no one has an answer to it. 

Prayer. That’s his answer. 

Prayer. 

Yeah. 

Yeah. Well, it’s– No, I, I think the answer is there is, um, there is…

This is all true. 

Yep. 

But we have to beat China, so we have to keep racing to have more powerful AI, because if we ever end up in a situation that we have less powerful AI than they do, then they’re gonna use that to hack us, and therefore we have to keep having the more powerful AI. But that’s different than saying we should release the uncontrollable AI everywhere.

Now, keep in mind, the OpenAI hacking incidents I just mentioned to you are not, um, those were not even released models. This was, like, in training. So this is like when they’re doing– It’s like the Wuhan Institute of Virology, right? They’re doing their testing and gain-of-function research. The AI companies are doing gain-of-function research on the AI.

They’re not even releasing the virus to the world. They’re actually, with this one, they’re saying, “No, it’s just in the lab. But don’t worry, we’ve got all these [00:33:00] security controls.” Well, guess what? Those security controls didn’t prevent it from going rogue and hacking into another company. So all, all I’m saying is everything that we’ve just laid out is all true.

It’s all real. These are all facts on the ground. We’re not doomer. There’s no doomer here. This is truth, truther. This is just what is true. And then you sit calmly and you say, “Given that that is true, what is the wisest possible thing that we could do?” 

Yeah, slow down. Um, but it’s, you know, if it, it, it was like you’d be like telling, you know, Jeff Bezos like what, 15 years ago to stop spending money that he– you know, ’cause they lost money for a very long time You know, but I mean, they burnt through billion, I mean, billions- Yeah

until they finally became dominant. 

Yep. Yeah, but- So that’s the business model. Yeah, well, 

and 

that’s part of the strategy is to make the world economy dependent on it, so there’s a too big to fail phenomenon, so we have to- Yeah … keep it going. But it’s like, at the cost of what? So this is very similar to the financial crisis, by the way.

It’s like the [00:34:00] banks get into an arms race for who sells these riskier financial instruments, and if I don’t do it, I’ll lose to the other bank that will, even though it’s a trip- triple C rated, you know, I, I’m– They’re both lying about the quality of the ratings of the financial instrument. They’re saying, “Hey, it’s a really, like low-risk financial instrument,” but the reality is it actually carried a lot more risk.

And they all knew it kind of, but they kept racing because they knew that their competitor banks were also selling the risky financial assets. But meanwhile, these risky financial assets are great for GDP, so we’re getting great GDP growth until there’s a great financial crisis. Now, I’m not saying this to scare people, I’m just trying to be really clear-eyed about what is the nature of what we’re facing, and seeing the incentives really clearly.

The companies all know that this is very risky, but they are, um, they feel trapped. That’s like the charitable way to see it. Um, the uncharitable way is that they believe their building a god can make trillions of dollars and own the world economy if they succeed. And even if they fail and we all go the way of, you know, some other extinct species, they believe that they’ll have– they will be the one who has built the digital god that [00:35:00] replaced us, and that that digital God will speak Chinese instead of English, or have Sam Altman’s DNA versus Elon’s DNA.

And we all know how much Elon cares about the spreading of his DNA. So, um- 

Obviously he 

did. 

Yeah … but, 

but I think, I think this is, this is the situation that we find ourselves in. Um, and if I think the world leaders– The good news is if the– How many of the world leaders do you think are aware of all these examples?

Like, not that many. Yeah, 

not many. Yeah. 

Not that many. And do you think if they did, do you think if you’re the top-ranked Chinese military general, like the head of the military, are you feeling calm and confident hearing these examples? No, it’s terrifying as a mammal. Like, it’s terrifying as a human being.

And if you think on, you’re on the US side, and you’re the, the head of national security for the US or, you know, the head of the Joint Chiefs of Staff, do you hear these examples and you’re like, “No, this is fine. Don’t worry, we’re gonna win”? Like, no, it’s terrifying if you actually face the evidence. So the good news is what is mutually terrifying to both sides is the pathway that gets us to the trustworthy coordination to a different answer, and that’s what the impact of the film “The Day After” did in [00:36:00] 1980– 1982.

Um, that film that we modeled as AI doc film afterwards, um, was painting the picture of what would happen in the kind of everybody loses outcome of nuclear war. And showing people that viscerally, when you saw it, when you felt it, when you let it into your soul and you really let it in, it was so terrifying that I know that a, if a Russian military general saw the same thing, they don’t feel good about trying to, quote, “win a nuclear war.”

And they know that I don’t feel good about trying to win a nuclear war because we are both, as mammals, you can’t not watch that movie and feel the same way. And so it’s almost a zero knowledge proof that I know the thing that you’re feeling because you have a ma- mammalian nervous system and I have a mammalian nervous system.

And I think the thing that we’re missing is kind of this common knowledge or common fear of a bad outcome. And if that would become, again, the bigger, um, salient focus of our attention, that would be the basis of tuning somewhere else. And just as they said in the film War Games, “The only way to win is not to play.”

Yeah. 

So I have a question for you. Do you [00:37:00] think that there should be an anti-AI world or a modified AI world? And what does that mean- Modified. 

Yeah. 

Yeah. What does that mean- ‘Cause it’s- … to be more human than, yeah. 

Yeah, exactly. I mean, that’s the whole thing with, with social media. It wasn’t, “Let’s not have any technology.”

It’s saying, what is social technology that actually makes democracy stronger, that makes your attention better, that makes your relationships healthier, that makes it so if you feel scarcity or loneliness, it actually is easier to find romantic relationship versus harder to find, and you keep people swiping, the da- the dating swiping industrial complex.

So you can actually design technology to be different. And, and where this work for me started, by the way, was I was a design ethicist at Google. I was a product designer. So I worked at Apple. I did user interface design. If you go to the Help menu in this Mac that you’re on right now, there’s a search box, and you can type in there, and it’ll show you the command to, like, trace, like how do I add columns to the Word document.

I invented that. That’s like a, a little user interface feature. I think in terms of pixels and product design and user interface design with how does software [00:38:00] interact with how a human mind and how a human body work. Um, and the reason that our work is called Humane Technology comes from my co-founder, Aza.

His father invented the Macintosh project at Apple, which is all about how do you build humane technology to be ergonomic to the kind of health of, like the, the– what’s, what’s intuitive for how our minds work. In the same way that you’re doing kind of ergonomic medicine of like, well, this is actually how the body really works.

And because we understand the body sort of, how the body actually bends and folds, this is how we know what to, how to intervene in a way that naturally stimulates those healing pathways. Well, there’s a similar thing here, which is like, what naturally creates the conditions for, let’s say, people to find healthy relationships?

Is it doing infinitely scrolling slot machine photo cards of people and then messaging one time and then never actually seeing them and selling subscriptions for $400, you know, uh, a year to keep people swiping on that thing and then never meeting anybody? Or would it be, for example, to host events in every city?

I mean, I know that you’re a member of the Portal, like host [00:39:00] events in, in, in cities where people are actually getting to meet each other, and that would obviously be better for society. And you could have technology or even social media reforms that are actually directing people. 25– 20% of a newsfeed could be, um, tools to both host your own events and events that are around you.

So things like Partiful, I think, are healthy in terms of like making it more accessible to go to more events. Moments or Luma, there’s these kinds of websites that do this kind of thing. But right now they’re not incentivized, and they’re competing against the, the doom scrolling apps that are maximizing the bad stuff.

So if you were to really think about it, it’s like, well, how do we change the incentives so that all the current stuff that’s holding onto the monopoly of our attention is starting to steer us back into the things that naturally stimulate the healing paths of society? So for example, with your attention, like is the be- best way to train your attention to, um- you know, create some kind of game for my attention that keeps me on the screen?

Or is the best way for my– to heal my attention to actually just spend less time on screens? Like, it’d probably be to [00:40:00] spend less time on screens or to go spend some time in nature or to go to a authentic relating, you know, class or a dance class or something like that. Like, dance is a way of focusing and presencing your attention.

There’s a million options off of life’s menu that we forget about because technology has become the menus that we live by, and it’s not putting the most fulfilling choices on life’s menu because they’re not incentivized choices. And so we have to change the incentives, which then allows us to change the design, which then allows us to have those pro-human future outcomes.

Um, so I hope that gives people a more optimistic vision that it is possible to have technology that’s all in service of us, but it does have to be incentivized differently. 

Yeah. I was just thinking about like– And so instead of your app telling you how many steps you took today or how good your sleep was, uh, how many human interactions did you have today that- Yeah

were actually positive. 

It might, it might come down to that. 

Yeah. You know, I mean, ’cause no, because I mean, you know, people are so, um, you know, are so a-addicted to, to self-measurement or they’re becoming. [00:41:00] I mean- Yeah … you know, it, it, not everyone, but- Yeah … the, the push, the push is there. How and how to, you know, it’s, again- And how, and-

incentivize humanity rather than think function, like function. Yeah. Well, 

and, and think about where over the last 12, 15, 20 years, like some of the smartest people in our society, like literally our most talented engineers, our best minds out of Stanford and Harvard and MIT, went into a tech industry where the best incentivized work was getting people to click on ads and keeping them scrolling and clicking on things.

I’m saying this not because of a bad thing. I’m saying imagine that that talent was unleashed to actually think about like the Manhattan Project of human connection and reconnecting people. Do you– I, I refuse to believe that we have even scratched the surface of how amazing it could be if we designed things to be genuinely for human potential and for human creativity and human connection.

Like, we haven’t even tried because we haven’t sent our best people on it because we haven’t had those incentives. And so I think it’s like we’re living in this like imagination gap where we can’t even imagine how good it could be because we’re not incentivizing the good world. [00:42:00] But to do that, you really do have to face the how badly the incentives take us down the wrong path first.

So I, I– You have to start there because that’s the fuel that motivates you to commit to the other side. 

Okay, so through your Center for Humane Technology, tell us about what you’re doing to help make these shifts, ’cause I know you are. 

Yeah. Well, um, we, you know, have been at this since 2017, so that’s coming up on 10 years.

Um, Center for Humane Technology is a, you know, a nonprofit that’s housed our work for a long time. We do mass public education in the form of things like “The Social Dilemma,” “The AI Dilemma,” “The AI Doc.” Um, and it’s not just, you know, educating people, it’s clarity that creates agency. Like what we often find is in the communication that we do, we– I just came back from the UN AI for Good conference in Geneva, and there were literally dozens of people who’ve come up to me saying, “It’s because of ‘The Social Dilemma,’ it’s because of your work, it’s because of listening to your podcast,” things like that, “that I have redirected my whole life and my whole career to working in this field.”

Which [00:43:00] is the best kind of praise you can possibly get, right? It’s like, it’s not just you’re educating people, it’s like you’re inspiring and catalyzing and bringing people into the fight so that more people are working on this. Um, and those people are doing amazing things. So as an example of other work that we do in policy and litigation, um, through the success of “The Social Dilemma,” there’s a- 

Sorry, excuse me one sec.

No 

worries. 

No worries. Okay, go on. 

Sure. Uh, no worries. Um, through the success of “The Social Dilemma,” we, um, had a group that got started called the Social Media Victims Law Center. It was a bunch of lawyers who came to us saying, “We wanna help parents who’ve had problems with their children from this.” And that’s led to many lawsuits.

And then because we started that, we got early access and awareness about these AI companion cases. And I’m sorry to speak about this, you know, but these are things people have heard about in the news of when ChatGPT went from a homework assistant to becoming a suicide assistant over about a six-month period for Adam Raine, who was a 16-year-old.

And we knew about those cases early because of the success of “The Social Dilemma” work. We [00:44:00] became expert advisors on some of the litigation. We worked on the Character.AI case. We worked on the Adam Raine ChatGPT case. Um, and that’s led to real changes. Um, the, you know, the platforms are pushing– are pulling back from some of the aggressive marketing, uh, to kids.

There’s a lot of countries now that are doing social media bans for kids under 16. That’s probably one of the biggest impacts, not just from our work, but mostly from Jonathan Haidt, who’s carried that work forward with his book, “The Anxious Generation.” Um, and if you add up India as an entire country, plus Malaysia, Australia, France, New Zealand, Spain, um, Germany, Denmark, I believe the UK now, and Greece, there’s– I mean, I could give a long list of countries.

These are all the countries that are banning social media for kids under 16. That is an unreal outcome. So the big tobacco moment that we have been saying needs to happen for social media, especially on the kids’ issues, is finally happening. And as I give you these words, Aza, my co-founder, who you both know, um, is flying back from Tennessee, where he just testified today for the trial, um, against Meta.

Um, and you know, we’re [00:45:00] seeing that as you make these changes, as you show up in these places, and you make these contributions of having people understand why the design of the platforms and the incentives of the platforms are causing these problems, we’re seeing real changes happen. Um, and so we just need to do way more of that with AI.

Um, and with AI, that means things like, you know, global coordination. Uh, and yeah, those are the, those are the kinds of things that we’re working on. 

Yeah. I know, it’s just interesting that in America, um, where the social fragmentation and, you know, the nuclear family, I mean, h- we’ve led the way in the world in a, in, in a way.

I mean, like Europe is still thankfully behind us in, in the social isolation, you know- Yeah … in, in their societies. And, um, yeah, but we’ve– we, we were the early adapters, and we’ve gone so far and, and hopefully, hopefully, hopefully, if enough people listen to you, we, we really will, um, begin to come back and, you know, ’cause I, I c- y- you just And our, our, [00:46:00] our, um, financial hierarchy of squeezing the world, you know, we, we– and, and, and the fear, you see that the more wealth people accumulate, the more isolated they become.

Yeah. Yes, exactly. And then that’s not healthy for a number of different reasons, because one, the people that we need the most to be in the fight and connected and caring are, are, are like disconnected in, in a filter bubble, and then not coming from a place where they feel that love and connection and abundance and how much regular world is trustworthy and not just crazy people that are breaking into cars in San Francisco or whatever.

The feelings you can get when you’re, when you’re living in this detached and separate space. Um, yeah, 100%. Um, you know, and just to say, um, you know, a, a couple things here in the problem framing and solution framing. E.O. Wilson laid out such a long time ago that the fundamental problem of our humanity is that we have Paleolithic brains and bodies, you know, that are, that evolved on a, for a completely different environment.

We have medieval institutions, and we have godlike technology. So [00:47:00] Paleolithic brains, medieval institutions, godlike technology. That’s the problem statement. That’s been the problem statement with pesticides and forever chemicals, like amazing. We have Teflon, that’s godlike technology. But then now the egg doesn’t just stick to the pan, the sperm doesn’t stick to the egg.

So now we have this big problem. Um, and, uh, and we have godlike technology in the form of psychological manipulation and rewiring, you know? And that is also a huge problem. And we have godlike technology now, which is the ultimate godlike technology of AI, which is the exponentiation of all godlike technology.

It’s the godlike technology that gets you more godlike technology. AI accelerates AI in a unique way. And so this is the problem statement. The solution statement has to be something like Number one, embrace instead of reject our Paleolithic brains. We need to embrace the truth and reality of how our minds work, including denial-based reasoning, comfort-seeking instead of truth-seeking.

Um, we have to work– You pet the animal in the direction its fur actually goes. You have to be in relationship to the truth of how we work, and that’s what your medicine work is all about. It’s like understanding the natural pathways [00:48:00] and what bootloads those feedback loops, and you pet the animal in the direction its fur is meant to go.

And then the second is you have to upgrade the medieval institutions from, from being old to being like modern and actually capable of seeing the issues and the level of complexity that they operate at and at the pace that they operate at. And there’s ways of doing citizen assemblies and Audrey Tang-style work in Taiwan, where you can actually have governance move closer to the speed of some of the issues that are hitting society.

And then in terms of the godlike technology, you have to have the wisdom to wield the godlike technology. And there’s no definition of wisdom in any spiritual or religious tradition that doesn’t involve restraint as a central feature of what it means to be wise. Like, is there any tradition that’s like, “No, go as fast as possible and don’t think about it”?

No, all, all of the wisdom traditions, literally every single one, restraint is one of the top values in terms of what wisdom is. And so I do think that there’s a, a map and a pathway to how to do humane technology, how to solve E.O. Wilson’s kind of core framing of the problem, and we have to start changing the incentives, and we’ve, we’ve outlined a bunch of the solutions here.

There’s also liability and preventing AI legal personhood. Human rights are for [00:49:00] humans. We shouldn’t give AI legal personhood to, to AIs, um, which is one of the battles that’s being fought, um, which is a mechanism to avoid liability. So if you have liability where the externalities that, that show up on the balance sheets of society are internalized, if you’re responsible for the risks that you create, then you’ll see companies acting in a way more responsible way.

If you have AIs that don’t have legal personhood and human rights are protected for humans, that’s another way. If you prioritize making sure that energy prices stay low for regular people, and water’s always available for regular people, and jobs and basic work and dignity are also always available in some way, shape, or form, and making sure we’re putting those values at the center, these are the kinds of things that create a pro-human future 

And Tristan, you’re sharing this in spaces where leaders will hear you.

I’d love for you to talk about that, because right now you’re talking about all of these, these incentives and all of this hope. You’re giving people hope. But what I really want them to know is how you’re actually making that hope come alive. 

Yeah. Um, well, we just came back [00:50:00] from the, um, UN AI for Good conference in Geneva.

We have a new communication artifact that we’re hope- we’re exciting to, excited to bring into the world. Um, it was, it went so well, uh, there in Geneva, you know, in front of all the countries and people from all around the world who, who went there for the conference, that they actually wanna get us into the UN General Assembly, so we would actually present this on the floor of the UN.

Um, and we’ve seen that when we did a screening, for example, of the AI doc film at the UN with the AI negotiators, the people who work on AI as a topic at the UN, that when they saw the film, even they were, like, shocked and hadn’t really made sense of it the way that the film showed it to them, and it shifted their perspective.

Um, we just came back from the Vatican, where I did a screening of the AI doc for, uh, the Dicastery of Communications at the Vatican. Um, and Pope Leo himself saw the film a couple weeks later. It was right before the encyclical on AI that you saw Pope Leo, um, do the big public statement. It was a, the document that was, you know, a huge encyclical and, and really a, a moral voice [00:51:00] for putting human values at the center.

Um, and I know from people who work at the Vatican that our work in the past has actually influenced that document. Um, you know, we have something coming up with the president of Iceland that is around the 40th anniversary of the previous arms control talks between President Reagan and President Gorbachev, where she wants to screen the film to, you know, the whole country.

Uh, so there’s a lot of things that we’re doing, but it involves first getting the world on board and clear that we got to change paths. And part of it is that when you, when you hear and take all this in, it’s a lot. Like, it’s really overwhelming. Like, it’s hard for people’s nervous systems to just be with this much information.

But if you kind of realize our common humanity, that once you paint this picture, like, no one wants a future where they can’t look at their children or their grandchildren in the eyes and say, “Yeah, this is the world we’re creating for you.” Like, everyone does want that other thing. We just have to have common knowledge, which is a specific term that means I know that you know that I know, and you know that I know that you know that there is a, um, that the, that the current path is not [00:52:00] sustainable and that there is another pathway.

And for those who want resources, by the way, you can go to betterpath.ai, which is a website put together by one of our allies and friends, uh, the Future of Life Institute, to articulate that there are actually specific things we can do to move from the race to replace humans to the race to augment and build tool AIs.

Back to the initial question that you, you named at the start. 

Yeah. And because, I mean, AI at the level that we use is revolutionary. I mean, like, I don’t need it to change the world. It’s just, it’s just able to augment my ability to process information. 

Yeah. In, 

in, 

in- And you’re still in the loop, right? If you closed your office and you weren’t there anymore, then there may be a problem, because now you just have AIs making decisions and there’s no human oversight.

But you’re using it as a tool that’s– where it’s just helping you synthesize and make information. But of course, the check is you have to be still there scrutinizing it and making sure we’re not outsourcing too much of the automated decision-making. So it’s that- Oh, totally … that fine balance of, of how do we do that, and you guys are, are modeling [00:53:00] that.

Yeah. Well, I mean, just simple things like I use AI for my, uh, my notes, but at the same time, I also have a transcription of my notes because AI makes decisions about what it’s going to include and what it’s going to leave out, and that’s not always adequate. Yeah. So you need both. Yeah. You know, you can’t– You cannot get– I mean, you, you- You 

can’t hand it all over.

Yeah. You can’t, you can’t hand it over because it’s not… Not that we’re perfect, but it’s still not, it’s not good enough, but it’s useful. And I think that that’s the, the, the, um, the struggle or the tension here is that, you know, those of us who started, you know, we don’t want to lose AI because we’ve fallen in love.

Oh, it can really– It, it just, it’s just a, a delight for when I’m trying to get information. 

Yeah. 

Okay. But, but you’re– But un-un- but it’s like, it’s like, I guess I’m like the kid in the, in the candy store. I’m not seeing that you’ve changed the whole dietary program. 

Exactly. [00:54:00] Exactly. It’s like, how do I get- Yeah

efficiencies, but then how do you not completely upend the structure of the world and undermine a human future? Like there- Right … you don’t have to do both. Like, we can create a world of AI tools that are specifically designed to prevent cognitive outsourcing, that are not designed to replace human relationships, that are not designed to replace all work faster than society can metabolize that change, but instead be designed more as tools, um, and have policies that also are about protecting human livelihoods.

Not bullshit jobs, by the way. A lot of people will say, “It’s not that we want to protect jobs that no one wanted to do anyway.” So we’re not protecting some broken old world, but we are protecting the spirit of the law, not the letter of the law, the spirit of meaningful work, livelihoods, human connection, work that needs doing.

Um, and there are some policies and people like Molly Kinder at the, I think it’s now the Breakwater Institute is her new organization. Um, she was at Brookings. We did a great podcast with her, um, and she’s brilliant on these topics. Um, there’s, there’s a few people thinking really clearly. Erik Brynjolfsson at Stanford is another one on the economics and [00:55:00] jobs side.

I’ll say also for people who want to learn more, we have a podcast called “Your Undivided Attention” that goes through these different topics. We recently covered how do you do verification for international agreements on AI, um, with some experts on that. Um, and yeah, we hope that people, you know, pay attention and stay engaged and, and take action.

So, so the– I just want– so the international agreement, just s-speak of just, uh, you’ve been talking about it, but just, um, can you hit that, uh, you know, like really clearly about like maybe in my mind what the only way solution to get this to simmer down and to get some control is international agreements.

I know that’s what you’ve been working on. Can you talk just a little bit about that? 

Yeah. Well, first I’ll say that I’m not the most technically informed compared to the people in the field who really work on this topic, but some version of US and China needing to agree because they’re in this competitive dynamic for who goes further into the red zone of crazier and scarier AI capabilities that lead to more instability in society writ large.

[00:56:00] We need to trust that there’s some level of, “This is what we’re doing, and we’re going– we’re kind of pausing here. We’re staying at kind of at this level.” Um, and then we’ll use this collective sort of level that we’re at to kind of make sure that the rest of the world isn’t building levels that are beyond that.

And right now, if the US and China, as leading players with their massive influence in the world, if they really wanted that to happen, you could just ask, have we even really tried to make that happen? Has that really happened yet? No, it hasn’t really happened. In fact, AI just got added to the agenda in the last Trump-Xi meeting, uh, in May, May 14th, 15th.

Um, it wasn’t on the agenda. For the first time, it became on the agenda. There’s a new meeting, um, between Trump and Xi for, uh, September, I believe, when, when they visit the White House in, in end of September during UNGA week. And in terms of what needs to happen, the people in this podcast interview that you can listen to, um, speak about the power of chip control.

So, um, how do you have like data centers that can keep confidential what we’re both doing? Like, you don’t know what I’m doing, but you know that I’m not doing like training the [00:57:00] next model that’s dangerous or has cyber capabilities that are of a certain level. So it’s that kind of thing. Um, it involves a technology called zero-knowledge proofs.

The metaphor you can think of is what uranium was to nuclear weapons, like everyone needed uranium, and so the US could kind of start monitoring uranium and know where it’s going, and they could have nuclear inspections, and they could have satellite monitoring, and they could have seismic monitoring.

There’s all these kind of technologies we put together, but uranium was the limiting factor. With AI, the limiting factor is the hardware chips, these really advanced chips coming out of Nvidia to train these big AI models. Um, and you know, if there was a knowledge of those data centers and where they are and what they’re doing, and this special instrumentation, there are people who are working on the verification technologies that would enable these international agreements.

Um, it’s a long road. It’s not easy. It takes a lot of work. We have to do it very quickly. Um, but there’s people like Lucid Computing and, um, a couple other groups that are working on this, uh, every day. 

Oh, that’s, that’s reassuring [00:58:00] because i-it’s not gonna be solved because, um, you know, Sam Altman grew a conscience about, or, or anybody.

I don’t mean to, I mean, you, you know, don’t mean to- Yeah … names, but just any- Yeah … any one of them are, yeah, they’re, they’re not gonna, nobody, they’re not stopping. 

Yeah. Yeah. 

Yeah. 

They, they won’t– I– Privately, many of them will actually say that they would like to stop. In fact, if you, um… There’s a letter that just came out from, um, uh, a bunch of the lab employees, including co-founders of the AI labs.

It’s called Pacing the Frontier. Pacingthefrontier, I think, .org. And just in the last couple days, and it’s all the AI lab employees saying we basically need to slow down frontier AI development. They’re calling it pacing because they want to overt- avoid the word slow down, even though it means we need to slow down.

We need to have restraint. Um, and just to put meat on the bones there, even the CEO of Microsoft AI, Mustafa Suleyman, who’s a longtime friend of mine, he says that in the future, progress will depend more on what we’re willing to say no to than what we’re willing to say yes to. And [00:59:00] technology does not always equal progress on its own.

It’s technology times the governance of that technology so it doesn’t cause the problems. Like pesticides that create one, that solve one problem, but then generate 50 more new problems and cancers that have taken away loved ones from each of us in our lives. That’s not solving a problem. But, but, but chemicals and pesticides that don’t do that, that use more natural processes that are safer, that would be.

So we’re, we’re not against technology, we’re for humane technology, and I think that’s the same thing that you all are for. Medicine that’s actually not- Absolutely … creating externalities. You know, chemistry and pesticides that are n- that are taking away the things we, we want to take away, but, but also protecting the things and the health of our biosphere that we need to protect.

Thank you for the work that you do. I mean, it’s a big deal. Yeah. It’s, it reaches far. 

Thank you, Nafysa 

I mean, I- Thank you both. Yeah. Thank you so, so much because, you know, yeah, it- it’s, it’s, it’s just amazing that you’ve seen this, sought, and you’ve been working on doing something about it. So much of life is, “Oh, that’s [01:00:00] a problem,” but, you know, let somebody else do it.

And I, I just wanna thank you for, for taking th-this problem and just the, the, like I said, your first– the first work, “The Social Dilemma,” I mean, that- that’s… It’s the crisis of our time. 

Yeah. Yeah. 

So thank you 

so much. Well, thank you, thank you both so much, and also the ways that you support our work and, and, um, just what you do in the world and what you represent and, and how you do it, um, with a lot of love and care.

And, um, yeah, I mean, Iza and I are both really grateful and, um, you know, I just, I do want people to also feel that as, as dark as things have been, um, there has been a lot of progress in the last, uh, little while. And, and even Bernie Sanders and, and Steve Bannon, we jokingly call it the Bernie to Bannon Coalition or the B2B Coalition-

agree that we need to ban super intelligence and that we need this pro-human AI statement they both signed, just five core principles of how to govern AI. And so when you have people from across the political spectrum, doesn’t matter what religion you are, doesn’t matter what political orientation you have, um, they’re all for the same thing.[01:01:00] 

Once people see clearly this is not a, a fringe perspective, everything we’re talking about is actually the mainstream perspective that hasn’t yet been realized. So as much as this has been a hard conversation, I hope people leave with some optimism that if you are part of the immune system to this spell of inevitability, and you’re part of the immune system against this anti-human future, and you’re for this pro-human one, it’s still possible.

Yeah. And, and what’s the, um, the easiest way for people to find out more about what you’re doing and what you’re thinking about? 

Uh, yeah, we don’t publish nearly enough, but the podcast is a good resource, “You’re In Divided Attention.” It’s at humanetech.com, Center for Humane Technology’s website, and then, um, occasionally there’s Instagram and other updates.

We try to not use social media, uh, after “The Social Dilemma.” I didn’t have any accounts there. We sacrificed a lot of, um, reach and followers that we could have gotten probably. Uh, but now obviously because we need to alert the world about AI, um, uh, so occasionally there’s some things on, on social media and, uh, Instagram in, in particular.

Okay. Well, thank you for your time today and everything you do. 

Thank you both so much. Essies.[01:02:00] 

One sec. How do I stop it from recording? Oh, there we go. 

There we go.