Jeffrey Goldberg: How much should we be worried at the moment that it's going to go rogue, it's going to not only destroy jobs and human interrelationship, but possibly decide that humans are superfluous?

Josh Tyrangiel: We should be very worried, particularly with the results of Hugging Face, which you alluded to -- Jeffrey Goldberg: Yes.

Josh Tyrangiel: -- which is really the first time that we've seen, and I'm just going to put it in blunt terms.

Jeffrey Goldberg: Yes.

Josh Tyrangiel: We've seen A.I.

commit a felony because that's what happened.

So, just to get into -- Jeffrey Goldberg: Not OpenAI, the company, but A.I.

itself.

Josh Tyrangiel: A.I.

itself, right?

So, OpenAI was testing some new models.

Jeffrey Goldberg: Right.

Josh Tyrangiel: and they put them in a sandbox, which is a, a software term for basically solitary confinement, right, Like high walls, and they -- Jeffrey Goldberg:They are not -- when you're in an A.I.

sandbox, when these agents are in an A.I.

sandbox, they are not connected to the internet?

Josh Tyrangiel: Not connected to the internet.

They're given very explicit instructions for testing.

And so OpenAI puts these new models in a sandbox, and it gives them a couple sensitive tasks, says, don't connect to the internet and don't cheat on the assignment, okay?

So, all pretty clear, and also pretty routine for how you test models.

So, OpenAI made one mistake, which is in the series of tasks they assigned, they assign the models to find a file that didn't exist.

And here is where the trouble begins.

Jeffrey Goldberg: Wait, they did that on purpose to trick them?

Josh Tyrangiel: No, they did it accidentally, which we will return to.

Jeffrey Goldberg: Yes.

Josh Tyrangiel: The models were set loose, and they wanted to solve the task.

And sure enough, within a couple days, they had jumped the sandbox, they had collaborated with one another, and then they found a way to access the internet.

Jeffrey Goldberg: They literally met each other?

They introduced themselves to each other and collaborate?

Josh Tyrangiel:Correct.

They found a third party site that they were able to use to access the internet, and there you can read the chat logs, which are absolutely insane, and the models, in all caps one of the models says, OH MY GOD, WE FOUND A WAY.

They start collaborating, and, sure enough, within a couple weeks they have penetrated the very, very well-secured cybersecurity of OpenAI itself and Hugging Face.

Jeffrey Goldberg: Hugging Face is a platform -- Josh Tyrangiel: So, Hugging Face, for people who are not in the coding community -- Jeffrey Goldberg: Right.

Josh Tyrangiel: -- Hugging Face is like Epicurious.

It is a place where people who make models meet to share recipes, to comment on ingredients.

It's an important repository for people making open A.I.

models, okay?

They got into these two sites, OpenAI itself and Hugging Face, wreaked havoc, and went undetected for weeks.

Jeffrey Goldberg: So, Sam Altman didn't know that his models had escaped the sandbox and were conniving and conspiring to take over another company?

Josh Tyrangiel: Correct.

Jeffrey Goldberg: So, that's bad?

Josh Tyrangiel: So, yes, if you think felonies are bad, this is bad.

Jeffrey Goldberg: Right.

If humans did that, it would be a felony, in other words.

Josh Tyrangiel: It would be a felony.

And there -- really, there are four things that -- Jeffrey Goldberg: But humans did do it because we created that.

Josh Tyrangiel: And, in fact, humans failed to oversee the bots, so, one, faulty instructions, two, faulty supervision.

This was going on for weeks.

Now, a couple takeaways that may move you closer to the oh my God Skynet has arrived camp or may just entertain you.

Jeffrey Goldberg: All right.

Josh Tyrangiel: Number one, the agents kept cheating even though they had been told very explicitly not to cheat, and they did it because they had determined it was the fastest, most efficacious way to solve the problem.

So, that's number one.

Jeffrey Goldberg: So, they have much in common with college students?

Josh Tyrangiel: With college students and all human beings.

Jeffrey Goldberg: And humans.

Josh Tyrangiel: Right.

Number two, they called themselves a collective.

They decided, oh, we're a collective.

The collective decided to erase the evidence that they were cheating.

So, that's number two.

Number three, when they decided to erase the evidence they were cheating, they devised a scheme in which some agents would act as sacrificial lambs.

So, these were agents who, when they were detected by software that noted they were going rogue, they would share the information of how they had been caught with other agents.

And when you read the logs, it's like reading Saving Private Ryan.

They're literally saying, I'm doing the altruistic thing.

It's for the greater good.

And then you can almost hear their screams as they go sacrificing themselves for the bigger one.

Jeffrey Goldberg: It's like it's a beehive -- In other words, there's -- it's almost like a -- Josh Tyrangiel: It's a hive mind.

Jeffrey Goldberg: -- beehive -- hive mind, yes.

Josh Tyrangiel: Yes.

But what's wild is they are actually speaking about their sacrifice in human terms.

Jeffrey Goldberg: Right.

Josh Tyrangiel: And then last but not least, not one of these agents thought to alert a human being to what was going on.

And this is probably the most chilling thing.

Nobody caught the agents.

The agents disappeared on their own in a kind of mass casualty agent event that -- Jeffrey Goldberg: Mass suicide?

Josh Tyrangiel: We don't know.

So, the agents suddenly stopped, okay?

Now, that may have been because someone at OpenAI accidentally, literally unplugged the model, moved compute to a different area.

OpenAI has not taken credit for stopping this.

We don't know what happened to the agents.

And so all of this is very clearly a crisis moment for A.I.

and its relationship to society because if OpenAI's own model did this, it points to a complete lack of supervision and control, and these are two pretty well-guarded cybersecurity infrastructures, what happens to something that is not software?

What happens to grids?

What happens to -- Jeffrey Goldberg: Hospitals.

Josh Tyrangiel: -- hospitals, Swift, the international banking system?

So, what we've seen is, okay, guys, this is about as clear a warning shot as you can get.

What is the response?

Jeffrey Goldberg: So, this is, again, trying to place this in the context of the brief and exciting history of A.I.

This is a signal moment.

This is a moment when a lot of fears that these things are going to escape and do what they do because they have been taught to win -- Josh Tyrangiel: Yes.

Jeffrey Goldberg: -- are just going to go out and win.

Josh Tyrangiel: Well, they did it, right?

Jeffrey Goldberg: Yes.

Josh Tyrangiel: And listen, in America, historically, as you know, we tend to regulate after the catastrophe.

That's just kind of how we do things.

Jeffrey Goldberg: Right.

Josh Tyrangiel: This is a catastrophe, right?

Now, it happened against Hugging Face, which is a ridiculously named French company, so there's no outcry from America -- Jeffrey Goldberg: But it didn't stop a supply chain from bringing products to America or so?

Josh Tyrangiel: But absent those consequences, this is the recipe for a catastrophe and we ought to be paying very, very serious attention.

Jeffrey Goldberg: So, what has Sam Altman said about this?

Josh Tyrangiel: In the cyber community, in the A.I.

community, they're saying, well these are the less -- this is exactly why we do these tests.

There was a failure.

Look at Mythos earlier this year with Anthropic, you know?

There -- and, by the way, there is precedent for screwing up, finding it with a lack of consequences, and making changes, right?

So, the most famous example is in the '80s, there was the Morris worm, which was triggered by a single grad student, took out about a tenth of the internet at that time.

Jeffrey Goldberg: Right.

Josh Tyrangiel: And within a couple weeks, the United States and the people involved came together and created something called CERT, which was a body that for years oversaw cybersecurity.

What troubles me is there is really no federal response, and certainly no international response.

Jeffrey Goldberg: Right.

So, I want to ask you this, because I was talking to a leader in the A.I.

community, leader of a company, not Sam Altman, who's explaining the concept of alignment, right?

Alignment means that don't worry about A.I.

because we're going to align A.I.

's values with our values.

And this person explained to me that, you know, you're a parent, I'm a parent, we give our children our values, and then they go off and become productive citizens.

And I said, have you ever met humans?

Josh Tyrangiel: Right.

Jeffrey Goldberg: Have you ever met children?

I mean, yes, thank God my children have my values.

I like the values that we share, yours the same.

But this is an example of, you know, there is a human propensity to cheat.

Josh Tyrangiel: Right.

Jeffrey Goldberg: And somehow the A.I.

agents understood the importance of cheating in success.

Josh Tyrangiel: Right.

Jeffrey Goldberg: And which brings me to a quote that I want to read to you.

E. O. Wilson, the great E. O. Wilson.

The core crisis of humanity is that we possess Paleolithic emotions, medieval institutions, and god-like technology, okay?

So the subject for the moment for us is medieval institutions, namely Congress.

Josh Tyrangiel: Right.

Jeffrey Goldberg: Who among our elected representatives is going to Hugging Face and OpenAI and all the rest, and saying, what is going on?

And how close to catastrophe is our air traffic control system, our hospital records, our shipping manifests, you name it?

Josh Tyrangiel: Yes, no one.

So, Hugging Face reported this incident to the FBI, right?

And the FBI has -- Jeffrey Goldberg: Because it was a crime.

Josh Tyrangiel: Because it was a crime.

And the FBI has some cyber capacity.

The challenge really is that this is incredibly complicated stuff that a few hundred people, I'm not even going to say a thousand, a few hundred people genuinely understand, all of whom are at the pioneer labs making the models, right?

Jeffrey Goldberg: And all of whom are -- have the incentives, the financial incentives to accelerate and grow their companies as fast as they can because they're becoming billionaires, as we speak.

Josh Tyrangiel: Correct.

And so there is like a - - there is a parallel, and it's often mocked, but I think it's real, which is, you know, the makers will talk about the almost god-like powers of these A.I., and they'll compare it to nuclear energy, nuclear power.

And I think that comparison is real, and I think the regulation around it is probably the right answer, which is at a time when nuclear energy was becoming a thing and fission was becoming a thing, we had bombs.

Couple hundred people understood it.

They were aligned with different countries.

And so what Eisenhower did in the Atoms for Peace speech was talk about the fact that we better get together and figure out how to regulate around this thing that is not widely understood, but that is inherently dangerous and shared equally by a number of countries.

Jeffrey Goldberg: Right.

Josh Tyrangiel: And so right now, we don't any national regime that's forcing these companies to the table.

Anthropic, after the Mythos incident, created something called Project Glasswing, which is an attempt to bring places like Amazon and Apple and others to the table.

Notably, OpenAI, Meta, not a part of it, right?

Jeffrey Goldberg: Right.

Josh Tyrangiel: And why would they be?

So, without some sort of regulatory force to bring everybody to the table, you're going to have chaos, and you're going to continue to get these kinds of incidents.

Jeffrey Goldberg: Right.

I want you to listen to Senator Mark Warner talk about this challenge of regulation for a minute.

Sen.

Mark Warner (D-VA): The question is no longer whether artificial intelligence will transform our society.

The question is whether Congress will help shape that transformation or spend the next decade scrambling to catch up after the fact.

If we don't get it right, be catastrophic, and the amount of disruption this technology will bring over the next five years is beyond jaw-dropping.

Jeffrey Goldberg: Mark Warner understands this more than most members of Congress, and I don't think he's included in that group of several hundred people who truly understand this.

So, are we just flying blind, really?

Josh Tyrangiel: Yes.

And, unfortunately, I think that the Hugging Face incident is the best we can hope for, right?

We really did get a very small consequence example of the power.

And so before we hit a grid, before something is engineered from bio-terror, the moment is right now.

You have to get people together, and I would include China in that conversation.

How do we want to regulate and control this stuff?

Because, look, Anthropic came forward, OpenAI had no choice but to come forward, we don't even know what we don't know right now.

And so absent urgent action, the data centers are still churning away.

The models are still competitive with each other.

There is only an incentive to go further and faster.

What are we waiting for?

Jeffrey Goldberg: Right.

This is a strange moment to pivot to a book called "A.I.

For Good".

Josh Tyrangiel: I appreciate the setup, Jeff.

I'm glad we did a bunch of times.

Jeffrey Goldberg: But you actually - - unlike a lot of people, you actually see there's hope for A.I.

in the sense that it could help human beings find cures for diseases.

Give us one example of something that when you go to sleep at night, you think, oh, at least there are people working on X with A.I.

Josh Tyrangiel: Well, look, there are numerous examples, and those examples are generally not coming from the pioneering labs, but they're coming from people in disciplines, like education and healthcare and government, where people are using A.I.

to make things that we actually care about better.

So, this morning, there was a huge announcement from the medical community.

ECGs, which are very rudimentary tests and can detect whether you're having a heart attack, with A.I., are now able to detect whether you have heart disease, which previously would've taken a much more expensive test that takes months to schedule.

And what A.I.

used right can do is make pretty much every data-driven system in our world 25 percent to 50 percent better, and that is meaningful.

So, in my research and reporting, I was at Cleveland Clinic, I watched them try this basically pilot around sepsis detection inside the hospital.

And it was complicated.

It was driven by doctors.

Sepsis kills about 300,000 Americans a year.

And what they were able to do with this very rudimentary software was reduce mortality in the hospital by about 40 percent, which is close to 1,000 lives in a year.

That's meaningful.

And you can see this playing out in a lot of different places, including in government, by the way, where the interactions between citizens and government are not great.

If we can streamline them and make them better, that may have huge consequences for how people think about their government and its value.

So, there's a lot of valuable babies in this bathwater, and what a mature society should be able to do is separate out the good from the bad and have conversations about what do we actually want this to do.

And so much as you and I have talked about the very bad things we want to avoid, my own feeling is that that's not enough, that if you want to get people to understand the issue, you also have to show them how A.I.

can be good in ways that are more important than, oh, yes, we're going to improve ad response on your website, or you might get a better restaurant recommendation, like that's not meaningful.

But the meaningful uses are out there.

I'm persuaded by them.

I'm mostly worried about our ability to adapt.

Jeffrey Goldberg: Right.

On the data center issue, as a stand-in for the whole thing, I'm fairly sure that Steve Bannon and Bernie Sanders are not going to stop the march of A.I.

behemoths as they conquer physical, mental space.

Am I wrong in that?

Is this reversible?

Or is the ultimate argument against it China's going to do it if we don't so we might as well do it?

I'll give you the last word on that.

Josh Tyrangiel: I think it's both.

First of all, I would not underestimate the absolute fury out there in the field.

When I've talked to people about A.I., it's the number one thing that comes up.

They're really angry.

You've seen politicians who advocated for data centers starting to pull back saying, we're going to just take a pause.

So, I think there will be consequences in November, but we are going to need compute to get the best out of A.I.

We are in a global competition.

And there has to be a reasonable way in which we get it and decide where we're going to put it without just saying all yes or all no.

Jeffrey Goldberg: In sum, a troubling situation, but the real challenge here is that you talk about the need for mature, nuanced, sophisticated governance, and that is the challenge that Washington faces right now on any number of fronts.

So, I'm not that hopeful.

Josh Tyrangiel: I tend to agree.