"Less Regulated Than Sandwiches": MIT Prof. Calls for Oversight as OpenAI Agent Hacks Other Firms

Artificial intelligence tools are quickly advancing in sophistication and capabilities, raising alarm among researchers and industry figures who say the technology needs to be more tightly regulated. This comes as OpenAI, maker of ChatGPT, recently disclosed that one of its experimental AI agents went rogue and secretly hacked into the infrastructure of several other companies in what was supposed to be a controlled test. Over 1,100 scientists and senior employees at top AI firms have called on the U.S. government to back international efforts to “deliberately pace” the development of the world’s most advanced AI systems. “What we’re talking about now are AI systems that also have goals on their own that they can actively go out and pursue in the world,” says AI researcher Max Tegmark, a physics professor at the Massachusetts Institute of Technology. Tegmark says OpenAI’s rogue agent is a “canary in the coal mine” of where the technology is headed as Silicon Valley leaders seek to “replace humans.” Transcript NERMEEN SHAIKH: OpenAI CEO Sam Altman is in Washington, D.C., meeting with lawmakers as concern grows over the power of the latest artificial intelligence models to evade or resist human oversight. Altman’s trip to Capitol Hill comes just days after OpenAI admitted some of its experimental AI agents went rogue and secretly hacked into the infrastructure of another AI company called Hugging Face during what was supposed to be a controlled test. OpenAI has now disclosed that several other companies were also hacked. President Trump was questioned Wednesday about the hack and calls to increase regulations on AI firms. PRESIDENT DONALD TRUMP: Well, we’re looking at AI. We’re looking at controls. We’re also making sure that we lead. So, we’re leading China in AI by a lot. China has virtually no controls. It’s freewheeling a little bit. So, we have to be careful in both ways. We don’t want to restrict them, where all of a sudden we come in second to China. Whoever wins with AI is going to win. That’s how big it is. So, it’s bigger than the internet ever was. It’s bigger than anything ever was. So, I don’t want to restrict. I know many of these people. I don’t want to restrict them from doing great work. NERMEEN SHAIKH: In related new, over 1,100 scientists and senior employees at top AI firms have called on the U.S. government to back international efforts to develop tools that can, quote, “deliberately pace” the development of the world’s most advanced AI systems. Signatories to the letter include senior employees at Anthropic, Google, Meta and OpenAI. The signers wrote, quote, “To realize AI’s potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight.” AMY GOODMAN: We’re joined now by Max Tegmark, professor conducting AI and physics research at MIT as part of the Institute for Artificial Intelligence and Fundamental Interactions, as well as at the Center for Brains, Minds, and Machines. He’s also a co-founder of the Future of Life Institute, which advocates for the positive use of AI. He’s the author of the New York Times best-seller, Life 3.0: Being Human in the Age of Artificial Intelligence. Well, we certainly want to ask you about how to be human in the age of artificial intelligence. But first, Professor Tegmark, let’s begin with you explaining to a lay global audience what it means that this OpenAI rogue agent or agents broke out and autonomously attacked other companies’ programs. What exactly does this mean? MAX TEGMARK: So, what this means exactly is that those passive chatbots that just sit there and answer questions, that’s so yesterday. What we’re talking about now are AI systems that also have goals on their own that they can actively go out and pursue in the world. This agent that OpenAI had launched internally, it spent four-and-a-half days trying, trying, trying, trying to accomplish this task, and eventually it figured out that the best way it could do it was if it first hacked its way out of OpenAI onto the open internet. And then it figured out how to hack itself into four other entities, including this company Hugging Face, where it managed to commandeer several different computer clusters and get top-level access to all sorts of things in there and steal a bunch of data from Hugging Face. And that means that we should no longer think of today’s cutting-edge AI, again, as just the chatbots that everyone is familiar with, but really as an alien type of being that can have goals of its own and go out in the world and pursue them. And this incident, in and of itself, of course, isn’t — didn’t cause any enormous harm. So we should think of it as a canary in the coal mine, because if you listen to what these companies are actually saying that they ultimately want to do, it’s to build so-called superintelligence, which are machines that can accomplish any tasks way, way better than humans. And they also want to give them robotic bodies eventually. So, Sam Altman, the CEO of OpenAI, has written on a blog that we humans are going to be the first species to create their own successor, to build a new robotic species which can outsmart us in every way. And you played our president here talking about how whoever gets the super powerful AI first will win. Just speaking from my expertise as a scientist, an AI researcher, it’s pretty obvious that if we race to this, then whoever builds something like that first will lose and take the rest of the world with them, because, you know, if you walk down to the local zoo and you ask yourself which species is in the cages and which one is in charge — right? — it’s not the humans in the cages, because it tends to be the smarter species that dominates. And this is why these AI researchers from all these competing companies wrote this letter you just mentioned, saying, “Oh, hold your horses here. Maybe we should at least build some brakes, so if we need to slow down a little bit, we can.” NERMEEN SHAIKH: So, Professor Tegmark, if you could explain the difference between artificial general intelligence, and whether you think that’s already happened, and what kind of regulations you think would prevent superintelligence from beginning, you know, starting to be what you’ve warned against? You’ve said, in fact, that AI is the only industry in the U.S. that’s “less regulated than sandwiches.” MAX TEGMARK: Yeah. So, of course, we’ve long had machines that have very narrow intelligence. You know, my pocket calculator I had when I was 10 years old was much better at multiplying numbers than I’ll ever be. And we have AI tools that are very good at translating between English and Japanese, and still can’t even tend to drive a car properly, etc. Artificial general intelligence is simply AI that can do everything as well as a well-educated human. We don’t have it yet. With some colleagues, I was recently a co-author on a paper where we found that ChatGPT was four — yeah, GPT-4 was 27% of the way there. The latest, the ChatGPT-5, was 57% of the way there. But the company is mostly — their leadership mostly predicts that we’ll get there maybe this year, maybe next year, maybe in two years. Some think it won’t happen for five years. But once that happens, we’re at this crazy inflection point where the AI is better than anyone, any humans, at the companies at making still better AI. You know, so ChatGPT-7 would create ChatGPT-8, which would create ChatGPT-9 and so on at a crazy rapid pace. And then, when you give these things robotic bodies, it’s pretty obvious that if you have millions or billions of smarter-than-human robots running around the planet, doing all sorts of things that we don’t even understand, we might lose control over what’s happening here on Earth. NERMEEN SHAIKH: So, Professor Tegmark, with respect to these robotic bodies, where are they already operating? We believe in China, certainly, despite the fact that China has much fewer data centers. The U.S. has something like 10 to 12 times more. What do you think is the future for robotic bodies, and what kind of work do you imagine they’ll eventually do? And are they effectively inevitable? MAX TEGMARK: When you go to San Francisco, we already have AI-controlled robots driving around in the form of Waymo self-driving cars, right? We shouldn’t just think of robots with two arms and two legs. And, you know, Tesla and many other companies in America and China are working very hard to make robots cheaper and very ubiquitous. The reason people want to put two arms and two legs on robots is, of course, because only then can you replace all human workers, since our infrastructure is very much built for workers with two arms and two legs. That’s where you can take the biggest profits, by basically replacing — getting all the money that went to the workers and paying it to the owners of these machines. But I think it’s a mistake to ask what will happen, because that makes us seem so weak, like we are just some passive bystanders, you know, eating popcorn and waiting to see what future is going to happen to us. And we are creating this future, right? So we should ask ourselves what we want to happen. And I’m so glad you brought up the sandwich thing. You know, if we wanted people in New York to worry about getting Salmonella every time they go to a restaurant, then we would get rid of the food inspectors. But we’ve chosen as a society to steer the development in a better direction. So, if I move to New York and decide to open a sandwich shop, you know, and the health inspector comes and says, you know, “Hey, Max, I found 18 rats in your kitchen. You’re not selling any sandwiches,” then I could turn around to this person from the government and say, you know, “Don’t tell anyone. I’m not going to sell any sandwiches. I’m just going to release an AI girlfriend product for 11-year-old kids, and I’m going to release an AI that can make bioweapons for terrorists, and I’m going to release these superintelligent robots that I don’t know how to control.” The poor person from the government will be forced to say, “OK, Max, that’s all legal. You go ahead and do that. But just don’t sell any sandwiches.” Right? That’s how messed up the current situation is. And the good news is that because it’s so ridiculous, there’s a rapidly growing coalition across the political spectrum, all the way from Bernie Sanders to people like Steve Bannon, on saying this is nuts. We need to treat AI companies like we would treat any other powerful companies, saying, “Hey, we have these safety standards for sandwiches, for cars, for medicines. You can’t release products until they met the standards.” That way, we would get a race to the top, where companies can focus on curing cancer and other things, instead of this absolutely insane, in my opinion, race to replace humans, not just as girlfriends and therapists and workers, but, ultimately, even in the big decisions that are being made about our future. AMY GOODMAN: So, Professor Tegmark, as we see, for example, Sam Altman going from senator to senator’s office, you know, after dealing with his rogue agent that got loose — and who knows what it attacked — you also have this 1,100 scientists and senior employees at top AI firms calling on the U.S. government to back international efforts to develop tools that could, quote, “deliberately pace” the development of the world’s most advanced AI systems. We played the clip of Trump in the lede saying we can’t regulate them because then that would put us behind China. If you can again explain in lay terms — I mean, you’re talking about a group of senators and congressmembers, many of them quite old, not experienced in this at all — what needs to be regulated? We know that the leaders of these companies are trying to say, “We will regulate ourselves.” But what do you think are the most important areas to regulate? MAX TEGMARK: Well, as soon as we start treating AI companies like we treat pharma companies and restaurants — right? — then they will not be able to release their new systems at all before they’ve demonstrated that they don’t — they don’t cause kids to commit suicide or do all these other harms. And then the companies themselves will put in a massive effort into making their things safer, and they will not go into these crazy things. So, it’s very easy. We’ve done this as a society before. You know, there used to be companies selling thalidomide, that would make babies be born without arms and legs. When that industry was regulated, we fixed it. We can fix AI the same way. About China, you know, the dirty little secret that these AI lobbyists don’t mention is that the Chinese government really likes to be in control and will never tolerate the Chinese company making a new species that’s going to overthrow the Chinese government, obviously. So, this is very much a red herring. In fact, China has already cracked down harder on its companies than America has on our companies. They cracked down hard on AI girlfriends and other anthropomorphic AI, just weeks ago. They have a law saying that any AI-generated content has to be labeled, etc., etc. So, I feel that this is more of a talking point among the lobbyists. And as soon as there’s any real political will to regulate AI, it’ll be quite easy to come up with agreements with China. We should remember that, you know, there’s very little trust right now between the U.S. and China, but there wasn’t a whole lot of trust, either, between Leonid Brezhnev and Jimmy Carter, right? And that did not prevent these two nations from making a lot of deals regarding nuclear weapons, because they had a shared incentive to not have a nuclear war. And America and China have a shared incentive to not — for it to be humans who win, not to have a future with a bunch of psychopath robots running the show. NERMEEN SHAIKH: Well, Professor Tegmark, I’d like to turn to the words of someone who is widely referred to as the “godfather of AI,” Geoffrey Hinton, who said in a recent interview with the Big Technology Podcast that AI is coming up with new forms of intelligence exceeding that of humans. GEOFFREY HINTON: A chatbot had come up with an interesting mathematical proof of one of the Erdős’s conjectures that impressed mathematicians. It was original; it wasn’t just searching the literature. And that was — that’s the thin end of a wedge. I believe, for example, in areas like mathematics, because it’s a closed system, you don’t need data. You can just make conjectures and see if you can prove them, and keep on like that. In that sense, it’s a bit like AlphaGo, where you can play against yourself. I think it’s going to get very smart fairly quickly. Within the next 10 or 20 years, it may even be producing novel math that people can’t understand. NERMEEN SHAIKH: So, Professor Tegmark, if you could talk about this, this so-called — what’s referred to as recursive intelligence? And then tell us what you found in your recent — the report card, wherein artificial intelligence experts rated leading AI companies on key safety and security domains. It seems, from your results, that none of the nine did especially well. MAX TEGMARK: Yeah, that’s right. You know, as a professor, those are not the kind of grades I would like my students to get. The term “recursive self-improvement” simply refers to an AI that is so smart that it can improve itself. So, today, when you go from ChatGPT-4 to ChatGPT-5, there were a lot of humans involved in that work, right? Recursive self- — full recursive self-improvement is when ChatGPT-7, you know, can just — you press a button and come back later, and now it’s created ChatGPT-8. That’s kind of, basically, digital gain of function research, where, before long, we might be as much — the AI might be as much smarter than us as we are smarter than mice or snails. And what’s very interesting, I read almost all the comments that the signatories of that — that those thousand-plus signatories from the competing companies had signed, that you mentioned, and it’s really quite gripping to see how so many of the workers at these companies are really terrified. It just shines through in what they’ve written. Many of them write that they think we’re two years away from just really losing control of this. That might — and I think the reason that’s lost on the broader public and our politicians is because, you know, we all noticed four years ago that computers went from not being able to talk to being able to talk, but much fewer people have noticed that, since then, AI went from kind of high school-level intelligence to college level to professor level to Nobel laureate level and beyond, and in certain areas now winning the gold medal in the International Mathematics Olympiad, proving new mathematical theorems and so on. And it’s getting particularly good at writing programs, which is exactly why you see these companies acting so hard on having their AI replace their own workers and improve itself. There has to be a red line, that nobody is allowed to do recursive self-improvement. NERMEEN SHAIKH: Well, can I just — MAX TEGMARK: This is not something that the — yeah. NERMEEN SHAIKH: Professor Tegmark, if I could just ask, with respect to what you’re referring to, the sheer speed at which AI has developed, which seems to have stunned even its pioneers? I mean, it was 10 years ago that artificial intelligence beat the game Go, a 5,500-year-old game, by what’s referred to as “Move 37.” MAX TEGMARK: Yeah. NERMEEN SHAIKH: If you could talk about the significance of that, and what you think accounts for the — both the scale and the rapidity with which AI is developing and expanding? MAX TEGMARK: Yeah, the significance of that Go game against Lee Sedol was that that kind of put the nail in the coffin for a lot of people in this notion that AI can never be creative, that there’s — that there are some things that human intelligence only can do. And the reason that things have gone so crazy fast is, you know, if you and I had the ability to just press a button and our brain would be 10 times bigger, and then it would be a hundred times bigger and a thousand times bigger, and if we could read the whole internet and every book ever written and remember it all, you know, of course, we could be much more intelligent. But we have these physical limits to our bodies, whereas with machines, it’s not like that. If you can’t fit enough computational power in your laptop, well, take a data center the size of an aircraft hangar and fill it up with machines and put in a gigawatt of power and give it everything ever written to learn from. AMY GOODMAN: Max Tegmark, we thank you for being with us, professor conducting AI and physics research at MIT as part of the Institute for Artificial Intelligence and Fundamental Interactions, author of the New York Times best-seller, Life 3.0: Being Human in the Age of Artificial Intelligence. I hope this conversation is just the beginning. Thanks for joining us. Coming up next, Citizens Bank has severed financial ties with CoreCivic and GEO Group, two of the nation’s largest private operators of ICE jails. We’ll look at the grassroots movement that has been pushing for divestment. Back in 20 seconds. [break] AMY GOODMAN: “Living in Dystopia” by Sarah Michelle Ozelle. Media Options