• OpenAI's o1 model was hailed as a breakthrough in September.
• By November, a Chinese AI lab had released a similar model called DeepSeek.
• On Thursday, Google came out with a challenger called Gemini 2.0 Flash Thinking.

In September, OpenAI unveiled a radically new type of AI model called o1. In a matter of months, rivals introduced similar offerings.

On Thursday, Google released Gemini 2.0 Flash Thinking, which uses reasoning techniques that look a lot like o1.

Even before that, in November, a Chinese company announced DeepSeek, an AI model that breaks challenging questions down into more manageable tasks like OpenAI's o1 does.

This is the latest example of a crowded AI frontier where pricey innovations are swiftly matched, making it harder to stand out.

"It's amazing how quickly AI model improvements get commoditized," Rahul Sonwalkar, CEO of the startup Julius AI, said. "Companies spend massive amounts building these new models, and within a few months they become a commodity."

The proliferation of multiple AI models with similar capabilities could make it difficult to justify charging high prices to use these tools. The price of accessing AI models has indeed plunged in the past year or so.

That, in turn, could raise questions about whether it's worth spending hundreds of millions of dollars, or even billions, to build the next top AI model.

September is a lifetime ago in the AI industry

When OpenAI previewed its o1 model in September, the product was hailed as a breakthrough. It uses a new approach called inference-time compute to answer more challenging questions.

It does this by slicing queries into more digestible tasks and turning each of these stages into a new prompt that the model tackles. Each step requires running a new request, which is known as the inference stage in AI.

This produces a chain of thought or chain of reasoning in which each part of the problem is answered, and the model doesn't move on to the next stage until it ultimately comes up with a full response.

The model can even backtrack and check its prior steps and correct errors, or try solutions and fail before trying something else. This is akin to how humans spend longer working through complex tasks.

DeepSeek rises

In a mere two months, o1 had a rival. On November 20, a Chinese AI company released DeepSeek.

"They were probably the first ones to reproduce o1," said Charlie Snell, an AI researcher at UC Berkeley who coauthored a Google DeepMind paper this year on inference-time compute.

He's tried DeepSeek's AI model and says it performs well on complex math problems that must be solved by thinking for longer and in stages. 

He noted that in DeepSeek's DeepThink mode, the model shows users every step of its thought process. With o1, these intermediate steps are hidden from users. 

"I've asked people at OpenAI what they think of it," Snell told BI. "They say it looks like the same thing, but they don't how DeepSeek did this so fast."

OpenAI didn't respond to a request for comment. On Friday, the startup previewed an o1 successor, called o3. Francois Chollet, a respected AI expert, called the update a "significant breakthrough."

Andrej Karpathy, an OpenAI cofounder, praised Google's new "Thinking" model for the same reasoning feature.

"The prominent and pleasant surprise here is that unlike o1 the reasoning traces of the model are shown," he wrote on X. "As a user I personally really like this because the reasoning itself is interesting to see and read — the models actively think through different possibilities, ideas, debate themselves, etc., it's part of the value add."

A DeepSeek demo

Snell shared a multistep math problem with Business Insider, which we used to test DeepSeek for ourselves: 

"Find a sequence of +, -, /, * which can be applied to the numbers 7, 3, 11, 5 to get to 24, where each of the given numbers is used exactly once."

BI put that prompt in DeepSeek's chat window on its website. The model responded initially by laying out the challenge ahead.

"Alright, so I've got this problem here: I need to use the numbers 7, 3, 11, and 5, and combine them with the operations of addition, subtraction, multiplication, and division, using each number exactly once, to get to 24," it replied. "At first glance, this seems a bit tricky, but I think with some systematic thinking, I can figure it out."

It then proceeded through multiple steps over roughly 16 pages of discussion that included mathematical calculations and equations. The model sometimes got it wrong, but it spotted this and didn't give up. Instead, it swiftly moved on to another possible solution. 

"Almost got close there with 33 / 7 * 5 ≈ 23.57, but not quite 24. Maybe I need to try a different approach," it wrote at one point. 

After a few minutes, it found the correct solution. 

"You can see it try different ideas and backtrack," Snell said in an interview on Wednesday. He highlighted this part of DeepSeek's chain of thought as particularly noteworthy:

"This is getting really time-consuming. Maybe I need to consider a different strategy," the AI model wrote. "Instead of combining two numbers at a time, perhaps I should look for a way to group them differently or use operations in a nested manner."

Then Google appears

Snell said other companies are likely working on AI models that use the same inference-time compute approach as OpenAI.

"DeepSeek does this already, so I assume others are working on this," he added on Wednesday.

The following day, Google released Gemini 2.0 Flash Thinking. Like DeepSeek, this new model shows users each step of its thought process while tackling problems. 

Jeff Dean, a Google AI veteran, shared a demo on X that showed this new model solving a physics problem and explained its reasoning steps. 

"This model is trained to use thoughts to strengthen its reasoning," Dean wrote. "We see promising results when we increase inference time computation!"

• Suchir Balaji helped OpenAI collect data from the internet for AI model training, the NYT reported.
• He was found dead in an apartment in San Francisco in late November, according to police.
• About a month before, Balaji published an essay criticizing how AI models use data.

The recent death of former OpenAI researcher Suchir Balaji has brought an under-discussed AI debate back into the limelight.

AI models are trained on information from the internet. These tools answer user questions directly, so fewer people visit the websites that created and verified the original data. This drains resources from content creators, which could lead to a less accurate and rich internet.

Elon Musk calls this "Death by LLM." Stack Overflow, a coding Q&A website, has already been damaged by this phenomenon. And Balaji was concerned about this.

Balaji was found dead in late November. The San Francisco Police Department said it found "no evidence of foul play" during the initial investigation. The city's chief medical examiner determined the death to be suicide.

Balaji's concerns

About a month before Balaji died, he published an essay on his personal website that addressed how AI models are created and how this may be bad for the internet. 

He cited research that studied the impact of AI models using online data for free to answer questions directly while sucking traffic away from the original sources.

The study analyzed Stack Overflow and found that traffic to this site declined by about 12% after the release of ChatGPT. Instead of going to Stack Overflow to ask coding questions and do research, some developers were just asking ChatGPT for the answers. 

Other findings from the research Balaji cited: 

• There was a decline in the number of questions posted on Stack Overflow after the release of ChatGPT.
• The average account age of the question-askers rose after ChatGPT came out, suggesting fewer people signed up to Stack Overflow or that more users left the online community.

This suggests that AI models could undermine some of the incentives that created the information-rich internet as we know it today.

If people can get their answers directly from AI models, there's no need to go to the original sources of the information. If people don't visit websites as much, advertising and subscription revenue may fall, and there would be less money to fund the creation and verification of high-quality online data.

MKBHD wants to opt out

It's even more galling to imagine that AI models might be doing this based partly on your own work. 

Tech reviewer Marques Brownlee experienced this recently when he reviewed OpenAI's Sora video model and found that it created a clip with a plant that looked a lot like a plant from his own videos posted on YouTube. 

"Are my videos in that source material? Is this exact plant part of the source material? Is it just a coincidence?" said Brownlee, who's known as MKBHD.

Naturally, he also wanted to know if he could opt out and prevent his videos from being used to train AI models. "We don't know if it's too late to opt out," Brownlee said.

'Not a sustainable model'

In an interview with The New York Times published in October, Balaji said AI chatbots like ChatGPT are stripping away the commercial value of people's work and services.

The publication reported that while working at OpenAI, Balaji was part of a team that collected data from the internet for AI model training. He joined the startup with high hopes for how AI could help society, but became disillusioned, NYT wrote. 

"This is not a sustainable model for the internet ecosystem," he told the publication.

In a statement to the Times about Balaji's comments, OpenAI said the way it builds AI models is protected by fair use copyright principles and supported by legal precedents. "We view this principle as fair to creators, necessary for innovators, and critical for US competitiveness," it added.

In his essay, Balaji disagreed.

One of the four tests for copyright infringement is whether a new work impacts the potential market for, or value of, the original copyrighted work. If it does this type of damage, then it's not "fair use" and is not allowed. 

Balaji concluded that ChatGPT and other AI models don't quality for fair use copyright protection. 

"None of the four factors seem to weigh in favor of ChatGPT being a fair use of its training data," he wrote. "That being said, none of the arguments here are fundamentally specific to ChatGPT either, and similar arguments could be made for many generative AI products in a wide variety of domains."

Talking about data

Tech companies producing these powerful AI models don't like to talk about the value of training data. They've even stopped disclosing where they get the data from, which was a common practice until a few years ago. 

"They always highlight their clever algorithms, not the underlying data," Nick Vincent, an AI researcher, told BI last year.

Balaji's death may finally give this debate the attention it deserves. 

"We are devastated to learn of this incredibly sad news today and our hearts go out to Suchir's loved ones during this difficult time," an OpenAI spokesperson told BI recently. 

If you or someone you know is experiencing depression or has had thoughts of harming themself or taking their own life, get help. In the US, call or text 988 to reach the Suicide & Crisis Lifeline, which provides 24/7, free, confidential support for people in distress, as well as best practices for professionals and resources to aid in prevention and crisis situations. Help is also available through the Crisis Text Line — just text "HOME" to 741741. The International Association for Suicide Prevention offers resources for those outside the US.

• Andrej Karpathy recently suggested AI could enhance e-book reading with interactive features.
• Amazon may already be thinking about this for its Kindle e-books.
• The company is looking for an applied scientist to improve the reading and publishing experience.

The AI pioneer and OpenAI cofounder Andrej Karpathy thinks AI can significantly improve how people read books. Amazon may already be thinking about how to do this for its Kindle e-books business.

In a series of posts on X this week, Karpathy proposed building an AI application that could read books together with humans, answering questions and generating discussion around the content. He said it would be a "huge hit" if Amazon or some other company built it.

A recent job post by Amazon suggests the tech giant may be doing just that.

Amazon is looking for a senior applied scientist for the "books content experience" team who can leverage "advances in AI to improve the reading experience for Kindle customers," the job post said.

The goal is "unlocking capabilities like analysis, enhancement, curation, moderation, translation, transformation and generation in Books based on Content structure, features, Intent, Synthesis and publisher details," it added.

The role will focus on not just the reading experience but also the broader publishing and distribution space. The Amazon team wants to "streamline the publishing lifecycle, improve digital reading, and empower book publishers through innovative AI tools and solutions to grow their business on Amazon," the job post said.

3 phases

Amazon identified three major phases of the book life cycle and thinks AI could improve each one.

• First up is the publishing part where books are created.
• Second is the reading experience where AI can help build new features and "representation" in books and drive higher reading "engagement."
• The third stage is "reporting" to help improve "sales & business growth," the job post said.

An Amazon spokesperson didn't immediately respond to a request for comment on Friday.

'I love this idea'

There seems to be huge demand for this type of service, based on the response to Karpathy's X post.

Stripe CEO Patrick Collison wrote under the post that it's "annoying" to have to build this AI feature on his own, adding that it would be "awesome when it's super streamlined."

Reddit's cofounder Alexis Ohanian wrote, "I love this idea."

Do you work at Amazon? Got a tip?

Contact the reporter, Eugene Kim, via the encrypted-messaging apps Signal or Telegram (+1-650-942-3061) or email ([email protected]). Reach out using a nonwork device. Check out Business Insider's source guide for other tips on sharing information securely.

• Amazon is emphasizing customer choice over market dominance with its AI strategy.
• Amazon unveiled a new series of AI models called Nova this week.
• Amazon's Bedrock tool supports diverse models from multiple providers, unlike OpenAI.

Amazon believes AI models are not in a winner-take-all market.

The company drilled down on this message during this week's re:Invent, the annual extravaganza for its Amazon Web Services cloud unit. Even after unveiling a new series of homegrown AI models called Nova, which, by some measures, are as powerful as other market leaders, Amazon stressed the goal is to provide more choice to customers.

AI models have become the new battleground for tech supremacy since OpenAI released its popular ChatGPT service in late 2022. Companies have rushed to up the ante, trying to outperform each other in model performance.

Amazon has largely been absent from this race. Instead, it has tried to stay neutral, arguing that the generative AI market is so big and varied that customers will want more model choices that fit their different needs. Amazon still believes this is the right approach.

"There are some that would want you to believe there's just this one magic model that could do everything — we never believed in it," Vasi Philomin, AWS's VP of generative AI, told Business Insider. "There'll be many, many winners and there are really wonderful companies out there building some amazing models."

Different positioning

As part of this, Amazon has used Bedrock, an AI development tool that gives access to many models, as its main horse in the AI race. This approach differed from OpenAI, and Meta, which mostly focused on building powerful models or chatbots. Google has a leading AI model in Gemini, but also provides access to other models through its Vertex cloud service, and Microsoft has a similar offering.

This week, Amazon further leaned into its strategy, announcing an array of new updates for Bedrock, including a marketplace for more than 100 specialized models and a distillation feature that fine-tunes smaller, more cost-effective models. It also unveiled new reasoning and "multi-agent" collaboration features that help build better models.

Swami Sivasubramanian, AWS's VP of AI and data, told BI that AWS "pioneered" the model-choice approach and intends to continue to promote it as a "core construct" of the business.

"GenAI is a lot bigger than a single chatbot or a single model to reach its full potential," Sivasubramanian said.

More companies appear to be taking the multi-model approach. According to a recent report by Menlo Ventures, companies typically use 3 or more foundation models in their AI services, "routing to different models depending on the use case or results."

As a result, Anthropic, which Menlo Ventures has backed, doubled its share in the AI model market to 24% this year, while OpenAI's share dropped from 50% to 34% year-over-year, according to the report.

'Choice matters'

Amazon may have no choice but to stick to this narrative. When OpenAI captivated the world with ChatGPT a couple of years ago, Amazon was caught flat-footed, leading to an internal scramble to find answers, BI previously reported. Its first in-house model, called Titan, drew little attention.

Having its own advanced, powerful AI models could help Amazon. It might attract the largest AI developers and promote AWS as the leader in the AI space. It would potentially also encourage those developers to continue building within AWS's broader cloud ecosystem.

Amazon isn't giving up on building its own advanced models. Last year, it created a new artificial general intelligence team under the mandate to build the "most ambitious" large language models. On Tuesday, Amazon unveiled the early results of that effort with its Nova series, which includes a multimodal model capable of handling text, image, and video queries.

Still, Amazon's CEO Andy Jassy downplayed any notion of Nova going after competitors. He said he's been surprised by the diversity of models developers use and that Nova is just one of the many options they will have.

"There is never going to be one tool to rule the world," Jassy said during a keynote presentation this week.

It's hard to know how successful this approach is as Amazon doesn't break out its AI revenue. But Jassy was even more bullish on the AI opportunity during October's call with analysts. He said AWS was now on pace to generate "multi-billion dollars" in AI-related revenue this year, growing at "triple-digit percentages year over year." Amazon's AI business is "growing three times faster at its stage of evolution than AWS did itself," he added.

Rahul Parthak, VP of data and AI, go-to-market, at AWS told BI that Nova's launch was partly driven by customer demand. Customers have been asking for Amazon's own model because some prefer to deal with one vendor that can handle every aspect of the development process, he said.

Amazon still wants other models to thrive because its goal isn't about beating competitors but offering customers "the best options," Parthak added. He said more companies, like Microsoft and Google, are following suit, offering more model choices via their own cloud services.

"We've been pretty thoughtful and clear about what we think customers need, and I think that's playing out," Parthak said.

Do you work at Amazon? Got a tip?

Contact the reporter, Eugene Kim, via the encrypted-messaging apps Signal or Telegram (+1-650-942-3061) or email ([email protected]). Reach out using a nonwork device. Check out Business Insider's source guide for other tips on sharing information securely.

• Amazon has faced repeated delays in launching a new AI-powered Alexa.
• Integration with partners like Uber and Ticketmaster has complicated troubleshooting processes.
• Latency and compatibility issues have also caused delays.

Amazon's Alexa seems like the perfect product for the generative AI era.

Getting this powerful technology to actually work well with the digital assistant is a monumental challenge that's been plagued by gnarly technical problems and repeated delays.

Customer-friction concerns, partnership hiccups, compatibility questions, latency problems, and accuracy issues have snarled progress, according to internal Amazon documents and multiple people involved in the project.

The Alexa team is under immense pressure to get something out. A decade ago it launched with Echo speakers and became a household name. But that early success fizzled and the business has so far failed to become profitable, leading to drastic cutbacks and layoffs in recent years.

Some company insiders consider this AI moment to be a seismic opportunity for Alexa, and potentially the last chance to reignite consumer interest in the voice assistant through the power of large language models.

A product of this scale is "unprecedented, and takes time," an Amazon spokesperson told Business Insider. "It's not as simple as overlaying an LLM onto the Alexa service."

"RED" warning

One of the main challenges facing the new Alexa relates to how the digital assistant will interact with other companies and services, and who is responsible for customers if their requests, orders, and payments don't go smoothly.

In late August, Amazon was working on integrating 8 third-party applications, including Uber and Ticketmaster, into the upcoming AI-powered Alexa to handle various user inquiries.

At that time, the goal was to launch the new Alexa around mid-October, according to one of the internal documents obtained by Business Insider. However, it was still unclear which companies would be responsible for customer support issues, like payment and delivery errors, this document stated.

The lack of clarity could cause Amazon to send "frequent" customer contacts to the partner companies. Then, those partners would sometimes redirect the users back to Amazon, the document explained.

"This level of support would cause significant customer friction, when some of the orders/purchases are time-sensitive (meal orders or rideshare trips) and purchase mistakes can be expensive (e.g. buy Taylor Swift tickets)," the document said, assigning it a "RED" warning.

Release dates pushed back

Snafus like this have caused Amazon to push back the release date, almost on a weekly basis, according to some of the people involved in the project, which has been codenamed "Banyan" or "Remarkable Alexa." BI's sources asked not to be identified because they're not authorized to talk to the press.

For example, without more clearly defined responsibilities with third-party partners, Amazon expected further delays in the launch. "Alignment on customer support plans between Product teams and the 3P partners may push this timeline further out if any delays occur," one of the documents warned.

The company had once planned for a June launch, but after repeated delays, it told employees late last month that the new Alexa would launch "no earlier" than November 20, one of the documents said.

A few of people BI spoke with recently are even talking about the Alexa upgrade rolling out in early 2025, which would miss the key holiday period. Bloomberg earlier reported on a 2025 launch plan.

As of late October, Amazon had not settled on an official brand for the updated voice assistant, and instructed employees to simply call it the "new Alexa" until further notice, one of the documents said.

Alexa's huge potential

To be sure, Alexa has significant long-term potential in the generative AI era — as long as Amazon can iron out problems relatively quickly.

Time is of the essence, partly because the existing Alexa business has lost momentum in recent years. According to a recent report from eMarketer, user growth for major voice assistants, including Alexa, has declined significantly in recent years.

The sudden rise of ChatGPT has showcased what is possible when powerful AI models are integrated smoothly with popular products that consumers and companies find useful.

Some Amazon leaders are bullish about the AI-powered Alexa and a new paid subscription service that could come with it. At least one internal estimate projected a 20% conversion rate for the paid subscription, one of the people said. That would mean that out of every 100 existing Alexa users, roughy 20 would pay for the upgraded offering. Amazon doesn't publicly disclose the number of active Alexa users but has said it has sold more than 500 million Alexa-enabled devices.

An internal description of the new Alexa shows Amazon's grand ambitions: "A personalized digital assistant that can handle a wide range of tasks, including drafting and managing personal communications, managing calendars, making reservations, placing orders, shopping, scouting for deals and events, recommending media, managing smart home devices, and answering questions on virtually any topic," one of the documents said.

Customers will be able to access the new Alexa "through natural language using voice, text, email, shared photos, and more across all their devices like Echo, Fire TV, mobile phones, and web browsers," it added.

Amazon CEO Andy Jassy shared a similar vision during last month's earnings call, saying the new Alexa will be good at not just answering questions, but also "taking actions."

In an email to BI, Amazon's spokesperson said the company's vision for Alexa is to build the world's "best personal assistant."

"Generative AI offers a huge opportunity to make Alexa even better for our customers, and we are working hard to enable even more proactive and capable assistance on the over half a billion Alexa-enabled devices already in homes around the world. We are excited about what we're building and look forward to delivering it for our customers," the spokesperson said.

Smaller AI models

Still, the project has grappled with several challenges, beyond customer friction and partnership problems.

Latency has been a particularly tough problem for the AI Alexa service. In some tests, the new Alexa took about 40 seconds to respond to a simple user inquiry, according to three people familiar with the test results. In contrast, a Google Search query takes milliseconds to respond.

To speed up, Amazon considered using a smaller AI model, like Anthropic's Claude Haiku, to power the new Alexa, one of the people said. But that dropped the quality and accuracy of the answers, leaving Amazon in limbo, this person said. In general, smaller language models generate quicker responses than larger models but can be less accurate.

Amazon had initially hoped to use a homegrown AI model, one of the people said. Last year, Alexa head scientist Rohit Prasad left the team to create a new Artificial General Intelligence group at Amazon. The stated goal of the new team was to create Amazon's "most expansive" and "most ambitious" large language models.

However, this AGI team has not produced notable results so far, which led Amazon to consider Anthropic's main Claude offering as the primary AI model for the new Alexa, this person said. Reuters previously reported that Amazon was going to mainly power Alexa with Claude.

Amazon's spokesperson said Alexa uses Amazon Web Services's Bedrock, an AI tool that gives access to multiple language models.

"When it comes to machine learning models, we start with those built by Amazon, but we have used, and will continue to use, a variety of different models — including Titan and future Amazon models, as well as those from partners — to build the best experience for customers," the spokesperson said.

The spokesperson also added a note of caution by highlighting the difficulties of successfully integrating large language models with consumer applications. These models are great for conversational dialogue and content creation, but they can also be "non-deterministic and can hallucinate," the spokesperson added.

Getting these models "to reliably act on requests (rather than simply respond) means it has to be able to call real-world APIs reliably and at scale to meet customer expectations, not just in select instances," the spokesperson explained.

New risks

In late August, Amazon discovered several new risk factors for the AI Alexa service.

Only 308 of more than 100,000 existing Alexa "skills," or voice-controlled applications, were compatible with the new Alexa, presenting a "high risk for customers to be frustrated," one of the documents explained.

Some older Echo devices would not be able to support the AI-powered Alexa, the document also warned. And there were no plans for expanding the new service to dozens of overseas markets where Alexa is currently available, leaving a large user base out of touch, it also noted. Fortune previously reported some of these risk factors.

Integration headaches

As of late August, Amazon had 8 "confirmed" partner companies to handle certain tasks for the new Alexa, as BI previously reported. The company hopes to onboard roughly 200 partners by the third year of the new Alexa's launch, one of the documents said.

Integrating with some of these companies has already created headaches. One document said that Amazon struggled to develop a consistent troubleshooting process across every partner service. Companies including Uber, Ticketmaster, and OpenTable have deprecated their existing Alexa skills, further disconnecting them from the voice assistant.

Amazon's spokesperson said that, as with any product development process, a lot of ideas are discussed and debated, but "they don't necessarily reflect what the experience will be when we roll it out for our customers."

Amazon has also anticipated customer complaints, at least in the early launch phase. One internal document from late August stated that the new Alexa was projected to receive 176,000 customer contacts in the first three months of its release. At one point, Amazon considered launching a new automated troubleshooting service for issues related to its devices and digital services, including Alexa, according to one of the internal documents. That was later shelved.

Do you work at Amazon? Got a tip?

Contact the reporter, Eugene Kim, via the encrypted-messaging apps Signal or Telegram (+1-650-942-3061) or email ([email protected]). Reach out using a nonwork device. Check out Business Insider's source guide for other tips on sharing information securely.

• The field of artificial intelligence is booming and attracting billions in investment. 
• Researchers, CEOs, and legislators are discussing how AI could transform our lives.
• Here are 17 of the major names in the field — and the opportunities and dangers they see ahead. 

Investment in artificial intelligence is rapidly growing and on track to hit $200 billion by 2025. But the dizzying pace of development also means many people wonder what it all means for their lives. 

Major business leaders and researchers in the field have weighed in by highlighting both the risks and benefits of the industry's rapid growth. Some say AI will lead to a major leap forward in the quality of human life. Others have signed a letter calling for a pause on development, testified before Congress on the long-term risks of AI, and claimed it could present a more urgent danger to the world than climate change. 

In short, AI is a hot, controversial, and murky topic. To help you cut through the frenzy, Business Insider put together a list of what leaders in the field are saying about AI — and its impact on our future. 

Geoffrey Hinton, a professor emeritus at the University of Toronto, is known as a "godfather of AI."

Hinton's research has primarily focused on neural networks, systems that learn skills by analyzing data. In 2018, he won the Turing Award, a prestigious computer science prize, along with fellow researchers Yann LeCun and Yoshua Bengio.

Hinton also worked at Google for over a decade, but quit his role at Google last spring, so he could speak more freely about the rapid development of AI technology, he said. After quitting, he even said that a part of him regrets the role he played in advancing the technology. 

"I console myself with the normal excuse: If I hadn't done it, somebody else would have. It is hard to see how you can prevent the bad actors from using it for bad things," Hinton said previously. 

Hinton has since become an outspoken advocate for AI safety and has called it a more urgent risk than climate change. He's also signed a statement about pausing AI developments for six months. 

Yoshua Bengio is a professor of computer science at the University of Montreal.

Yoshua Bengio also earned the "godfather of AI" nickname after winning the Turing Award with Geoffrey Hinton and Yann LeCun.

Bengio's research primarily focuses on artificial neural networks, deep learning, and machine learning. In 2022, Bengio became the computer scientist with the highest h-index — a metric for evaluating the cumulative impact of an author's scholarly output — in the world, according to his website. 

In addition to his academic work, Bengio also co-founded Element AI, a startup that develops AI software solutions for businesses that was acquired by the cloud company ServiceNow in 2020. 

Bengio has expressed concern about the rapid development of AI. He was one of 33,000 people who signed an open letter calling for a six-month pause on AI development. Hinton, Open AI CEO Sam Altman, and Elon Musk also signed the letter.

"Today's systems are not anywhere close to posing an existential risk," he previously said. "But in one, two, five years? There is too much uncertainty."

When that time comes, though, Bengio warns that we should also be wary of humans who have control of the technology.

Some people with "a lot of power" may want to replace humanity with machines, Bengio said at the One Young World Summit in Montreal. "Having systems that know more than most people can be dangerous in the wrong hands and create more instability at a geopolitical level, for example, or terrorism."

Sam Altman, the CEO of OpenAI, has catapulted into a major figure in the area of artificial intelligence since launching ChatGPT last November.

Altman was already a well-known name in Silicon Valley long before, having served as the president of the startup accelerator Y-Combinator 

While Altman has advocated for the benefits of AI, calling it the most tremendous "leap forward in quality of life for people" he's also spoken candidly about the risks it poses to humanity. He's testified before Congress to discuss AI regulation.

Altman has also said he loses sleep over the potential dangers of ChatGPT.

French computer scientist Yann LeCun has also been dubbed a "godfather of AI" after winning the Turing Award with Hinton and Bengio.

LeCun is professor at New York University, and also joined Meta in 2013, where he's now the Chief AI Scientist. At Meta, he has pioneered research on training machines to make predictions based on videos of everyday events as a way to enable them with a form of common sense. The idea being that humans learn an incredible amount about the world based on passive observation. He's has also published more than 180 technical papers and book chapters on topics ranging from machine learning to computer vision to neural networks, according to personal website.

LeCun has remained relatively mellow about societal risks of AI in comparison to his fellow godfathers. He's previously said that concerns that the technology could pose a threat to humanity are "preposterously ridiculous". He's also contended that AI, like ChatGPT, that's been trained on large language models still isn't as smart as dogs or cats.

Fei-Fei Li is a professor of computer science at Stanford University and a former VP at Google.

Li's research focuses on machine learning, deep learning, computer vision, and cognitively-inspired AI, according to her biography on Stanford's website.

She may be best known for establishing ImageNet — a large visual database that was designed for research in visual object recognition — and the corresponding ImageNet challenge, in which software programs compete to correctly classify objects.  Over the years, she's also been affiliated with major tech companies including Google — where she was a VP and chief scientist for AI and machine learning — and Twitter (now X), where she was on the board of directors from 2020 until Elon Musk's takeover in 2022. 

 

 

UC-Berkeley professor Stuart Russell has long been focused on the question of how AI will relate to humanity.

Russell published Human Compatible in 2019, where he explored questions of how humans and machines could co-exist, as machines become smarter by the day. Russell contended that the answer was in designing machines that were uncertain about human preferences, so they wouldn't pursue their own goals above those of humans. 

He's also the author of foundational texts in the field, including the widely used textbook "Artificial Intelligence: A Modern Approach," which he co-wrote with former UC-Berkeley faculty member Peter Norvig. 

Russell has spoken openly about what the rapid development of AI systems means for society as a whole. Last June, he also warned that AI tools like ChatGPT were "starting to hit a brick wall" in terms of how much text there was left for them to ingest. He also said that the advancements in AI could spell the end of the traditional classroom. 

Peter Norvig played a seminal role directing AI research at Google.

He spent several in the early 2000s directing the company's core search algorithms group and later moved into a role as the director of research where he oversaw teams on machine translation, speech recognition, and computer vision. 

Norvig has also rotated through several academic institutions over the years as a former faculty member at UC-Berkeley, former professor at the University of Southern California, and now, a fellow at Stanford's center for Human-Centered Artificial Intelligence. 

Norvig told BI by email that "AI research is at a very exciting moment, when we are beginning to see models that can perform well (but not perfectly) on a wide variety of general tasks." At the same time "there is a danger that these powerful AI models can be used maliciously by unscrupulous people to spread disinformation rather than information. An important area of current research is to defend against such attacks," he said. 

 

Timnit Gebru is a computer scientist who’s become known for her work in addressing bias in AI algorithms.

Gebru was a research scientist and the technical co-lead of Google's Ethical Artificial Intelligence team where she published groundbreaking research on biases in machine learning.

But her research also spun into a larger controversy that she's said ultimately led to her being let go from Google in 2020. Google didn't comment at the time.

Gebru founded the Distributed AI Research Institute in 2021 which bills itself as a "space for independent, community-rooted AI research, free from Big Tech's pervasive influence."

She's also warned that AI gold rush will mean companies may neglect implementing necessary guardrails around the technology. "Unless there is external pressure to do something different, companies are not just going to self-regulate," Gebru previously said. "We need regulation and we need something better than just a profit motive."

 

British-American computer scientist Andrew Ng founded a massive deep learning project called "Google Brain" in 2011.

The endeavor lead to the Google Cat Project: A milestone in deep learning research in which a massive neural network was trained to detect YouTube videos of cats.

Ng also served as the chief scientist at Chinese technology company Baidu where drove AI strategy. Over the course of his career, he's authored more than 200 research papers on topics ranging from machine learning to robotics, according to his personal website. 

Beyond his own research, Ng has pioneered developments in online education. He co-founded Coursera along with computer scientist Daphne Koller in 2012, and five years later, founded the education technology company DeepLearning.AI, which has created AI programs on Coursera.  

"I think AI does have risk. There is bias, fairness, concentration of power, amplifying toxic speech, generating toxic speech, job displacement. There are real risks," he told Bloomberg Technology last May. However, he said he's not convinced that AI will pose some sort of existential risk to humanity — it's more likely to be part of the solution. "If you want humanity to survive and thrive for the next thousand years, I would much rather make AI go faster to help us solve these problems rather than slow AI down," Ng told Bloomberg. 

 

Daphne Koller is the founder and CEO of insitro, a drug discovery startup that uses machine learning.

Koller told BI by email that insitro is applying AI and machine learning to advance understanding of "human disease biology and identify meaningful therapeutic interventions." And before founding insitro, Koller was the chief computing officer at Calico, Google's life-extension spinoff. Koller is a decorated academic, a MacArthur Fellow, and author of more than 300 publications with an h-index of over 145, according to her biography from the Broad Institute, and co-founder of Coursera.  

In Koller's view the biggest risks that AI development pose to society are "the expected reduction in demand for certain job categories; the further fraying of "truth" due to the increasing challenge in being able to distinguish real from fake; and the way in which AI enables people to do bad things."

At the same time, she said the benefits are too many and too large to note. "AI will accelerate science, personalize education, help identify new therapeutic interventions, and many more," Koller wrote by email.

Daniela Amodei cofounded AI startup Anthropic in 2021 after an exit from OpenAI.

Amodei co-founded Anthropic along with six other OpenAI employees, including her brother Dario Amodei. They left, in part, because Dario — OpenAI's lead safety researcher at the time — was concerned that OpenAI's deal with Microsoft would force it to release products too quickly, and without proper guardrails. 

At Anthropic, Amodei is focused on ensuring trust and safety. The company's chatbot Claude bills itself as an easier-to-use alternative that OpenAI's ChatGPT, and is already being implemented by companies like Quora and Notion. Anthropic relies on what it calls a "Triple H" framework in its research. That stands for Helpful, Honest, and Harmless. That means it relies on human input when training its models, including constitutional AI, in which a customer outlines basic principles on how AI should operate. 

"We all have to simultaneously be looking at the problems of today and really thinking about how to make tractable progress on them while also having an eye on the future of problems that are coming down the pike," Amodei previously told BI.

 

Demis Hassabis has said artificial general intelligence will be here in a few years.

Hassabis, a former child chess prodigy who studied at Cambridge and University College London, was nicknamed the "superhero of artificial intelligence" by The Guardian back in 2016. 

After a handful of research stints, and a venture in videogames, he founded DeepMind in 2010. He sold the AI lab to Google in 2014 for £400 million where he's worked on algorithms to tackle issues in healthcare, climate change, and also launched a research unit dedicated to the understanding the ethical and social impact of AI in 2017, according to DeepMind's website. 

Hassabis has said the promise of artificial general intelligence — a theoretical concept that sees AI matching the cognitive abilities of humans — is around the corner. "I think we'll have very capable, very general systems in the next few years," Hassabis said previously, adding that he didn't see why AI progress would slow down anytime soon. He added, however, that developing AGI should be executed in a "in a cautious manner using the scientific method." 

In 2022, DeepMind co-founder Mustafa Suleyman launched AI startup Inflection AI along with LinkedIn co-founder Reid Hoffman, and Karén Simonyan — now the company's chief scientist.

The startup, which claims to create "a personal AI for everyone," most recently raised $1.3 billion in funding last June, according to PitchBook. 

Its chatbot, Pi, which stands for personal intelligence, is trained on large language models similar to OpenAI's ChatGPT or Bard. Pi, however, is designed to be more conversational, and offer emotional support. Suleyman previously described it as a "neutral listener" that can respond to real-life problems. 

"Many people feel like they just want to be heard, and they just want a tool that reflects back what they said to demonstrate they have actually been heard," Suleyman previously said. 

 

 

USC Professor Kate Crawford focuses on social and political implications of large-scale AI systems.

Crawford is also the senior principal researcher at Microsoft, and the author of Atlas of AI, a book that draws upon the breadth of her research to uncover how AI is shaping society. 

Crawford remains both optimistic and cautious about the state of AI development. She told BI by email she's excited about the people she works with across the world "who are committed to more sustainable, consent-based, and equitable approaches to using generative AI."

She added, however, that "if we don't approach AI development with care and caution, and without the right regulatory safeguards, it could produce extreme concentrations of power, with dangerously anti-democratic effects."

Margaret Mitchell is the chief ethics scientist at Hugging Face.

Mitchell has published more than 100 papers over the course of her career, according to her website, and spearheaded AI projects across various big tech companies including Microsoft and Google. 

In late 2020, Mitchell and Timnit Gebru — then the co-lead of Google's ethical artificial intelligence — published a paper on the dangers of large language models. The paper spurred disagreements between the researchers and Google's management and ultimately lead to Gebru's departure from the company in December 2020. Mitchell was terminated by Google just two months later, in February 2021

Now, at Hugging Face — an open-source data science and machine learning platform that was founded in 2016 — she's thinking about how to democratize access to the tools necessary to building and deploying large-scale AI models.  

In an interview with Morning Brew, where Mitchell explained what it means to design responsible AI, she said, "I started on my path toward working on what's now called AI in 2004, specifically with an interest in aligning AI closer to human behavior. Over time, that's evolved to become less about mimicking humans and more about accounting for human behavior and working with humans in assistive and augmentative ways."

Navrina Singh is the founder of Credo AI, an AI governance platform.

Credo AI is a platform that helps companies make sure they're in compliance with the growing body of regulations around AI usage. In a statement to BI, Singh said that by automating the systems that shape our lives, AI has the capacity "free us to realize our potential in every area where it's implemented."

At the same time, she contends that algorithms right now lack the human judgement that's necessary to adapt to a changing world. "As we integrate AI into civilization's fundamental infrastructure, these tradeoffs take on existential implications," Singh wrote. "As we forge ahead, the responsibility to harmonize human values and ingenuity with algorithmic precision is non-negotiable. Responsible AI governance is paramount."

 

Richard Socher, a former Salesforce exec, is the founder and CEO of AI-powered search engine You.com.

Socher believes we have ways to go before AI development hits its peak or matches anything close to human intelligence.

One bottleneck in large language models is their tendency to hallucinate — a phenomenon where they convincingly spit out factual errors as truth. But by forcing them to translate questions into code — essential "program" responses instead of verbalizing them — we can "give them so much more fuel for the next few years in terms of what they can do," Socher said. 

But that's just a short-term goal. Socher contends that we are years from anything close to the industry's ambitious bid to create artificial general intelligence. Socher defines it as "a form of intelligence that can "learn like humans" and "visually have the same motor intelligence, and visual intelligence, language intelligence, and logical intelligence as some of the most logical people," and it could take as little as 10 years, but as much as 200 years to get there. 

And if we really want to move the needle toward AGI, Socher said humans might need to let go of the reins, and their own motives to turn a profit, and build AI that can set its own goals.

"I think it's an important part of intelligence to not just robotically, mechanically, do the same thing over and over that you're told to do. I think we would not call an entity very intelligent if all it can do is exactly what is programmed as its goal," he told BI. 

• The United Arab Emirates wants to become an AI leader by 2031.
• It's leveraging its oil wealth to attract new talent and fund new research initiatives.
• The UAE's AI minister believes we'll have "centers and nodes of excellence across the world."

The AI revolution is expanding far beyond Silicon Valley.

From the shores of Malta to the streets of Paris, hubs for AI innovation are forming worldwide. And the United Arab Emirates is emerging as a key center in the Middle East.

In October, the UAE made headlines by participating in the most lucrative funding round in Silicon Valley history: the $6.6 billion deal closed by OpenAI. The investment was made through MGX, a state-backed technology firm focused on artificial intelligence and semiconductors.

The move was part of the UAE's bid to become a global AI leader by 2031 through strategic initiatives, public engagement, and research investment. Last year, the country's wealthiest emirate, Abu Dhabi, launched Falcon — its first open-source large language model. State-backed AI firm G42 is also training large language models in Arabic and Hindi to bridge the gap between English-based models and native speakers of these languages.

Another indication of the UAE's commitment to AI is its appointment of Omar Sultan Al Olama as the country's AI Minister in 2017.

The minister acknowledges that the UAE faces tough competition from powerhouses like the United States and China, where private investment in AI technology in 2023 totaled $67.2 billion and $7.8 billion, respectively, according to Stanford's Center for Human-Centered Artificial Intelligence.

So he says he is embracing cooperation over competition.

"I don't think it's going to be a zero-sum game where it's only going to be AI that's developed in the US, or only going to be AI that's developed in China or the UAE," Al Olama said at an event hosted by the Atlantic Council, a DC think tank, in April. "What is going to happen, I think, is that we're going to have centers and nodes of excellence across the world where there are specific use cases or specific domains where a country or player or a company is doing better than everyone else."

The UAE's strengths are evident.

It is one of the wealthiest countries in the world, mostly due to its vast oil reserves. The UAE is among the world's 10 largest oil producers, with 96% of that coming from its wealthiest emirate, Abu Dhabi, according to the International Trade Administration.

Abu Dhabi's ruling family also controls several of the world's largest sovereign wealth funds, including the Abu Dhabi Investment Authority and Mubadala Investment Company, a founding partner of MGX.

These funds have been used to diversify the country's oil wealth and could now be diverted to funding new AI companies. AI could contribute $96 billion to the UAE economy by 2030, making up about 13.6% of its GDP, according to a report by PwC, the accounting firm.

But capital is only part of the equation. The bigger question is whether the tiny Gulf nation can attract the requisite talent to keep up with Silicon Valley.

Recent developments show promise. Between 2021 and 2023, the number of AI workers in the UAE quadrupled to 120,000, Al Olama said at the Atlantic Council event. In 2019, it rolled out a 'golden visa' program for IT professionals, making entry easier for AI experts. It's also making the most of its existing talent. In May, Dubai launched the world's biggest prompt engineering initiative. Its goal is to upskill 1 million workers over the next three years.

However, it's also faced criticism for its treatment of workers, especially lower-skilled migrant workers. Migrant workers comprise 88% of the country's population and have been subject to a range of labor abuses, including exposure to extreme heat, exploitative recruitment fees, and wage theft, according to Human Rights Watch. The UAE has responded by passing several labor laws that address protections for workers around hours, wages, and competition.

Abu Dhabi, meanwhile, has — over the last decade — become a nexus for AI research and education.

In 2010, New York University launched a branch in Abu Dhabi that has since developed a focus on AI. And, in 2019, Mohamed bin Zayed University of Artificial Intelligence opened as a "graduate research university dedicated to advancing AI as a global force for good." Professors from the university also helped organize the inaugural International Olympiad in Artificial Intelligence in August, which drew students from over 40 countries worldwide.

"Abu Dhabi may not directly surpass Silicon Valley, however, it has the potential to become a significant AI hub in its own right," Nancy Gleason, an advisor to leadership on AI at NYU Abu Dhabi and a professor of political science, told Business Insider by email. Its "true strengths lie in the leadership's strategic vision, substantial investments in AI research and compute capacity, and government-led initiatives in industry. The UAE has also made strategic educational investments in higher education like the Mohamed bin Zayed University of Artificial Intelligence and NYU Abu Dhabi."

Beyond that, she noted, it's "very nice to live here."

• Modern factories, supply chains and Amazon have turned 'stuff' into a commodity. 
• The same inevitable supply-and-demand dynamic could wash over us again with generative AI.
• The ultimate outcome may be a new limited-edition luxury item: Humans. 

"Live experiences are the new luxury good," Kevin Hartz said in 2013 when Eventbrite, the ticketing startup he cofounded, got a big new funding round.

By that point, modern factories, supply chains, and Amazon had boiled down "stuff" to a commodity. You can now buy an overwhelming variety of tennis shoes, or spatulas, or sweatpants online. This abundance has taken much of the satisfaction away from purchasing physical things. This is why experiences, which by definition are finite, became more valuable.

There are only a few opportunities to see Taylor Swift on stage, versus the availability to purchase more than 20,000 kinds of tennis shoes on Amazon. So the price of Eras tickets soar, and shoes are cheap.

The same inevitable supply-and-demand dynamic is about to wash over us again with large language models and generative AI.

The ultimate outcome could be a new limited-edition luxury item: Humans. 

Unlimited content vs 'finite resources'

AI models can now automatically generate text, software code, medical diagnoses, images, voices, music, video, and lots more. The barriers to using this technology are falling away quickly. Anyone can fire up ChatGPT, GPT-4, DALL-E and other tools to produce an almost unlimited quantity of content.

This should be a boon to society. Many tasks will be completed more efficiently, making products and services more affordable and accessible, as venture capitalist Marc Andreessen recently explained.

There will be a reaction though: In a world of machine-generated abundance, human-centered services and experiences will become increasingly rare, valuable, and therefore desirable.

"The world's information is being turned into 1s and 0s and all this is being commoditized," Hartz told BI. "What can't be commoditized is finite resources like real estate, travel, seeing the sunset on Mediterranean, or surfing in Fiji. These are the luxury goods of the power elite."

Cooks, tutors, and robo-advisors

The more that AI automates restaurants, the more we'll want personal chefs such as John Barone, who cooks five days a week in the home of a wealthy Silicon Valley couple.

As AI tutor bots proliferate in education, the richest will pay for more exclusive access to the best human tutors for their kids.

The more robo-advisors handle our money, the stronger the urge of the wealthy to recruit savvy human experts to manage their family offices.

A new flood of automated emails

Email marketing is a simple example that some technologists are already worried about.

Generative AI tools are making it much quicker and easier to write marketing copy. The end result will be a flood of new emails that will overwhelm recipients and make them even less likely to open the messages.

"And our own machines will read those AI automated sales emails," Hartz quipped.

So, either your marketing email won't reach the humans you're trying to engage, or another AI bot will open it and you'll never be quite sure who read the message. A hand-typed email from a real human will be, relatively speaking, a rare and beautiful thing (complete with typos).

AI tutors versus small classrooms

AI models are beginning to revolutionize education, according to Sal Khan, the founder of Khan Academy. His organization has been working with OpenAI models to coach students in powerful new ways and help teachers develop class plans.

The gold standard throughout history has always been to have a personal tutor, and AI models can help personalize the education experience to bring some of this curated approach to more students, he explained during a No Priors podcast earlier this year.

"We don't have the resources to give everyone a tutor," he said during the podcast. "A generative AI tutor supporting students. That's going to be mainstream in 3 to 5 years," he added.

Pricey schools and a personal carpenter

And yet, Silicon Valley's top private schools, where many tech execs send their kids, are all about getting access to human teachers in small group settings. 

Castillja in Palo Alto highlights a student to faculty ratio of 7 to 1. Nueva, a Silicon Valley school for gifted kids, promises a similar ratio. The Menlo School in Menlo Park says it has a student-teacher ratio of 10 to 1 in the upper school.

These institutions cost $58,000 to $60,000 a year and I don't see any drop-off in demand among the tech elite. They're still jostling to get their kids into these bespoke, human-centered learning environments.

One persistent, apocryphal Silicon Valley story illustrates this point. On weekends, one tech billionaire has been known to hire a personal carpenter to hand-make wooden toys for their kids build and play with.

Who manages the money?

What about when it comes to managing fortunes amassed by successful tech entrepreneurs? The wealthiest rely on talented financial advisors who are hired directly to oversee this money in family offices.

Bill Gates has his own private investment firm, Cascade, which has been run by money manager Michael Larson since 1994. Elon Musk's family office, Excession, has been run by a former Morgan Stanley banker called Jared Birchall for years.

Using AI for trading has been tough so far. AI models are trained on masses of data from the past. When new situations arise, they struggle to adapt quickly enough.

Even quantitative hedge fund firms, which use machine learning and other automated techniques, rely on humans. Two Sigma, a famous quant firm, is for the first time exploring ways to add traders who rely on their human judgment to make money, Bloomberg reported recently.

"The major challenge with using things like reinforcement learning for trading is that it's a non-stationary environment," AI researcher Noam Brown said on the No Priors podcast in April. He's worked on algorithmic trading strategies in the past and was a researcher at Meta before recently joining OpenAI.

"So you can have all this historical data but it's not a stationary system," he explained, referring to how markets respond swiftly to world events and other developments.

Part of the problem relates to what he calls sample efficiency. Humans are good at learning quickly from a small amount of data, while AI models need mountains of information to train on.

"Humans are very good at adapting to novel situations," he added. "And you run into these novel situations pretty frequently in financial markets."

Social media bots vs. martial arts

AI is making social media increasingly machine-driven, too. Soon, human content creators will be vying for attention with content generated by AI models.

Last month, Meta CEO Mark Zuckerberg unveiled more than 25 new AI assistants with different personalities that use celebrities' images. Users will be able to interact with these bots on Meta's platforms in the future.

In a recent podcast, he described this new supply-and-demand situation well, saying human creators can't keep up with demand from followers.

"There are both people who out there who would benefit from being able to talk to an AI version of you," Zuckerberg explained. "You and other creators would benefit from being able to keep your community engaged."

So Meta will make an AI version of celebrities that can post constantly. Again, this will be infinite. And actually interacting with the real human celebrity will become more rare and valuable.

Meanwhile, when Zuckerberg is relaxing outside of work, he spends some of that time pursuing a very human pastime: Rolling around with other humans in martial arts contests.

Medical models and human doctors

AI models, such as Google DeepMind's Med-PaLM 2, are becoming incredibly good at answering medical questions and analyzing x-rays and other health data. But when wealthy parents have really sick children, they will still seek out the smartest doctors in the relevant fields of medicine.

You can see this in Silicon Valley's embrace of medical concierge services that provide special access to doctors and other human health specialists.

One Medical succeeded by offering better access to human doctors, and Amazon ended up buying it for almost $4 billion.

"We're inspired by their human-centered, technology-forward approach," an Amazon executive said when the deal was announced.

'Utility, value and signaling'

Hartz, a venture capitalist who now chairs Eventbrite's board, says successful technologists will continue to spend heavily on human experiences. But he says this depends on the activity and the motivations behind different actions.

He breaks this into "utility, value and signaling."

Many standard, common situations can be handled by software bots or even physical machines. Repetitive tasks at work and some educational functions are examples of these utility-type solutions.

In other situations, users will get more value from having machines handle the work, so humans can focus on more valuable tasks. If you're a well-paid machine-learning engineer, it will be better to have a robot clean your house so you can focus more on your job, he explained.

And then there will still many situations where humans will want to enjoy their success and signal the fruits of their achievements. And these activities will increasingly focus on finite human resources and experiences, Hartz said.

"You can't put on headset and pretend to be in Fiji," he added.

• AI leaders are rethinking data-heavy training for large language models.
• Traditional models scale linearly with data, but this approach may hit a dead end.
• Smaller, more efficient models and new training methods are gaining industry support.

For years, tech companies like OpenAI, Meta, and Google have focused on amassing tons of data, assuming that more training material would lead to smarter, more powerful models.

Now, AI leaders are rethinking the conventional wisdom about how to train large language models.

The focus on training data arises from research showing that transformers, the neural networks behind large language models, have a one-to-one relationship with the amount of data they're given. Transformer models "scale quite linearly with the amount of data and compute they're given," Alex Voica, a consultant at the Mohamed bin Zayed University of Artificial Intelligence, previously told Business Insider.

However, executives are starting to worry that this approach can only go so far, and they're exploring alternatives for advancing the technology.

The money going into AI has largely hung on the idea that this scaling law "would hold," Scale AI CEO Alexandr Wang said at the Cerebral Valley conference this week, tech newsletter Command Line reported. It's now "the biggest question in the industry."

Some executives say the problem with the approach is that it's a little mindless. "It's definitely true that if you throw more compute at the model, if you make the model bigger, it'll get better," Aidan Gomez, the CEO of Cohere, said on the 20VC podcast. "It's kind of like it's the most trustworthy way to improve models. It's also the dumbest."

Gomez advocates smaller, more efficient models, which are gaining industry support for being cost-effective.

Others worry this approach won't reach artificial general intelligence — a theoretical form of AI that matches or surpasses human intelligence — even though many of the world's largest AI companies are banking on it.

Large language models are trained simply to "predict the next token, given the previous set of tokens," Richard Socher, a former Salesforce executive and CEO of AI-powered search engine You.com, told Business Insider. The more effective way to train them is to "force" these models to translate questions into computer code and generate an answer based on the output of that code, he said. This will reduce hallucinations in quantitative questions and enhance their abilities.

Not all industry leaders are sold that AI has hit a scaling wall, however.

"Despite what other people think, we're not at diminishing marginal returns on scale-up," Microsoft chief technology officer Kevin Scott said in July in an interview with Sequoia Capital's Training Data podcast.

Companies like OpenAI are also seeking to improve on existing LLMs.

OpenAI's o1, released in September, still relies on the token prediction mechanism Socher refers to. Still, the model is specialized to better handle quantitative questions, including areas like coding and mathematics — compared to ChatGPT, which is considered a more general-purpose model.

Part of the difference between o1 and ChatGPT is that o1 spends more time on inference or "thinking" before it answers a question.

"To summarize, if we were to anthropomorphize, gpt-4 is like your super know-it-all friend who when you ask them a question starts talking stream-of-consciousness, forcing you to sift through what they're saying for the gems," Waleed Kadous, a former engineer lead at Uber and former Google principal software engineer, wrote in a blog post. "o1 is more like the friend who listens carefully to what you have to say, scratches their chin for a few moments, and then shares a couple of sentences that hit the nail on the head."

One of o1's trade-offs, however, is that it requires much more computational power, making it slower and costlier, according to Artificial Analysis, an independent AI benchmarking website.

• Amazon on Friday announced another $4 billion investment in the AI startup Anthropic.
• The deal includes an agreement for Anthropic to use Amazon's AI chips more.
• The cloud giant is trying to challenge Nvidia and get developers to switch away from those GPUs.

Amazon's Trainium chips are about to get a lot busier — at least that's what Amazon hopes will happen after it pumps another $4 billion into the AI startup Anthropic.

The companies announced a huge new deal on Friday that brings Amazon's total investment in Anthropic to $8 billion. The goal of all this money is mainly to get Amazon's AI chips to be used more often to train and run large language models.

Anthropic said that in return for this cash injection, it would use AWS as its "primary cloud and training partner." It said it would also help Amazon design future Trainium chips and contribute to building out an Amazon AI-model-development platform called AWS Neuron.

This is an all-out assault on Nvidia, which dominates the AI chip market with its GPUs, servers, and CUDA platform. Nvidia's stock dropped by more than 3% on Friday after the Amazon-Anthropic news broke.

The challenge will be getting Anthropic to actually use Trainium chips in big ways. Switching away from Nvidia GPUs is complicated, time-consuming, and risky for AI-model developers, and Amazon has struggled with this.

Earlier this week, Anthropic CEO Dario Amodei didn't sound like he was all in on Amazon's Trainium chips, despite another $4 billion coming his way.

"We use Nvidia, but we also use custom chips from both Google and Amazon," he said at the Cerebral Valley tech conference in San Francisco. "Different chips have different trade-offs. I think we're getting value from all of them."

In 2023, Amazon made its first investment in Anthropic, agreeing to put in $4 billion. That deal came with similar strings attached. At the time, Anthropic said that it would use Amazon's Trainium and Inferentia chips to build, train, and deploy future AI models and that the companies would collaborate on the development of chip technology.

It's unclear whether Anthropic followed through. The Information reported recently that Anthropic preferred to use Nvidia GPUs rather than Amazon AI chips. The publication said the talks about this latest investment focused on getting Anthropic more committed to using Amazon's offerings.

There are signs that Anthropic could be more committed now, after getting another $4 billion from Amazon.

In Friday's announcement, Anthropic said it was working with Amazon on its Neuron software, which offers the crucial connective tissue between the chip and the AI models. This competes with Nvidia's CUDA software stack, which is the real enabler of Nvidia's GPUs and makes these components very hard to swap out for other chips. Nvidia has a decadelong head start on CUDA, and competitors have found that difficult to overcome.

Anthropic's "deep technical collaboration" suggests a new level of commitment to using and improving Amazon's Trainium chips.

Though several companies make chips that compete with or even beat Nvidia's in certain elements of computing performance, no other chip has touched the company in terms of market or mind share.

Amazon's AI chip journey

Amazon is on a short list of cloud providers attempting to stock their data centers with their own AI chips and avoid spending heavily on Nvidia GPUs, which have profit margins that often exceed 70%.

Amazon debuted its Trainium and Inferentia chips — named after the training and inference tasks they're built for — in 2020.

The aim was to become less dependent on Nvidia and find a way to make cloud computing in the AI age cheaper.

"As customers approach higher scale in their implementations, they realize quickly that AI can get costly," Amazon CEO Andy Jassy said on the company's October earnings call. "It's why we've invested in our own custom silicon in Trainium for training and Inferentia for inference."

But like its many competitors, Amazon has found that breaking the industry's preference for Nvidia is difficult. Some say that's because of CUDA, which offers an abundant software stack with libraries, tools, and troubleshooting help galore. Others say it's simple habit or convention.

In May, the Bernstein analyst Stacy Rasgon told Business Insider he wasn't aware of any companies using Amazon AI chips at scale.

With Friday's announcement, that might change.

Jassy said in October that the next-generation Trainium 2 chip was ramping up. "We're seeing significant interest in these chips, and we've gone back to our manufacturing partners multiple times to produce much more than we'd originally planned," Jassy said.

Still, Anthropic's Amodei sounded this week like he was hedging his bets.

"We believe that our mission is best served by being an independent company," he said. "If you look at our position in the market and what we've been able to do, the independent partnerships we have Google, with Amazon, with others, I think this is very viable."