Large language models represent text using tokens, each of which is a few characters. Short words are represented by a single token (like "the" or "it"), whereas larger words may be represented by several tokens (GPT-4o represents "indivisible" with "ind," "iv," and "isible").
When OpenAI released ChatGPT two years ago, it had a memory—known as a context window—of just 8,192 tokens. That works out to roughly 6,000 words of text. This meant that if you fed it more than about 15 pages of text, it would “forget” information from the beginning of its context. This limited the size and complexity of tasks ChatGPT could handle.
AI training data has a big price tag, one best-suited for deep-pocketed tech firms. This is why Harvard University plans to release a dataset that includes in the region of 1 million public-domain books, spanning genres, languages, and authors including Dickens, Dante, and Shakespeare, which are no longer copyright-protected due to their age. The new […]
Google reportedly wants the US Federal Trade Commission (FTC) to end Microsoft's exclusive cloud deal with OpenAI that requires anyone wanting access to OpenAI's models to go through Microsoft's servers.
Someone "directly involved" in Google's effort told The Information that Google's request came after the FTC began broadly probing how Microsoft's cloud computing business practices may be harming competition.
As part of the FTC's investigation, the agency apparently asked Microsoft's biggest rivals if the exclusive OpenAI deal was "preventing them from competing in the burgeoning artificial intelligence market," multiple sources told The Information. Google reportedly was among those arguing that the deal harms competition by saddling rivals with extra costs and blocking them from hosting OpenAI's latest models themselves.
Alexis Conneau thinks a lot about the movie “Her.” For the last several years, he’s obsessed over trying to turn the film’s fictional voice technology, Samantha, into a reality. Conneau even uses a picture of Joaquin Phoenix’s character in the movie as his banner on Twitter. With ChatGPT’s Advanced Voice Mode, a project Conneau started […]
Software engineer Vishnu Mohandas decided he would quit Google in more ways than one when he learned that the tech giant had briefly helped the US military develop AI to study drone footage. In 2020 he left his job working on Google Assistant and also stopped backing up all of his images to Google Photos. He feared that his content could be used to train AI systems, even if they weren’t specifically ones tied to the Pentagon project. “I don't control any of the future outcomes that this will enable,” Mohandas thought. “So now, shouldn't I be more responsible?”
Mohandas, who taught himself programming and is based in Bengaluru, India, decided he wanted to develop an alternative service for storing and sharing photos that is open source and end-to-end encrypted. Something “more private, wholesome, and trustworthy,” he says. The paid service he designed, Ente, is profitable and says it has more than 100,000 users, many of whom are already part of the privacy-obsessed crowd. But Mohandas struggled to articulate to wider audiences why they should reconsider relying on Google Photos, despite all the conveniences it offers.
Then one weekend in May, an intern at Ente came up with an idea: Give people a sense of what some of Google’s AI models can learn from studying images. Last month, Ente launched https://Theyseeyourphotos.com, a website and marketing stunt designed to turn Google’s technology against itself. People can upload any photo to the website, which is then sent to a Google Cloud computer vision program that writes a startlingly thorough three-paragraph description of it. (Ente prompts the AI model to document small details in the uploaded images.)
On Friday, Anthropic announced that Amazon has increased its investment in the AI startup by $4 billion, bringing its total stake to $8 billion while maintaining its minority investor position. Anthropic makes Claude, an AI assistant rival to OpenAI's ChatGPT.
One reason behind the deal involves chips. The computing demands of training large AI models have made access to specialized processors a requirement for AI companies. While Nvidia currently dominates the AI chip market with customers that include most major tech companies, some cloud providers like Amazon have begun developing their own AI-specific processors.
Under the agreement, Anthropic will train and deploy its foundation models using Amazon's custom-built Trainium (for training AI models) and its Inferentia chips (for AI inference, the term for running trained models). The company will also work with Amazon's Annapurna Labs division to advance processor development for AI applications.
AI is finding its way into every corner of biotech and pharmaceutical research, but like other industries, it’s never quite as straightforward to implement as one would like. Converge Bio has built a tool for companies to make their biology-focused LLMs actually work, from “enriching” their data to explaining their answers. The company has raised […]
