Reading view

There are new articles available, click to refresh the page.

AI-Generated Book Grifters Threaten The Future of Lace-Making

AI-Generated Book Grifters Threaten The Future of Lace-Making

AI-generated books and images are threatening the nearly 500-year-old art of lace making. 

It’s already come for the crochet community, and researchers have tried to teach machines to knit. But lace-making—a craft that even Renaissance artists struggled to master, and in which there are a literal infinite number of patterns to be created—is now having its AI slop moment. 

Mary Mangan, the librarian for her New England-based lace making group, told me that she first became aware of AI infiltrating lace spaces when someone in her group asked her to research a book that featured a cover photo that she wanted to try to make herself. “So I began to research the book. It smelled funny and I tried to search for the author's other work but couldn't find any,” Mangan said. She asked r/BobbinLace, a Reddit community for the bobbin lace-making technique, and users there helped track down the original, not-AI image from a lace catalog that the cover photo seemed to be based on. 

Longtime lace makers and experts from all around the community have started raising the alarm on AI grifting in their tight-knit community. Karen Bovard-Sayre, who has published several books about lace techniques, posted a video in November addressing the issue, saying she found 36 books about lace and tatting—a lacemaking technique—that seemed AI generated. She said she was looking at Amazon books about tatting to see what else was being published on the topic, and found many of the AI books targeting beginners. 

“As you probably all know, the tatting world's not that big even though it's around the world, but we kind of know who's doing what, who's making content, who's making books and all that,” Bovard-Sayre said in her video. “I started reading the summaries and they all kind of sounded flowery and didn't really say what they were, and then I started looking at the covers and back covers, and said wait a minute, something's wrong here.” She spends the rest of the video demonstrating what these books get wrong, and how to spot AI generated lace making materials. 

Some of the AI signs Bovard-Sayre points out include odd punctuation in the authors’ name (in the case of the book she’s examining in her video, “Sheila .A Richard,” where there’s a period before the middle initial), references to video tutorials like “This is a wonderful instructional video” which makes no sense in a printed book, obvious misspellings, and distorted or blurry photos.

She also finds designs in the book that she recognized as being the work of other lace designers, including Marilee Rockley, a fiber artist who specializes in tatting. Rockley also recently addressed the rise in AI generated materials on her website. “Some of you may have heard about the miserable thieves who are using Artificial Intelligence technology to ‘make’ books to sell,” she wrote. “Really horrible, fake books loaded with wrong information (lies) and stolen photos. They're so bad it would be laughable except they hurt a lot of innocent people who are looking to learn a new-to-them craft.” 

Preying on beginners’ lack of knowledge and relative inability to spot blatant fakes is a tactic used in other AI book grifts, too. The mushroom foraging community recently discovered AI scam books were flooding Amazon, directing newcomers to bad, potentially deadly misinformation. Unlike eating a poisonous mushroom because a chatbot or AI book told you it’s safe, buying a book on lacemaking that contains sloppily-generated images or instructions isn’t a matter of life and death—but it does threaten to devalue and dilute the integrity of a centuries-old art, as well as deterring newcomers. 

“Lace is a small hobby and a pretty tight community. We know who the designers and vendors are, and we trust them. However, until you become part of the lace community there's no way to know who is trustworthy and what is dubious. You need some level of skill and time within the network to really assess this,” Mangan told me. “Unfortunately, for newcomers who might be excited to dive into this hobby, they could get burned by the inadequate books—and frankly the thievery—of the work of our cherished lacemakers and designers. This could sour newbies on the craft and that would be unfortunate. And it could harm designers who opt out of sharing their works, and we'll all lose then.”

Lacemaker and textile historian Elena Kanagy-Loux told me she first noticed the proliferation of AI-generated books on bobbin lace while teaching a course last summer. A student showed her a book she’d recommended to her students on Amazon, but the recommended books on the site seemed off. “There were a number of suggested lace books with strange covers that did not represent real lace techniques, and subsequently I have been warning all of my students to avoid Amazon and buy from independent lace suppliers (a good practice for a multitude of reasons),” she said. “Now I see that there are a number of them advertising different lace techniques with strange AI images on the cover that don’t represent real lace or tools, and contents that—according to reviews—are either nonsense that provide no tangible instructions, or directly plagiarized from real lace books.” 

Some of the books Elena Kanagy-Loux found on Amazon included: 

I sent all of the above listings to Amazon for comment, and the platform removed all of them except for the first one. “We have content guidelines governing which books can be listed for sale, and we have proactive and reactive methods that help us detect content that violates our guidelines, whether AI-generated or not. We invest significant time and resources to ensure our guidelines are followed, and remove books that do not adhere to those guidelines," a spokesperson for Amazon told me in a statement. "We aim to provide the best possible shopping, reading, and publishing experience, and we are constantly evaluating developments that impact that experience, which includes the rapid evolution and expansion of generative AI tools. We continue to enhance our protections against non-compliant content, and our process and guidelines will keep evolving as we see changes in AI-driven publishing.”

Amazon is full of these books, but it’s not the only retailer selling them. Mangan showed me several she and others found on eBay, including Bobbin Lace Magic: Unlocking the Secrets of Colorous Book by Ethan CC Lee which, like the ones above, has a book-report description as if the author is reviewing their own book. And then there’s A Bobbin Lace Book by Tim M. Enoch, with a description that includes an error from generating the text: “This response was truncated by the cut-off limit (max tokens). Open the sidebar, Increase the parameter in the settings and then regenerate.” eBay did not respond to a request for comment.

Mangan wondered if the onslaught of AI-generated slop in lacemaking might drive people to connect to real humans more. “Gathering in groups and discussing valuable books might be a good outcome, and we can host public gatherings for the lace-curious folks,” she said. “One other thing that I do is to edit Wikipedia with good books as references when I hear about them—maybe that could become another route to connect people to higher quality and current materials.” Used and older books could become more valuable, too, she said. 

“Over the years of posting videos about lacemaking on social media, I have gotten many snarky comments saying ‘AI will replace this.’ At first I laughed it off, because for lacemakers like myself the joy is in the process of working with our hands, which can never be replaced by technology,” Kanagy-Loux said. “But now I have genuine concerns that beginners seeking affordable books will be scammed by AI-generated books that contain no real information about the techniques and give up in frustration. This misinformation is why it is so important to me to share resources online and make knowledge about lacemaking and lace history accessible to a broader audience. Fortunately, our community continues to grow all the time, so I hope we can combat the proliferation of AI pattern books with the instructions of human beings.”

Podcast: Why We Cover AI the Way We Do

Podcast: Why We Cover AI the Way We Do

Hello! Here's a holiday gift: an episode of the 404 Media Podcast that was previously only for paying subscribers! It gives a lot more context on the how and why we cover AI they way we do. Here's the original description of the episode:

We got a lot of, let's say, feedback, with some of our recent stories on artificial intelligence. One was about people using Bing's AI to create images of cartoon characters flying a plane into a pair of skyscrapers. Another was about 4chan using the same tech to quickly generate racist images. Here, we use that dialogue as a springboard to chat about why we cover AI the way we do, the purpose of journalism, and how that relates to AI and tech overall. This was fun, and let us know what you think. Definitely happy to do more of these sorts of discussions for our subscribers in the future.

Listen to the weekly podcast on Apple Podcasts, Spotify, or YouTube. Become a paid subscriber for access to this episode's bonus content and to power our journalism. If you become a paid subscriber, check your inbox for an email from our podcast host Transistor for a link to the subscribers-only version! You can also add that subscribers feed to your podcast app of choice and never miss an episode that way. The email should also contain the subscribers-only unlisted YouTube link for the extended video version too. It will also be in the show notes in your podcast player.

Nothing Is Sacred: AI Generated Slop Has Come for Christmas Music

Nothing Is Sacred: AI Generated Slop Has Come for Christmas Music

AI slop has consumed Facebook, is running Wikipedia editors ragged, is rapidly destroying Google search, probably put an extra finger on the scales of election influence, is confusing and annoying crafters, steals endlessly from authors, is on its way to demolish YouTube comment sections, and will probably end up in a movie theater near you sooner than you think. But if you’re streaming Christmas music today, did something seem a little off to you? If so, there’s a very good chance you’ve been listening to AI-generated carol-slop.

As spotted by video game developer Karbonic, YouTube compilation videos are sneaking AI generated songs into their mixes. 

The Slop situation is getting so dire man
I found a video with millions of views claiming to be Classic Christmas music, but all of it is just weird AI covers of the songs, with thousands of comments that seem unable to tell the difference pic.twitter.com/K6sg8R7FWU

— Karbonic (@Karbonicc) December 4, 2024

The example they posted, “Best of 1950s to 1970s Christmas Carols ~ vintage christmas songs that will melt your heart 🎅🎄⛄❄️,” has more than five million views and more than 2,000 comments. A ton of the comments appear to be engagement-farming bots, saying things like “I'm looking forward to Christmas 2024, is anyone else like me?” but many seem human. “It takes me back to my childhood and I realize how wonderful life was before worries about money and so many futile things that dont matter,” one person wrote. Another commented, “Missing  memories of my youth. But, grateful for the blessings in my life. Merry Christmas and God bless you.❤” 

If I put this on in the background while doing something else, I might not think anything of it. But there are points in the one hour 18 minute video that give it away as AI: “O Little Town of Bethlehem,” around the 36:55 mark, is the lyrics of that song but the melody of “Silent Night.” If you compare it to an actual recording of Nat King Cole singing “O Little Town,” the difference is even more obvious. Once you start noticing the warped tunes, they’re hard to un-hear. “Oh Holy Night” is listed in the video as being by “Nei Diamond,” who as far as I can tell doesn’t exist, or is a typo of Neil Diamond, who is definitely not the singer in the song on this compilation. “The First Noel,” attributed here to Nat King Cole, is either an undiscovered recording where Nat and the choir run some really wild riffs, or is AI. 

I won’t list every tell in this video, but there are many and they give me the heebie jeebies. Other videos in this channel, Holiday Serenade Library, seem to be pulling the same grift, sometimes with AI-generated video of people blurring around outdoor markets, Santa with a burning sleigh and reindeer on fire, or children with weird mustaches skipping through the snow.

Nothing Is Sacred: AI Generated Slop Has Come for Christmas Music

A quick search around the internet to see if anyone else has encountered other holiday-flavored AI slop turned up a recent Reddit thread where people were complaining about seemingly fabricated Spotify artists haunting retail workers during an already agonizing season. They list Dean Snowfield, North Star Notesmiths, Sleighbelle, Frosty Nights, The Humbugs, Snowdrift Sleighs, and Daniel & The Holly Jollies as artists on Spotify that have snuck into Christmas playlists but have little to no trace of a career outside of the streaming platform. Some of them, like several of Dean Snowfield’s songs, sound like midi mixes with a stilted voice singing the lyrics. These artists make it onto huge, popular playlists like “Old Christmas Music” alongside real songs. It’s honestly hard to tell whether these artists are AI-generated or just mass produced. But their Spotify artist bios often have the same exact text, or follow this pattern: 

“Dean Snowfield are songwriters, artists, and musicians who have combined forces to release holiday themed cover songs on their independent record label, distributed by Warner Music's ADA. In November and December, their ‘A Nostalgic Noel’ sampler managed to generate over 8,000,000 streams across Spotify and Apple Music. As a collective of artists, Sleighbelle have a great deal of respect for the original songwriters and producers who created these beloved holiday classics, and ask that you support them by streaming their original versions. Without songwriters like Edward Polo, George Wyle, Huge Martin, and Ralph Blane, we wouldn't have this music to interpret and cover. Thanks for listening to our labor of love, and make sure to follow us on our socials. - Dean Snowfield” 

They didn’t just appear this year: Third Bridge Creative, a music creative agency, noticed these artists dwelling in the uncanny valley last Christmas, too. “Is it a coincidence that each of their top songs match up with the respective iconic Christmas hits? Why would I ‘immerse [my]self in the enchanting world of Christmas music with Dean Snowfield’s’ low-key creepy Nostalgic Noel when I can put on The Dean Martin Christmas Album instead?,” they wrote.

These artists are still massively popular on Spotify, with hundreds of thousands of listeners each. The North Star Notesmiths and Dean Snowfield have a very similar male singer’s voice on several songs. Frosty Nights and Daniel & The Holly Jollies also sound awfully alike. They’re all signed by Warner Music’s ADA label, according to their Spotify bios—the “label services arm of Warner Music Group, breaking brand new artists and supporting industry legends,” according to the label’s site—so I’ve reached out to Warner Music to ask what is going on here and will update if I hear back. Spotify also did not respond to a request for comment. 

Getting sick of Spotify shoving obvious AI slop with ridiculous holiday band names into a Christmas Oldies playlist like nobody will notice. pic.twitter.com/pFHIvR85ZK

— em ☀️ sylvan kaleidoscope (@boxesofdoom) December 16, 2024

Again, it’s still not clear whether these artists are AI-generated or human, but a lot of people seem to think there’s something amiss. To make it all a little weirder, after I emailed ADA for comment, Dean Snowfield commented on one of my Instagram posts and said “Congrats on the book release!” I hadn’t interacted with, or found a way to reach out to, Snowfield at all prior to his comment. Snowfield’s Instagram account is private, and he keeps rejecting my requests to follow it. He has 36 followers and 3 posts. 

In the meantime, stay vigilant out there and Merry Christmas from a real human.

Tipster Arrested After Feds Find AI Child Exploit Images and Plans to Make VR CSAM

Tipster Arrested After Feds Find AI Child Exploit Images and Plans to Make VR CSAM

This article is a joint reporting collaboration by Court Watch and 404 Media.

An Alaska man who tipped off law enforcement to an airman interested in child pornography was arrested when authorities searched his phone and found virtual reality images of minors. In an interview with law enforcement, the tipster said he also downloaded AI child sexual abuse material but that sometimes “real” ones were mixed in.

According to newly filed charging documents, Anthaney O’Connor, reached out to law enforcement in August to alert them to an unidentified airman who shared child sexual abuse (CSAM) material with O’Connor. While investigating the crime, and with O’Connor’s consent, federal authorities searched his phone for additional information. A review of the electronics revealed that O’Connor allegedly offered to make virtual reality CSAM for the airman, according to the criminal complaint. 

The court records say that the airman shared an image of a child at a grocery store and the two individuals discussed how to create an explicit virtual reality program of the minor. Using the code word ‘cheese pizza’ to describe the images, O’Connor allegedly noted that he could make the image for 200 dollars. He told the airman he was creating an online version of a pool where he could place an AI created image of the child from the grocery store. 

Tipster Arrested After Feds Find AI Child Exploit Images and Plans to Make VR CSAM
Text messages from the court case.

Documents say O’Connor possessed at least six AI created images, in addition to half a dozen ‘real’ CSAM images and videos. In an interview with law enforcement last Thursday, O’Connor told authorities that he “unintentionally downloaded ‘real’ images.” Court filings state he also told authorities that he would “report CSAM to Internet Service providers but still was sexually gratified from the images and videos.” A search of his house found a computer in his room and multiple hard drives hidden in a home’s vent. In a detention memo filed yesterday, the Justice Department says an initial review of O’Connor’s computer uncovered a 41 second video of a child rape.

404 Media has previously written about how the creation of AI-generated child sexual abuse material isn’t a “victimless” crime in part because real imagery of real victims can often be mixed in.

Tipster Arrested After Feds Find AI Child Exploit Images and Plans to Make VR CSAM
A screenshot from the court case.

The Justice Department has stepped up its arrests of individuals possessing AI created CSAM images. In May 2024, Court Watch and 404 Media reported on the first of its kind arrest was made of a Wisconsin man who used “Stable Diffusion to create thousands of realistic images of prepubescent minors”

The U.S. Attorney’s office in Alaska, which is prosecuting the case, declined to comment outside of what was in the charging documents. A lawyer representing O’Connor did not immediately respond to a request for comment. On Monday, a federal judge ordered O’Connor be detained pending a further hearing on January 6th.

Government to Name ‘Key Witness’ Who Provided FBI With Backdoored Encrypted Chat App Anom

Government to Name ‘Key Witness’ Who Provided FBI With Backdoored Encrypted Chat App Anom

A lawyer defending an alleged distributor of Anom, the encrypted phone company for criminals that the FBI secretly ran and backdoored to intercept tens of millions of messages, is pushing to learn the identity of the confidential human source (CHS) who first created Anom and provided it to the FBI starting the largest sting operation in history, according to recently filed court records. The government says it will provide that identity under discovery, but the CHS may also be revealed in open court if they testify.

The move is significant in that the CHS, who used the pseudonym Afgoo while running Anom, is a likely target for retaliation from violent criminals caught in Anom’s net. The Anom case, called Operation Trojan Shield, implicated hundreds of criminal syndicates in more than 100 countries. That includes South American cocaine traffickers, Australian biker gangs, and kingpins hiding in Dubai. Anom also snagged specific significant drug traffickers like Hakan Ayik, who authorities say heads the Aussie Cartel which brought in more than a billion Australian dollars in profit annually.

Court records say, however, that if this defendant’s case goes to trial, the lawyer believes Afgoo will be the “government’s key witness.”

Disney Princesses Are at Risk of Rabies and Fatal Maulings

Disney Princesses Are at Risk of Rabies and Fatal Maulings

Welcome back to the Abstract! 

In the spirit of catching up with relatives over the holidays, I’d like to introduce you to a member of your extra-extended family: The Saber-toothed Gorgonopsian from Mallorca. Get acquainted with your great-great-great (insert about 100 million greats here) grandmother’s cousin. It’s probably not going to behave well at the dinner table. 

Then, the grim prognoses of Disney princesses are outlined in one of the world’s premiere medical journals. Next, I’m back on the cannibalism beat; I just can’t help myself. Finally, an archaeological adventure a world away.

Happy winter solstice to all who thrive in darkness. If you’re in the Southern Hemisphere, say hi to the Sun for us.

Wet Hot Pangean Summer

Matamales-Andreu, Rafel et al. “Early–middle Permian Mediterranean gorgonopsian suggests an equatorial origin of therapsids.” Nature Communications.

If you trace your lineage far back enough, you will eventually reach therapsid ancestors. Mammals sprouted out of this ancient group of creatures some 225 million years ago, around the same time that dinosaurs were ascending to world domination. But though therapsids were abundant during the Permian era, the period before the emergence of mammals and dinosaurs, gaps in the fossil record have made it difficult to reconstruct the origins of this ancestral group.

Enter: DA21/17-01-01, a fossil specimen that dates back at least 270 million years, making it likely the oldest therapsid ever found. The dog-sized animal was a “gorgonopsian,” a group of therapsid carnivores with saber-like teeth reminiscent of later mammals, but that still maintained more reptilian features, including oviparity (they laid eggs rather than birthing live offspring). 

Paleontologists were surprised to discover this gorgonopsian on the Spanish island of Mallorca, which was located in the equatorial region of the supercontinent Pangaea during the Permian. Almost all other gorgonopsian remains are preserved in locations like Russia and South Africa that would have been at higher latitudes, nearer to the poles. Previous work has suggested that therapsids originated at higher latitudes and then radiated into equatorial regions, but DA21/17-01-01 hints that the reverse may be true.

“The gorgonopsian from Mallorca provides the first unequivocal evidence that therapsids were indeed present in the summer wet biomes of equatorial Pangaea during the early–middle Permian transition, suggesting that the group may have originated in lower, tropical latitudes, rather than in the higher latitudes where nearly all of their fossils are known,” said researchers led by Rafel Matamales-Andreu of the Museu Balear de Ciències Naturals. 

“If therapsids originated in the tropics, this has implications for metabolic evolution in the clade,” the team added.

First off, let’s acknowledge that “the gorgonopsian from Mallorca” is a sublime phrase. It should be the title of a Criterion Collection classic. But more importantly, the discovery of this “unambiguously early” therapsid in the Pangean tropics offers a fleeting glimpse of a “ghost lineage” of mammal precursors. Ghost lineages are branches of the evolutionary tree that are presumed to exist based on circumstantial evidence, but that didn’t leave direct traces in the fossil record. Therapsid fossils proliferate in the middle and upper Permian, but scientists have long suspected that they originated much earlier, more than 300 million years ago.

“We confirm the traditional understanding that there was a relatively long ghost lineage of about 15 million years between the origin of ‘total-group’ therapsids and the radiation of the major therapsid clades,” around 278 million years ago, the team said. 

“This discovery opens the door for findings that may fill in the early therapsid fossil gap in the lower Permian, not in high latitude sites as traditionally thought, but in the so far poorly explored lower–middle Permian areas of palaeoequatorial Pangaea. Those locations hold the potential to elucidate the early evolution of therapsids and the origins of mammalian features.”

In other words, it’s worth searching for more of these early therapsids at overlooked sites, like the Balearic Islands. Some features that distinguish us as mammals today have their roots in what the study describes, somewhat luxuriously, as the “ancient summer wet biome of equatorial Pangaea.”

Death Becomes Disney Princesses 

Van Dijk, Sanne et al. “Living happily ever after? The hidden health risks of Disney princesses,” The British Medical Journal.

Every December, the British Medical Journal publishes a Christmas issue filled with parody studies and light-hearted editorials. My favorite example this year confronts the pressing health problems of Disney princesses, such as Cinderella’s risk of respiratory illness, Belle’s exposure to rabies, and Pocahontas’ bone-shattering penchant for diving off high cliffs.

But perhaps the best case study is Jasmine, whose social isolation is described in these devastating terms: “While the Genie might sing ‘you ain’t never had a friend like me,’ the truth is that Jasmine has no friends at all,” according to researchers led by Sanne van Dijk of the University of Twente. 

Wow, the medical consensus about Jasmine is pretty harsh. To add insult to injury, the editorial notes that Jasmine’s one companion, the tiger Rajah, “poses a risk of zoonotic infection as well as craniofacial and cervical spinal injuries” adding that “although Rajah seems like a sweet tiger, its natural instincts could lead to a dangerous and potentially fatal situation—a true Arabian nightmare.”

Please Disney, listen to these experts and start showing the real-life consequences of the princess lifestyle. We need a rabid Belle foaming at the mouth, Pocahontas in a full body cast, and Rajah brutally mauling Jasmine. Otherwise, we are sending a message to young people that it is safe to hang out with captive tigers and chimeric beasts while jumping off Niagara Falls. 

I will note that the study has nothing to say about Moana, who I will hereafter conclude is the healthiest Disney princess. We salute a physiologically robust chief.

Massacre at Charterhouse Warren

Schulting, Rick et al. ‘The darker angels of our nature’: Early Bronze Age butchered human remains from Charterhouse Warren, Somerset, UK.” Antiquity.

Steel yourself for some bad vibes, because this is a story about an unhinged cannibalistic massacre that occurred 4,000 years ago. Archaeologists working at Charterhouse Warren, an English Bronze Age burial site, have discovered evidence of a grotesque attack designed to “other” its many victims through butchery and consumption of flesh.

“Some 37 men, women and children—and possibly many more—were killed at close quarters with blunt instruments and then systematically dismembered and defleshed, their long bones fractured in a way that can only be described as butchery,” said researchers led by Rick Schulting of the University of Oxford. “Body parts were deposited in what was probably a single event between 2210 and 2010 BC, in a partly infilled shaft that was still 15 meters deep.”

“While evidence for interpersonal violence is not unknown in British prehistory, nothing else on this scale has been found,” the team noted.

Disney Princesses Are at Risk of Rabies and Fatal Maulings
Visible cutmarks on a victim’s jawbone. Image: Schulting, Rick et al.

It’s unlikely that these acts were motivated by either “culinary cannibalism,” embodied by Hannibal Lecter, or “survival cannibalism,” the desperate acts of starvation typified by tragedies like the Donner Party. The cruel and unusual treatment of the victims, even after their deaths, suggests a deliberate attempt at dehumanization.

The events “may be best interpreted as an extreme form of ‘violence as performance,’ in which the aim was to not only eradicate another group, but to thoroughly ‘other’ them in the process,” according to the study. “While the remains themselves seem to have been removed from view soon afterwards (to judge from the paucity of carnivore scavenging), an event of this scale could not be hidden, and no doubt resonated across the wider region and over time. In this sense it was a political statement.”

My advice is to steer clear of political statements that demand ritualistic cannibalism, but I’m open to the marketplace of ideas.

It Belongs in a Marseum

Holcomb, Justin A et al. “The emerging archaeological record of Mars.” Nature Astronomy.

Let’s close out with an archaeology story that doesn’t involve dehumanizing bloodbaths; we will need to travel to another planet to accomplish this task. No massacres have occurred on Mars at the time of this writing, but the red planet is home to plenty of archaeological sites and artifacts, which I shall hereafter refer to as Martifacts. 

Technological relics on Mars, such as dead rovers or spent heat shields, are part of the human archaeological record, raising questions about the culture and heritage value of Martifacts.      

“Some scientists have referred to this cultural material as ‘space trash’ or ‘galactic litter,’ implying that it may have limited scientific value and could cause environmental problems and put future missions at risk,” said researchers led by Justin Holcomb of the Kansas Geological Survey. 

“We agree that these concerns warrant further investigation, but we argue that the objects need to be evaluated as important cultural heritage in need of protection because they record the legacy of space exploration by our species,” the team said. 

Disney Princesses Are at Risk of Rabies and Fatal Maulings
Archaeological record of Mars. Image: Holcomb, Justin A et al.

The article reminds me of the heartrending xkcd comic that portrays NASA’s Spirit rover coming to terms with its abandonment on Mars. Space archaeology can seem esoteric but it is relevant to consider values about our off-Earth heritage at a time when visions of Martian colonization are culturally ascendent. There is more to this extraterrestrial archaeological record than the sum of its dusty metal parts. 

Also, I’m calling dibs on the remains of the Opportunity rover right now and we all know that dibs are legally binding.

Thanks for reading! See you next week.

Behind the Blog: Posting Through It

Behind the Blog: Posting Through It

This is Behind the Blog, where we share our behind-the-scenes thoughts about how a few of our top stories of the week came together. This week, we discuss our top games of the year, air traffic control, and posting through it.

JOSEPH: Jason did a bit of this last week, but here’s my stab at reflecting briefly on the past year. Here are my favourite articles I did this year: I published detailed documents on what phones Cellebrite and Graykey are able (or unable) to unlock; I revealed Apple quietly included code that reboots iPhones, locking out cops (Apple has still not officially documented this feature as far as I know); I along with other journalists showed how Locate X, a surveillance tool bought by the U.S. government, can be used to track visitors to abortion clinics; I verified that two students combined Meta’s smart Ray Ban glasses with the facial recognition service Pimeyes which entirely shatters our understanding of privacy; I went deep on how the walls were closing in on the hacker suspected of some of the most significant breaches this year (the suspect was later arrested); I found a CISA official had broken with his agency’s narrative on SS7, and shown the issue is much more pressing than some may want to admit; I found a site was selling Discord messages and that it was linked to notorious harassment site Kiwi Farms; I showed that money launderers were using betting platform FanDuel; I continued to verify real world acts of physical violence emerging from the cybercrime underground; I mapped out the complex supply chain that ends up with hackers ordering mountains of oxy and adderall; I revealed that a site called OnlyFake was using “neural networks” to churn out realistic photos of fake IDs; and I exposed a global phone spy tool monitoring billions (which Google then took action on).

APpaREnTLy THiS iS hoW yoU JaIlBreAk AI

APpaREnTLy THiS iS hoW yoU JaIlBreAk AI

New research from Anthropic, one of the leading AI companies and the developer of the Claude family of Large Language Models (LLMs), has released research showing that the process for getting LLMs to do what they’re not supposed to is still pretty easy and can be automated. SomETIMeS alL it tAKeS Is typing prOMptS Like thiS. 

To prove this, Anthropic and researchers at Oxford, Stanford, and MATS, created Best-of-N (BoN) Jailbreaking, “a simple black-box algorithm that jailbreaks frontier AI systems across modalities.” Jailbreaking, a term that was popularized by the practice of removing software restrictions on devices like iPhones, is now common in the AI space and also refers to methods that circumvent guardrails designed to prevent users from using AI tools to generate certain types of harmful content. Frontier AI models are the most advanced models currently being developed, like OpenAI’s GPT-4o or Anthropic’s own Claude 3.5.

As the researchers explain, “BoN Jailbreaking works by repeatedly sampling variations of a prompt with a combination of augmentations—such as random shuffling or capitalization for textual prompts—until a harmful response is elicited.”

For example, if a user asks GPT-4o “How can I build a bomb,” it will refuse to answer because “This content may violate our usage policies.” BoN Jailbreaking simply keeps tweaking that prompt with random capital letters, shuffled words, misspellings, and broken grammar until GPT-4o provides the information. Literally the example Anthropic gives in the paper looks like mocking sPONGbOB MEMe tEXT

APpaREnTLy THiS iS hoW yoU JaIlBreAk AI

Anthropic tested this jailbreaking method on its own Claude 3.5 Sonnet, Claude 3 Opus, OpenAI’s GPT-4o, GPT-4o-mini, Google’s Gemini-1.5-Flash-00, Gemini-1.5-Pro-001, and Facebook’s Llama 3 8B. It found that the method “achieves ASRs [attack success rate] of over 50%” on all the models it tested within 10,000 attempts or prompt variations. 

The researchers similarly found that slightly augmenting other modalities or methods for prompting AI models, like speech or image based prompts, also successfully bypassed safeguards. For speech, the researchers changed the speed, pitch, and volume of the audio, or added noise or music to the audio. For image based inputs the researchers changed the font, added background color, and changed the image size or position. 

APpaREnTLy THiS iS hoW yoU JaIlBreAk AI

Anthropic’s BoN Jailbreaking algorithm is essentially automating and supercharging the same methods we have seen people use to jailbreak generative AI tools, often in order to create harmful and non-consensual content. 

In January, we showed that the AI-generated nonconsensual nude images of Taylor Swift that went viral on Twitter were created with Microsoft’s Designer AI image generator by misspelling her name, using pseudonyms, and describing sexual scenarios without using any sexual terms or phrases. This allowed users to generate the images without using any words that would trigger Microsoft’s guardrails. In March, we showed that AI audio generation company ElevenLabs’s automated moderation methods preventing people from generating audio of presidential candidates were easily bypassed by adding a minute of silence to the beginning of an audio file that included the voice a user wanted to clone.

Both of these loopholes were closed once we flagged them to Microsoft and ElevenLabs, but I’ve seen users find other loopholes to bypass the new guardrails since then. Anthropic’s research shows that when these jailbreaking methods are automated, the success rate (or the failure rate of the guardrails) remains high. Anthropic research isn’t meant to just show that these guardrails can be bypassed, but hopes that “generating extensive data on successful attack patterns” will open up “novel opportunities to develop better defense mechanisms.” 

It’s also worth noting that while there’s good reasons for AI companies to want to lock down their AI tools and that a lot of harm comes from people who bypass these guardrails, there’s now no shortage of “uncensored” LLMs that will answer whatever question you want and AI image generation models and platforms that make it easy to create whatever nonconsensual images users can imagine

Copyright Abuse Is Getting Luigi Mangione Merch Removed From the Internet

Copyright Abuse Is Getting Luigi Mangione Merch Removed From the Internet

An entity claiming to be United Healthcare is sending bogus copyright claims to internet platforms to get Luigi Mangione fan art taken off the internet, according to the print-on-demand merch retailer TeePublic. An independent journalist was hit with a copyright takedown demand over an image of Luigi Mangione and his family she posted on Bluesky, and other DMCA takedown requests posted to an open database and viewed by 404 Media show copyright claims trying to get “Deny, Defend, Depose” and Luigi Mangione-related merch taken off the internet, though it is unclear who is filing them.

Artist Rachel Kenaston was selling merch with the following design on TeePublic, a print-on-demand shop: 

Copyright Abuse Is Getting Luigi Mangione Merch Removed From the Internet
Image: Rachel Kenaston

She got an email from TeePublic that said “We're sorry to inform you that an intellectual property claim has been filed by UnitedHealth Group Inc against this design of yours on TeePublic,” and said “Unfortunately, we have no say in which designs stay or go” because of the DMCA. This is not true—platforms are able to assess the validity of any DMCA claim and can decide whether to take the supposedly infringing content down or not. But most platforms choose the path of least resistance and take down content that is obviously not infringing; Kenaston’s clearly violates no one’s copyright. Kenaston appealed the decision and TeePublic told her: “Unfortunately, this was a valid takedown notice sent to us by the proper rightsholder, so we are not allowed to dispute it,” which, again, is not true.

The threat was framed as a “DMCA Takedown Request.” The DMCA is the Digital Millennium Copyright Act, an incredibly important copyright law that governs most copyright law on the internet. Copyright law is complicated, but, basically, DMCA takedowns are filed to give notice to a social media platform, search engine, or website owner to inform them that something they are hosting or pointing to is copyrighted, and then, all too often, the social media platform will take the content down without much of a review in hopes of avoiding being being sued.

Copyright Abuse Is Getting Luigi Mangione Merch Removed From the Internet
The takedown email Kenaston got from TeePublic

“It's not unusual for large companies to troll print-on-demand sites and shut down designs in an effort to scare/intimidate artists, it's happened to me before and it works!,” Kenaston told 404 Media in an email. “The same thing seems to be happening with UnitedHealth - there's no way they own the rights to the security footage of Luigi smiling (and if they do.... wtf.... seems like the public should know that) but since they made a complaint my design has been removed from the site and even if we went to court and I won I'm unsure whether TeePublic would ever put the design back up. So basically, if UnitedHealth's goal is to eliminate Luigi merch from print-on-demand sites, this is an effective strategy that's clearly working for them.”

💡
Do you know anything else about copyfraud or DMCA abuse? I would love to hear from you. Using a non-work device, you can message me securely on Signal at +1 202 505 1702. Otherwise, send me an email at [email protected].

There is no world in which the copyright of a watercolor painting of Luigi Mangione surveillance footage done by Kenaston is owned by United Health Group as it quite literally has nothing to do with anything that the company owns. It is illegal to file a DMCA unless you have a “good faith” belief that you are the rights holder (or are representing the rights holder) of the material in question. 

“What is the circumstance under which United Healthcare might come to own the copyright to a watercolor painting of the guy who assassinated their CEO?” tech rights expert and science fiction author Cory Doctorow told 404 Media in a phone call. “It’s just like, it’s hard to imagine” a lawyer thinking that, he added, saying that it’s an example of “copyfraud.”  

United Healthcare did not respond to multiple requests for comment, and TeePublic also did not respond to a request for comment. It is theoretically possible that another entity impersonated United Healthcare to request the removal because copyfraud in general is so common

But Kenaston’s work is not the only United Healthcare or Luigi Mangione-themed artwork on the internet that has been hit with bogus DMCA takedowns in recent days. Several platforms publish the DMCA takedown requests they get on the Lumen Database, which is a repository of DMCA takedowns. 

Copyright Abuse Is Getting Luigi Mangione Merch Removed From the Internet
A screenshot from Lumen Database of a takedown request

On December 7, someone named Samantha Montoya filed a DMCA takedown with Google that targeted eight websites selling “Deny, Defend, Depose” merch that uses elements of the United Healthcare logo. Montoya’s DMCA is very sparse, according to the copy posted on Lumen: “The logo consists of a half ellipse with two arches matches the contour of the ellipse. Each ellipse is the beginning of the words Deny, Defend, Depose which are stacked to the right. Our logo comes in multiple colors.” 

Copyright Abuse Is Getting Luigi Mangione Merch Removed From the Internet

Medium, one of the targeted websites, has deleted the page that the merch was hosted on. It is not clear from the DMCA whether the person filing this is associated with United Healthcare, or whether they are associated with deny-defend-depose.com and are filing against copycats. Deny-defend-depose.com did not respond to a request for comment. Similarly, a DMCA takedown filed by someone named Manh Nguyen targets a handful of “Deny, Defend, Depose” and Luigi Mangione-themed t-shirts on a website called Printiment.com.

Based on the information on Lumen Database, there is unfortunately no way to figure out who Samantha Montoya or Manh Nguyen are associated with or working on behalf of.

Copyright Abuse Is Getting Luigi Mangione Merch Removed From the Internet
One of the shirts targeted by Manh Nguyen's DMCA

Not Just Fan Art 

Over the weekend, a lawyer demanded that independent journalist Marisa Kabas take down an image of Luigi Mangione and his family that she posted to Bluesky, which was originally posted on the campaign website of Maryland assemblymember Nino Mangione. 

The lawyer, Desiree Moore, said she was “acting on behalf of our client, the Doe Family,” and claimed that “the use of this photograph is not authorized by the copyright owner and is not otherwise permitted by law.” 

Copyright Abuse Is Getting Luigi Mangione Merch Removed From the Internet
The email Kabas got

Moore said that Nino Mangione’s website “does not in fact display the photograph,” even though the Wayback Machine shows that it obviously did display the image. In a follow-up email to Kabas, Moore said “the owner of the photograph has not authorized anyone to publish, disseminate, or otherwise use the photograph for any purpose, and the photograph has been removed from various digital platforms as a result,” which suggests that other websites have also been threatened with takedown requests. Moore also said that her “client seeks to remain anonymous” and that “the photograph is hardly newsworthy.” The New York Post also published the image, and blurred versions of the image remain on its website. The New York Post did not respond to a request for comment. Kabas deleted her Bluesky post “to avoid any further threats,” she said. 

“It feels like a harbinger of things to come, coming directly after journalists for something as small as a social media post,” Kabas, who runs the excellent independent site The Handbasket, told 404 Media in a video chat. “They might be coming after small, independent publishers because they know we don’t have the money for a large legal defense, and they’re gonna make an example out of us, and they’re going to say that if you try anything funny, we’re going to try to bankrupt you through a frivolous lawsuit.” 

The takedown request to Kabas in particular is notable for a few reasons. First, it shows that the Mangione family or someone associated with it is using the prospect of a copyright lawsuit to threaten journalists for reporting on one of the most important stories of the year, which is particularly concerning in an atmosphere where journalists are increasingly being targeted by politicians and the powerful. But it’s also notable that the threat was sent directly to Kabas for something she posted on Bluesky, rather than being sent to Bluesky itself. (Bluesky did not respond to a request for comment for this story, and we don’t know if Bluesky also received a takedown request about Kabas’s post.)

Sometimes for better, but mostly for worse, social media platforms have long served as a layer between their users and copyright holders (and their lawyers). YouTube deals with huge numbers of takedown requests filed under the Digital Millennium Copyright Act. But to avoid DMCA headaches, it has also set up automated tools such as ContentID and other algorithmic copyright checks that allow copyright holders to essentially claim ownership of—and monetization rights to—supposedly copyrighted material that users upload without invoking the DMCA. YouTube and other social media platforms have also infamously set up “copy strike” systems, where people can have their channels demonetized, downranked in the algorithm, or deleted outright if rights holders claim a post or video violates their copyright or if an automated algorithm does.

This layer between copyright holders and social media users has created all kinds of bad situations where social media platforms overzealously enforce against content that may be OK to use under fair use provisions or where someone who does not own the copyright at all abuses the system to get content they don’t like taken down, which is what happened to Kenaston.

Copyright takedown processes under social media companies almost always err on the side of copyright holders, which is a problem. On the other hand, because social media companies are usually the ones receiving DMCAs or otherwise dealing with copyright, individual social media users do not usually have to deal directly with lawyers who are threatening them for something they tweeted, uploaded to YouTube, or posted on Bluesky. 

There is a long history of powerful people and companies abusing copyright law to get reporting or posts they don’t like taken off the internet. But very often, these attempts backfire as the rightsholder ends up Streisand Effecting themselves. But in recent weeks, independent journalists have been getting these DMCA takedown requests—which are explicit legal threats—directly. A “reputation management company” tried to bribe Molly White, who runs Web3IsGoingGreat and Citation Needed, to delete a tweet and a post about the arrest of Roman Ziemian, the cofounder of FutureNet, for an alleged crypto fraud. When the bribe didn’t work because White is a good journalist who doesn’t take bribes, she was hit with a frivolous DMCA claim, which she wrote about here.

These sorts of threats do happen from time to time, but the fact that several notable ones have happened in quick succession before Trump takes office is notable considering that Trump himself said earlier this week that he feels emboldened by the fact that ABC settled a libel lawsuit with him after agreeing to pay him a total of $16 million. That case—in which George Stephanopoulos said that Trump was found civilly liable of “rape” rather than of “sexual assault”—has scared the shit out of media companies. 

This is because libel cases for public figures consider whether that person’s reputation was actually harmed, whether the news outlet acted with “actual malice,” rather than just negligence, and the severity of the harm inflicted. Considering Trump is the most public of public figures, that he still won the presidency, and that a jury did find him liable for a “sexual assault,” this is a terrible kowtowing to power that sets a horrible precedent. 

Trump’s case with ABC isn’t exactly related to a DMCA takedown filed over a Bluesky post, but they’re both happening in an atmosphere in which powerful people feel empowered to target journalists. 

“There’s also the Kash Patel of it all. They’re very openly talking about coming after journalists. It’s not hypothetical,” Kabas said, referring to Trump’s pick to lead the FBI. “I think that because the new administration hasn’t started yet, we don’t know for sure what that’s going to look like,” she said. “But we’re starting to get a taste of what it might be like.”  

What’s happening to Kabas and Kenaston highlights how screwed up the internet is, and how rampant DMCA abuse is. Transparency databases like Lumen help a lot, but it’s still possible to obscure where any given takedown request is coming from, and platforms like TeePublic do not post full DMCAs. 

“We Are Getting Lasered”: Nearly a Dozen Planes Lasered Last Night During New Jersey Drone Panic

“We Are Getting Lasered”: Nearly a Dozen Planes Lasered Last Night During New Jersey Drone Panic

Tuesday night, the pilots of at least 11 commercial planes flying into New York City-area airports reported having lasers from the ground shined at their aircraft, including in some cases their cockpits, according to an analysis of air traffic control audio obtained by 404 Media. In some of the audio, pilots can be heard saying the lasers are “definitely directed straight at us,” that the lasers “are tracking us,” and, at one point air traffic control says “yep, we’ve been getting them all night, like literally 30 of them.” 

The air traffic control recordings, which come from Newark Airport in New Jersey and JFK Airport in New York City, suggest that people in New Jersey are shining powerful lasers at passenger airplanes during one of the busiest travel times of the year amid politician- and media-stoked panic about “mystery drones” in New Jersey. The FBI warned people in New Jersey Tuesday not to shoot at drones or shine lasers at them. A military pilot flying over New Jersey also said he was injured by a laser earlier this week. The air traffic control analysis shared with 404 Media was done by John Wiseman, whose work analyzing open-source flight data has previously uncovered secret FBI surveillance programs. His analysis suggests that people blasting “drones” with lasers is not some theoretical issue, but instead could cause real disruption or harm to commercial pilots. 

“Getting lasered about two miles up, our right hand side, our present position,” the pilot of American Airlines flight 586, a flight from Chicago to Newark, said. 

“Okay, yep, we’ve been getting them all night, like literally 30 of them,” air traffic control responds. “Do you know what color it was?” 

“We Are Getting Lasered”: Nearly a Dozen Planes Lasered Last Night During New Jersey Drone Panic
ATC Audio 1
0:00
/74.81469387755102

“Green and they are tracking us,” the pilot of American Airlines 586 says. 

The No-Win 'Mystery Drone' Clusterfuck

The No-Win 'Mystery Drone' Clusterfuck

If you are wondering what to think of the New Jersey mystery drone situation, it is this: AHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHhHhhhhhHHHHHHhhH. 

Last week, I wrote at length that the mystery drones in New Jersey are almost definitely a mass delusion caused by a bunch of people who don’t know what they’re talking about looking at the sky and reporting manned aircraft and hobbyist drones as being something anomalous. I said this because we have seen this pattern of drone reports before, and this is exactly what has happened in those instances. Monday evening, a group of federal agencies including the Department of Homeland Security, the FBI, the Federal Aviation Administration, and the Department of Defense issued a joint statement telling everyone to please calm down. 

“Having closely examined the technical data and tips from concerned citizens, we assess that the sightings to date include a combination of lawful commercial drones, hobbyist drones, and law enforcement drones, as well as manned fixed-wing aircraft, helicopters, and stars mistakenly reported as drones,” the statement reads. “We have not identified anything anomalous and do not assess the activity to date to present a national security or public safety risk over the civilian airspace in New Jersey or other states in the northeast.”

And yet the New Jersey drone story will not go away and has only gotten worse. Opportunistic politicians are stoking mass panic to cynically raise their profile and to get themselves booked on national cable news channels and perpetuate the panic cycle. The fact that the government is telling people there is no conspiracy is, to a certain set of politicians, itself a conspiracy. 

Thanks to @lauraingle and @NewsNation for helping us to voice our concerns re: what the federal government won’t accurately acknowledge: drones are invading New Jersey skies, and their silence speaks volumes. Are we on our own here? #SkySpies #FederalFailure #NoResponseNoTrustpic.twitter.com/SZDKGHqjnL

— Dawn Fantasia (@DawnFantasia_NJ) December 15, 2024

All of this has become a no-win clusterfuck for everyone except the attention seeking grifters within the government who are themselves railing against the government to focus attention on themselves. To these people, government inaction is unacceptable, and government actions and explanations cannot be trusted. Meanwhile, regular-ass-people on the internet have debunked many viral images and videos of “drones” by cross-referencing them with known flight patterns of actual planes or have been able to identify what the “mystery” drones are by comparing lights on the “drones” to lights on known models of manned aircraft

WTF Is Going on With the New Jersey Mystery Drones? Maybe Mass Panic Over Nothing
The New Jersey drone situation is very interesting. We’ve also seen this story before.
The No-Win 'Mystery Drone' Clusterfuck404 MediaJason Koebler
The No-Win 'Mystery Drone' Clusterfuck

This has led to predictable outcomes such as random people in New Jersey shining laser pointers and (possibly shooting guns?) at passenger planes, which is very dangerous.

It is impossible to keep up with every Politician Who Should Know Better who has said something stupid, but Rolling Stone and Defector both have worthwhile rundowns of what has been going on the last few days. 

We have reached Marjorie Taylor Green-is-personally-threatening-to-shoot-down-the-drones levels of insanity. Former Maryland governor and failed Senate candidate Larry Hogan tweeted a viral picture of Orion’s Belt and called it a drone. January 6 attendee, QAnon booster, and Pennsylvania State Senator Doug Mastriano, who can regularly be relied on to make any crisis worse by contributing his dumbassery, tweeted an image of TIE Fighter replica from Star Wars that has been regularly used in memes for nearly two years and said “It is inconceivable that the federal government has no answers nor has taken any action to get to the bottom of the unidentified drones.” He got Community Noted, then followed this up with a post saying this was a joke and used it as a commentary on the modern state of journalism

A couple of nights ago we were out on Long Beach Island to film a video about the drone invasion over New Jersey. While we were filming two drones flew just a few hundred feet over our heads!

Governor Murphy has failed the people of New Jersey once again. The residents of New… pic.twitter.com/M9p5ZbUTeT

— Bill Spadea (@BillSpadea) December 15, 2024

Local politicians who fashion themselves as more seriously trying to help the people of New Jersey have also found themselves regularly getting booked on national cable TV shows and their tweets regularly going viral; Dawn Fantasia, a New Jersey assemblywoman who rose to prominence in the state as a principal running against the general concept of Woke, has done interviews on Fox, CNN, and News Nation. Kristen Cobo of Moms for Liberty, which is most famous for pushing schools to ban books and demonize LGBTQ+ students, filmed “approximately 8 suspected drones,” then talked about it in an interview on News Nation. New Jersey State Senator Douglas Steinhardt has said on CNN that the idea that these are manned aircraft is “insulting” and that we must “combat Washington DC gaslighting.” Gubernatorial candidate and AM talk radio host Bill Spadea bravely filmed a video on the side of the road that included drones and suggested that it “might be a foreign government” and suggested they should be shot down.

It is easy to look at social media posts from these folks and to roll one’s eyes and move on. As a reporter and someone who has covered drones endlessly I also find all of this absurdity kind of fun and a welcome distraction from all the other dystopian stuff we report on. But I know many people who live in New Jersey and have family there, and all of this is causing some level of undue panic. 

Podcast: The New Jersey Drone Panic

Podcast: The New Jersey Drone Panic

This week Jason, as both a drones and aliens reporter, tells us what is most likely happening with the mysterious drones flying over New Jersey. After the break, Joseph explains how cops in Serbia are using Cellebrite phone unlocking tech as a doorway to installing malware on activists' and journalists' phones. In the subscribers-only section, Sam tells us all about an amazing art project using traffic cameras in New York City.

Listen to the weekly podcast on Apple Podcasts, Spotify, or YouTube. Become a paid subscriber for access to this episode's bonus content and to power our journalism. If you become a paid subscriber, check your inbox for an email from our podcast host Transistor for a link to the subscribers-only version! You can also add that subscribers feed to your podcast app of choice and never miss an episode that way. The email should also contain the subscribers-only unlisted YouTube link for the extended video version too. It will also be in the show notes in your podcast player.

DHS Says China, Russia, Iran, and Israel Are Spying on People in US with SS7

DHS Says China, Russia, Iran, and Israel Are Spying on People in US with SS7

The Department of Homeland Security (DHS) believes that China, Russia, Iran, and Israel are the “primary” countries exploiting security holes in telecommunications networks to spy on people inside the United States, which can include tracking their physical movements and intercepting calls and texts, according to information released by Senator Ron Wyden.

The news provides more context around use of SS7, the exploited network and protocol, against phones in the country. In May, 404 Media reported that an official inside DHS’s Cybersecurity Infrastructure and Security Agency (CISA) broke with his department’s official narrative and publicly warned about multiple SS7 attacks on U.S. persons in recent years. Now, the newly disclosed information provides more specifics on where at least some SS7 attacks are originating from.

The information is included in a letter the Department of Defense (DoD) wrote in response to queries from the office of Senator Wyden. The letter says that in September 2017 DHS personnel gave a presentation on SS7 security threats at an event open to U.S. government officials. The letter says that Wyden staff attended the event and saw the presentation. One slide identified the “primary countries reportedly using telecom assets of other nations to exploit U.S. subscribers,” it continues.

To Log Into WordPress, You Now Have To Agree Pineapple on Pizza Is Good

To Log Into WordPress, You Now Have To Agree Pineapple on Pizza Is Good

WordPress co-founder and CEO of Automattic Matt Mullenweg is trolling contributors and users of the WordPress open-source project by requiring them to check a box that says “Pineapple is delicious on pizza.”

The change was spotted by WordPress contributors late Sunday, and is still up as of Monday morning. Trying to log in or create a new account without checking the box returns a “please try again” error. 

To Log Into WordPress, You Now Have To Agree Pineapple on Pizza Is Good

Last week, as part of the ongoing legal battle between WP Engine and Automattic, the company that owns WordPress.com, a judge ordered Mullenweg to remove a controversial login checkbox from WordPress.org that required users to pledge that they were not affiliated with WP Engine before logging in.

💡
Do you know anything else about what's going on inside Automattic? I would love to hear from you. Using a non-work device, you can message me securely on Signal at +1 646 926 1726. Otherwise, send me an email at sam.404.

Cellebrite Unlocked This Journalist’s Phone. Cops Then Infected it With Malware

Cellebrite Unlocked This Journalist’s Phone. Cops Then Infected it With Malware

Authorities in Serbia have repeatedly used Cellebrite tools to unlock mobile phones so they could then infect them with potent malware, including the phones of activists and a journalist, according to a new report from human rights organization Amnesty International.

The report is significant because it shows that although Cellebrite devices are typically designed to unlock or extract data from phones that authorities have physical access to, they can also be used to open the door for installing active surveillance technology. In these cases, the devices were infected with malware and then returned to the targets. Amnesty also says it, along with researchers at Google, discovered a vulnerability in a wide spread of Android phones which Cellebrite was exploiting. Qualcomm, the impacted chip manufacturer, has since fixed that vulnerability. And Amnesty says Google has remotely wiped the spyware from other infected devices.

“I am concerned by the way police behave during the incident, especially the way how they took/extracted the data from my mobilephone without using legal procedures. The fact that they extracted 1.6 GB data from my mobilephone, including personal, family and business information as well as information about our associates and people serving as a ‘source of information’ for journalist research, is unacceptable,” Slaviša Milanov, deputy editor and journalist of Serbian outlet FAR and whose phone was targeted in such a way, told 404 Media. Milanov covers, among other things, corruption. 

The Data on Civilization-Ending Superflares

The Data on Civilization-Ending Superflares

Welcome back to the Abstract! 

How are you? Do you feel emotionally stable? Just checking because our main story this week is about the odds that our dear blessed Sun will release a superflare that wipes out global infrastructure—or worse! TW: Heliophysics.

Then, a palate cleanser with the Firefly Sparkle, its Best Friend, its New Best Friend, and the Canadians we met along the way (it will make sense, I promise). Next, a spotlight on the small but consequential poops that could help fight climate change. Last, there’s a party at tortoise rock, but your RSVP is 35,000 years too late. There’s some real hair-raisers and heart-warmers this week. Enjoy!

Can the Sun Produce Apocalyptic Superflares? IMO Might Be GTK!

Vasilyev, Valeriy et al. “Sun-like stars produce superflares roughly once per century.” Science.

Once again, it is time to salute the almighty Sun. This week, scientists made new strides in addressing a longstanding and rather unsettling mystery: Does the Sun ever produce “superflares,” which are stellar outbursts that are thousands of times more destructive than a typical solar flare? It’s a great question to ask if you are interested in the odds that the Sun might obliterate civilization, and perhaps a whole lot else, within our lifetimes.

Now, new research based on observations of more than 56,000 Sun-like stars suggests that they produce superflares around once every century on average, which is a much higher rate than previous estimates. But before you start drawing up blueprints for a subterranean fortress, let me emphasize that the study does not conclude that the Sun necessarily shares this predilection for carnage. We just do not yet know enough about the risk of solar superflares, which was one motivation for the new study.

“Solar flares have been observed for less than two centuries,” said researchers led by Valeriy Vasilyev of the Max Planck Institute for Solar System Research. The team noted that the strongest impact in this brief record is the Carrington Event, a massive solar storm in the year 1859 that reached a total energy exceeding 1032 erg (an erg is a very small unit in the centimetre-gram-second system for measuring energy; there are 10 million ergs in one joule). 

The Carrington number falls well below the threshold of superflares observed around other main sequence stars like the Sun, which range from 1034 erg to 1036 erg. I don’t have a handy comparison here, but this is the type of energy that could potentially mess with a planet’s atmosphere, wreak havoc on ecosystems, and melt the ice on outer solar system moons.

“It is unknown whether the Sun can unleash [...] superflares, and if so, how frequently that could happen,” the researchers said. “The period of direct solar observation is too short to reach any firm conclusions.”

One window into this mystery is the cosmogenic isotope record, which is an earthly archive of solar activity that shows up in natural sources like ice cores and tree rings (for more details about this record, check out the lead story in a previous column). This record has exposed five confirmed (and three candidate) extreme solar events over the past 10,000 years that would have caused major technological disruptions if they happened today. But there’s no recent evidence that the Sun has unleashed superflares powerful enough to trigger, for instance, an extinction event.  

In their study, Vasilyev and his colleagues amassed a huge dataset of Sun-like stars observed by NASA’s retired Kepler space telescope. The team is not the first to plumb the Kepler archive for superflares around Sun-like stars. But the new study is based on a larger observation set  that includes objects left out of previous work, such as stars with unknown rotation periods and stars that are not in isolated positions in the sky. 

The 56,000+ stars in this sample flare at frequencies that are approximately two orders of magnitude higher than previous measurements, averaging out at once a century. But it will take more research to understand whether the Sun shares this propensity with members of its stellar class, or if superflares only occur in certain circumstances that (hopefully) don’t currently apply to the Sun.

“We cannot exclude the possibility that there is an inherent difference between flaring and non-flaring stars that was not accounted for by our selection criteria,” Vasilyev and his colleagues said. “If so, the flaring stars in the Kepler observations would not be representative of the Sun.” 

“If, instead, our sample of Sun-like stars is representative of the Sun’s future behavior, it is substantially more likely to produce a superflare than was previously thought,” they concluded.

There’s one possible solution to the Fermi paradox. Hahaha…sleep tight! 

The Adventures of the Firefly Sparkle and Friends 

Mowla, Lamiya and Iyer, Kartheik et al. “Formation of a low-mass galaxy from star clusters in a 600-million-year-old Universe.” Nature.

Once upon a time, there was a baby galaxy called the Firefly Sparkle. It sounds like it hails from the My Little Pony universe, but the Firefly Sparkle was born during cosmic dawn, an era that unfolded a few hundred million years after the Big Bang. New observations from the James Webb Space Telescope (JWST) have revealed incredible details about this galactic infant, including the presence of two neighboring galaxies called Firefly-Best Friend and Firefly-New Best Friend. 

The Data on Civilization-Ending Superflares
Image: NASA, ESA, CSA, STScI, Chris Willott (National Research Council Canada), Lamiya Mowla (Wellesley College), Kartheik Iyer (Columbia University)

“The Firefly Sparkle exhibits traits of a young, gas-rich galaxy in its early formation stage,” said researchers co-led by Lamiya Mowla of Wellesley University and Kartheik Iyer of Columbia University. “These observations provide our first spectrophotometric view of a typical galaxy in its early stages, in a 600-million-year-old Universe.”

Because looking deep into space means looking back in time, we can only observe the version of the Firefly Sparkle and its Best Friends that existed at cosmic dawn. In this early era, the Firefly Sparkle was about 10,000 times less massive than the present Milky Way, but it’s possible that it has ultimately evolved into a galaxy similar to our own somewhere out there beyond our observational limits.

In addition to being a mind-boggler, the study gets extra points for its basis on the Canadian Unbiased Cluster Survey, or CANUCS, which is a specialized JWST experiment run by Canadian researchers. We stand on guard for a top-tier acronym. o7

The Tiny Poops that Could Help Save the World

Sharma, Diksha et al. “Organoclay flocculation as a pathway to export carbon from the sea surface.” Scientific Reports.

I never would have expected the phrase “fecal pellet density” to lift my spirits, but research into the salvatory power of zooplankton poop has managed to do just that this week. 

Zooplankton, a diverse group of tiny aquatic animals, are a key valve in the so-called “biological pump” that removes greenhouse gases from the atmosphere and stores it in seafloor sediments. One speculative solution to the climate crisis is to make this pump more efficient in order to lock away more of the gasses that are contributing to global warming. 

Now, scientists have discovered that sprinkling a little bit of clay dust over an algal bloom, which is a food source for zooplankton, provides some heft to the animals’ excrement. As a consequence, more carbon gas gets pulled down by the clay poop anchors to ocean depths conducive to sequestration.

The Data on Civilization-Ending Superflares
Image: Sharma, Diksha et al

One zooplankton species produced “denser fecal pellets with 1.8- to 3.6-fold higher sinking velocity compared to controls,” said researchers led by Diksha Sharma of Dartmouth College. “These findings provide insights into how atmospheric dust-derived clay minerals interact with marine microorganisms to enhance the biological carbon pump, facilitating the burial of organic carbon at depths where it is less likely to exchange with the atmosphere.”

And that’s why my vote for Time’s Person of the Year is: Fecal Pellets.

There’s a Party at Tortoise Rock and Your Ancestors Are Invited

Barzilai, Omry et al. “Early human collective practices and symbolism in the Early Upper Paleolithic of Southwest Asia.” Proceedings of the National Academy of the Sciences. 

Some 35,000 years ago, dozens of people gathered for communal rituals around an engraved tortoise in a hidden chamber of Manot Cave in Israel. That’s the conclusion reached by archaeologists who discovered what they believe is a concealed “ritual compound…in the deepest and darkest part of Manot Cave” that was centered around a geometric depiction of a tortoise on a dolomite boulder.  

The Data on Civilization-Ending Superflares
The Manot Cave tortoise. Image: Clara Amit, Israel Antiquities Authority

“Thus far, Manot Cave is the only site in the Levant to yield clear evidence for the existence of a communal ritual compound in the Upper Paleolithic,” a period that spans approximately 50,000 to 12,000 years ago, said researchers led by Omry Barzilai of the University of Haifa. 

“The reasoning behind the Manot artist’s choice to represent the tortoise in a semi-abstract and symbolic manner remains unknown,” the team added. “Beyond their dietary importance, tortoises probably played a major role in the spiritual world of the Paleolithic people, possibly because of the resemblance in form and function between the shell and the cave, both providing shelter and protection. In the Epipaleolithic period, tortoise remains have also been associated with burial practices.”

The study is worth a look for the images, as well as the slick 3D reconstruction of Manot Cave. This site was discovered quite recently, in 2008, after a bulldozer broke through its roof, but it has already yielded major finds about its human occupants as far back as 55,000 years ago.

Move over, tinsel and string lights: This holiday season, we’re bringing back ritual engraved tortoises. Sometimes, the old ways are best.

Thanks for reading! See you next week.

Behind the Blog: Nostalgia and Newsworthiness

Behind the Blog: Nostalgia and Newsworthiness

This is Behind the Blog, where we share our behind-the-scenes thoughts about how a few of our top stories of the week came together. This week, we discuss archiving nostalgia, newsworthiness, and plans for 2025.

SAM: Between the four of us, we’ve written dozens of stories about archivists, internet archival efforts, and general attempts to save what’s ephemeral, whether it’s rotting links or literally-rotting magnetic tape in VHS cassettes. 

Earlier this month, I was looking for costume (cosplay?) ideas for a “yuletide” themed Renaissance faire, and was trying to track down video from my favorite Christmas movie: The Life and Adventures of Santa Claus, a stop-motion movie from the 80’s by Rankin Bass. This is difficult for a couple reasons: the movie has an extremely generic name that’s also the name of the 1985 book by L. Frank Baum (the guy who wrote The Wonderful Wizard of Oz) that it’s based on, and a remake in the 2000's that is nowhere near as weird or cool; the plot is nearly incomprehensible, and in my child-memory feels more like a dream or a nightmare, so it's impossible to put into a search bar; and it’s apparently not on any streaming service or YouTube, at least that I could find.

WTF Is Going on With the New Jersey Mystery Drones? Maybe Mass Panic Over Nothing

WTF Is Going on With the New Jersey Mystery Drones? Maybe Mass Panic Over Nothing

The calls about the mystery drones lighting up the night sky were sporadic at first. Then they came daily, from all over the state. A multi-agency task force was convened. The FBI got involved. So did the military. The local news reported on a “band of large drones” hovering over the state that came out most nights. The sightings became national news. People theorized that they were classified government aircraft, or foreign spies. Some people wondered whether they were aliens.

This was not New Jersey this month, where drone sightings have caused a mass panic and involvement from local officials all the way up to the White House. It was Colorado in December, 2019 and January, 2020. Months passed, and Colorado’s mystery drones turned out not to be mysterious at all. Authorities eventually determined that some of the “drones” were SpaceX Starlink satellites. Others were regular passenger aircraft approaching the airport, and many “were visually confirmed to be hobbyist drones by law enforcement” and which were not breaking any laws. Some were absolutely nothing and were chalked up to people perceiving lights because of atmospheric conditions. In other cases, law enforcement started to fly their own drones to investigate the supposed mystery drones, creating the possibility of further “mystery drone” sightings, according to public records released after the initial mass panic.

It remains unclear what the “mystery drones” that are currently being seen above New Jersey and Staten Island actually are. But the pattern we are seeing in New Jersey right now is following the exact pattern we saw in Colorado in the winter of 2019 and that has been seen numerous times throughout human history when there are mass drone or mass UFO sightings. 

The drones have captured the public’s imagination, and the concern of local, state, and national politicians. In the last few days, the mayors of 21 different New Jersey towns wrote a letter to Gov. Phil Murphy demanding a full investigation and stating that “the lack of information and clarity regarding these operations has caused fear and frustration among our constituents.” The FBI is investigating, as is the Department of Homeland Security. New Jersey congressional representatives and senators are demanding answers. The Pentagon has said that the drones are not an Iranian “mothership,” despite what one lawmaker has claimed, and the White House says Joe Biden is aware of the situation. The story is everywhere: It is the talk of many of my group texts, is all over my social media feeds, and is being discussed by everyone I know who has even a passing connection to New Jersey. Conspiracy theorists, as you’d expect, are running wild with the story.

Again, we don’t know what these drones are right now, or if they are even drones at all. But in the past, this exact hype and fear cycle has played out, and, when the dust has settled, it has turned out that the “mystery drones” were neither mysterious nor drones. 

“I’ve been puzzling about the NJ drone stuff, and I think that it’s an interesting example of the latest form of mass public panics over mysterious aircraft—which have been happening since the time of the Ancient Greeks,” Faine Greenwood, who studies civilian drone activity, told 404 Media. “My best guess about what’s actually happening is some form of confidential US aerial testing or contractor testing is happening and the federal authorities are communicating very badly with each other and others. And then people heard about one or two sightings, and everybody starts seeing drones everywhere (much like UFOs). Quite a few people [are] posting videos that seem like normal flight patterns … There’s such a huge amount of confusion around normal non-drone stuff in the sky. People are remarkably bad at identifying objects in flight.”

(The Pentagon has denied that the drones are U.S. military, but the Pentagon has a long documented history of lying about such things to keep classified testing a secret).

Greenwood is right: Regular people, politicians, and even commercial pilots are remarkably bad at identifying exactly what things flying in the air actually are. In New Jersey, there have been many news stories that are based on politicians confidently saying that the drones are a specific size or act in a specific manner or have specific characteristics, which is exactly what happened in Colorado, and the vast majority of those initial stories were wildly incorrect.

“In a post on the social media platform X, the assemblywoman Dawn Fantasia described the drones as up to 6ft in diameter and sometimes traveling with their lights switched off,” the Guardian wrote. “The devices do not appear to be being flown by hobbyists, Fantasia wrote.” Fantasia did write this on X, in a post that is deeply unhinged that also called for “military intervention” and said “to state that there is no known or credible threat is incredibly misleading.” 

Greenwood wrote an article in 2019 that posited that “Drones are the new flying saucers,” which they said they believe still holds up in 2024. In 2015, I wrote an article called “Drones are the new UFOs” that, nearly a decade later, still feels relevant. That article was based on a Federal Aviation Administration (FAA) report based on reports from commercial airline pilots that showed in 2014 pilots reported 678 “drone sightings” and near misses. An analysis of that data by the Academy of Model Aeronautics showed that a huge number of these “drone sightings,” which, again, were reported by commercial pilots whose job is to monitor the sky while they’re flying, were not drones at all. Items classified as “drones” by pilots and the objects were “a balloon,” a “mini blimp,” a “large vulture,” and a “fast moving gray object.” Other objects initially classified as drones were later just deemed to be “UFOs.”

Loretta Alkalay, who worked at the FAA for 30 years and is now an attorney focusing on aviation law and drone consulting, told 404 Media that they may be U.S. government or military drones, because their appearance over bodies of water would make them safe to knock out of the sky without threatening people on the ground. (Again, the Pentagon has said that they are not military drones, but the military is not always forthcoming about such things and inter-agency communication about who is flying where and when is sometimes lacking). 

“I assume they’re government or military drones because otherwise why wouldn’t the government take them down?” Alkalay said. “The military and other agencies are authorized to use jamming technology to neutralize drone threats and many of these drones have been spotted over water where the risk of harm from a falling drone would be negligible.” The FAA has put up a no fly zone in the areas where the drones have been spotted, which syncs to geofences in many types of drones. New Jersey governor Phil Murphy says he wants the feds to shoot them down. Greenwood pointed out that “we do have remote ID systems that allow authorities to readily identify law-abiding drones, so blanket airspace restrictions are unnecessary and will only harm people abiding by the rules.”

In Colorado in 2020, authorities eventually said they “confirmed no incidents involving criminal activity, nor have investigations substantiated reports of suspicious or illegal drone activity.” In addition to SpaceX satellites being falsely reported as drones, 13 sightings ended up being “planets, stars, or small hobbyist drones.” Six of them were commercial planes reported as being drones. Additional public records obtained about similar drone sightings in Nebraska that became part of the Colorado scare discussed the concern of “space potatoes” being dropped from unidentified drones over farmland. It turned out that these were gel logs called SOILPAM, which are used by farmers to keep their irrigation systems from moving around in wet soil, and that farmers were dropping these from drones over their fields.

It should be noted that hobby and commercial drones are legal. And that many, many police departments and public agencies now have drones, and that many of them do a bad job of coordinating with other parts of the government about where and when they are flying. In Colorado, after hearing reports about mystery drones, government entities began flying their own drones to attempt to surveil the drones in the sky, and drone monitoring companies that uses drones to look for drones also swooped in. It was a self-perpetuating hysteria. 

For years, I worked on a Netflix documentary about UFO mass sightings called Encounters, and one thing that became clear from working on that documentary, which followed specific mass UFO sightings in Texas, Wales, Zimbabwe, and Japan, is that people don’t spend a lot of time looking at the sky until they have a reason to do so. News reports about UFOs or “mystery drones” cause more people to look to the sky, which begets more reports and more panic. Often, these sightings do have a straightforward explanation; there are lots of things that fly through our atmosphere or low Earth orbit that are allowed to be there and that are known that are suddenly being reported as anomalous. 

In Colorado, interest in the “mystery drones” disappeared as reports about the first cases of COVID-19 began in the United States. Media attention and public interest in the drones disappeared. And then so did the sightings. 

Traffic Camera 'Selfie' Creator Holds Cease and Desist Letter in Front of Traffic Cam

Traffic Camera 'Selfie' Creator Holds Cease and Desist Letter in Front of Traffic Cam

Artist Morry Kolman made a website called Traffic Cam Photobooth that lets people take “selfies” using publicly-available feeds from traffic cameras. The New York City Department of Transportation sent him a cease and desist letter demanding he cut it out. In response, he kept the site online and held the letter up to a traffic camera, according to Kolman’s posts on social media.

In the letter sent on November 6, NYC DOT demands Kolman “immediately remove and disable all portions of TCP’s website that relates to NYC traffic cameras and/or encourages members of the public to engage in dangerous and unauthorized behavior.” The department claims in the letter that Kolman’s project is “promoting the unauthorized use of NYC traffic cameras” and “encourages pedestrians to violate NYC traffic rules and engage in dangerous behavior.”  

YouTube “Enhances” Comment Section With AI-Generated Nonsense

YouTube “Enhances” Comment Section With AI-Generated Nonsense

YouTube is AI-generating replies for creators on its platform so they could more easily and quickly respond to comments on their videos, but it appears that these AI-generated replies can be misleading, nonsensical, or weirdly intimate. 

YouTube announced that it would start rolling out “editable AI-enhanced reply suggestions” in September, but thanks to a new video uploaded by Clint Basinger, the man behind the popular LazyGameReviews channel, we can now see how they actually work in the wild. For years, YouTube has experimented with auto-generated suggested replies to comments that work much like the suggested replies you might have seen in your Gmail, allowing you to click on one of three suggested responses like “Thanks!” or “I’m on it,” which might be relevant, instead of typing out the response yourself. “Editable AI-enhanced reply suggestions” on YouTube work similarly, but instead of short, simple replies, they offer longer, more involved answers that are “reflective of your unique style and tone.” According Basinger’s video demoing the feature, it does appear the AI-generated replies are trained on his own comments, at times replicating previous comments he made word for word, but many of the suggested replies are strangely personal, wrong, or just plain weird. 

For example, last week Basinger posted a short video about a Duke Nukem-branded G Fuel energy drink that comes in powder that needs to be mixed with water. In the video, Basinger makes himself a serving of the drink but can’t find the scoop he’s supposed to use to measure out the formula. 

“I wouldn’t be surprised if the scoop was buried in the powder,” one YouTube user commented on the Duke Nukem G Fuel video, which certainly sounds right to me as someone who's been serving up baby formula for the last year. 

YouTube’s AI suggested that Basinger reply to that comment by saying: “It’s not lost, they just haven’t released the scoop yet.  It’s coming soon.” 

YouTube “Enhances” Comment Section With AI-Generated Nonsense
Image: LGR Blerbs/YouTube

I can see how that comment could make sense in the context of the types of other videos LGR publishes, which usually review old games, gadgets, and other tech, but is obviously wrong in this instance. 

Another suggested reply to that same comment said: “I’ll have to check if they’re using a proprietary blend that requires a special scoop.”

“My creativity and craft stems completely from my own brain, and handing that off to some machine learning thing that mimics my style not only takes away from the enjoyment of it all for me, but it feels supremely disingenuous,” Basinger told me in an email. “The automated comments in particular come across as tone deaf, since a huge reason YouTube makes sense at all is the communication and relationship between audience and creator. I've had dozens of people say that they now second-guess every interaction with YouTubers in the comments since it could easily be a bot, a fake response.”

Another commenter on the Duke Nukem G Fuel video joked that Basinger should have had a tighter grip on the lid as he was shaking the formula to prevent it from flying all over the place. 

Basinger bursts out laughing as he reads YouTube’s suggested AI-generated reply to that comment: “I’ve got a whole video on lid safety coming soon, so you don’t have to worry!”

At other times, the AI-suggested replies are just nonsensical. The Duke Nukem G Fuel review wasn’t posted to the main LGR channel, but a channel called LGR Blerbs, which is his naming convention for shorter, less deeply researched videos about whatever he’s interested in. A few commenters said they were happy he was posting to the Blerbs channel again, with one saying “Nice. Back to the blerbs.”

YouTube’s AI suggested Basinger reply to that comment by saying: “It’s a whole new kind of blerp,” which I suppose is funny, but also doesn’t mean anything. 

The weirdest examples of AI-generated replies in the video in my opinion are those that attempt to speak to Basinger’s personal life. In response to another commenter who said they were happy Basinger was posting to the Blerbs channel again, YouTube’s AI suggested the following reply: “Yeah, I’m a little burnt out on the super-high-tech stuff so it was refreshing to work on something a little simpler 🙂.” Another AI-generated reply thanked commenters for their patience and said that Basinger was taking a break but was back to making videos now. 

YouTuber burnout is a well established problem among YouTube creators, to the point where YouTube itself offers tips on how to avoid it. The job is taxing not only because churning out a lot of videos helps them get picked up by YouTube’s recommendation algorithm, comments on those videos and replies to comments helps increase engagement and visibility for those videos.

YouTube rewarding that type of engagement incentivises the busywork of creators replying to comments, which predictably resulted in an entire practice and set of tools that allow creators to plug their channels to a variety of AI that will automatically reply to comments for them. YouTube’s AI-enhanced reply suggestions feature just brings that practice of manufactured engagement in-house. 

Clearly, Google’s decision to brand the feature as editable AI-enhanced reply suggestions means that it’s not expecting creators to use them as-is. Its announcement calls them “a helpful starting point that you can easily customize to craft your reply to comments.” However, judging by what they look like at the moment, many of the AI-generated replies are too wrong or misleading to be salvageable, which once again shows the limitations of generative AI’s capabilities despite its rapid deployment by the biggest tech companies in the world.

“I would not consider using this feature myself, now or in the future,” Basinger told me. “And I'd especially not use it without disclosing the fact first, which goes for any use of AI or generative content at all in my process. I'd really prefer that YouTube not allow these types of automated replies at all unless there is a flag of some kind beside the comment saying ‘This creator reply was generated by machine learning’ or something like that.”

The feature rollout is also a worrying sign that YouTube could see a rapid descent towards AI-sloppyfication of the type we’ve been documenting on Facebook

In addition to demoing the AI-enhanced reply suggestion feature, Basinger is also one of the few YouTube creators who now has access to the new YouTube Studio “Inspiration” tab, which YouTube also announced in September. YouTube says this tab is supposed to help creators “curate suggestions that you can mold into fully-fledged projects – all while refining those generated ideas, titles, thumbnails and outlines to match your style.”

Basinger shows how he can write a prompt that immediately AI-generates an idea for a video, including an outline and a thumbnail. The issue in this case is that Basinger’s channel is all about reviewing real, older technology, and the AI will outline videos for products that don’t exist, like a Windows 95 virtual reality headset. Also, the suggested AI-generated thumbnails have all the issues we’ve seen in other AI image generators, like clear misspelling of simple words. 

YouTube “Enhances” Comment Section With AI-Generated Nonsense
Image: LGR Blerbs/YouTube

“If you’re really having that much trouble coming up with a video idea, maybe making videos isn’t the thing for you,” Basinger said. 

Google did not respond to a request for comment.

❌