All Halloween candy is the same now. Nothing stands out, apart from the quantity of candy dumped out on the floor back home. Every house seems to buy a variety bag of seemingly random candy, and consequently every stop metes out a little more of the same treats as the previous one did or the next one will. A harvesttime holiday that evolved from traditional reflections on death is now merely another household chore to carry out on the way to mortal life’s certain end.
In my memory, the trick-or-treating procedure used to go down like this: An older lady or stern man made some idle chatter about costumes and then dispensed one or three candies into my bag directly from their basket. The candy provided was particular and intentional: You got a Hershey’s Special Dark mini and a wee Mr. Goodbar, perhaps, or a few Smarties cylinders, or, alas, a handful of Bit-O-Honeys. Children identified houses by what they dispensed: Don’t miss the quiet bungalow that yielded a surprising payload of Rolos, but don’t bother with the big, well-manicured property on the corner where only Brach’s Jelly Nougats are on offer. Such impressions sometimes became sufficiently evocative—the Rolo house—to characterize a street’s residences at other times of the year.
Not anymore. Kids’ hauls at each house are bigger than ever, but also more similar. Back when the seasonal candy was more diverse, it communicated something about the trick-or-treatee’s aesthetics and even their values. Hershey’s bars or Tootsie Roll Pops suggested traditionalism. So did shortened tubes of Necco wafers (in an esoteric way), or cellophane-wrapped, spherical hard candies (in an antiquated one). Baby Ruth or Butterfinger insinuated indulgence, moderated—Reese’s Peanut Butter Cups meant it immoderately. A miniature box of Nerds or a Blow Pop would have come across as progressive, at least during my youth. A packet of Pop Rocks would have felt positively punk rock.
Kids might share intelligence about the house offering Boston Baked Beans (actually good, but considered undesirable by peers) or Red Hots (symbolically righteous but actually underwhelming). A handful of Sixlets—previously unheard-of—implied a larger universe than a pre-internet child might have imagined possible. Some houses matched a kid’s unexplained taste in quiet, affirming ways. For me, Spree was a peculiar pleasure, and Chuckles jelly candies, and 3 Musketeers bars, with their gentle, whipped nougat. Some doors suggested eccentricity or extremism. Even after the razor-blade panic and Tylenol scare, some homes still gave out plastic-wrapped, homemade popcorn balls, an earnest but unwelcome attempt to retain the disappearing past. Others came off as unhinged. Dentists sometimes offered pencils or crayons instead of candy, for obvious reasons. My parents insisted on throwing handfuls of nickels into awaiting pillowcases, a practice that caused me enormous shame despite the fact that some neighborhood children appreciated it.
This opportunity to express distinctiveness at the Halloween stoop, and to receive it, has become much harder. In part, that’s because trick-or-treating has declined in general. The reasons are many: The total number of trick-or-treat-age children has decreased by a quarter since 1970; community trunk-or-treats or school events in the days leading up to Halloween started supplanting the door-to-door tradition; parents and kids decamp for “better” neighborhoods, whose betterness begets ever-increasing Hallows’ Eve inequity. Neighborhoods are less connected in general, and some municipalities enforce nanny-state ordinances around the practice.
But the candy has also changed. Like soda or fast food, candy was largely regional until well beyond the middle of the century. The Oh Henry! Bar, for example, was invented around 1920, by one account named after an exclamation a clerk at George Williamson’s Chicago candy store uttered to a frequent patron named Henry. But consolidation came to the candy industry. Hershey bought Reese’s in 1963, then Almond Joy and others. Mars ended up with M&M’s, Snickers, Twix, Milky Way, and many more. These days, those two companies account for around two thirds of the U.S. chocolate market. Along the way, products such as Oh Henry! fell by the wayside. That bar changed hands four times from 1965 to 2019, when it was taken off the market in the United States. Now no American child needs to ask a parent what the heck an Oh Henry! bar is, because they definitely won’t receive one in their plastic jack-o’-lantern.
October retail flyers of the past reveal much greater choices. In 1970 and 1985, Kmart advertised bags of Milk Duds, Whoppers, Dots, Sugar Daddy, Good & Plenty, and Junior Mints, among others. A 1988 coupon ad from the Leaf candy company (once the fourth-largest North American confectioner, most of whose brands were bought by Hershey in 1996) sold packs of Chuckles (my heart!), Slo Poke, Good & Fruity, Zagnut, and Rain-Blo bubble gum. Even products from companies that have survived brand acquisitions used to be more varied: Hershey’s Kisses were a common trick-or-treat offering, for example, and Tootsie Roll Industries sold bags of Tootsie Roll Pops. Now, at my local supermarket, Haribo Goldbears represent the sole independent exception.
More often today, Halloween candy comes in those nefarious variety bags. This practice isn’t entirely new; a variety of Hershey’s minis—Milk Chocolate, Special Dark, Krackel, and Mr. Goodbar—were a tradition of office reception desks as much as dark neighborhoods on October 31. Now assortments are the norm. This year, I found several nearly identical Hershey’s bundles at my local Dierbergs, each containing its namesake chocolate bars along with Reese’s cups, KitKat, Almond Joy—and then an oddball from the farm leagues, such as Milk Duds, Hershey’s Cookies ‘n’ Creme, or Rolo. Reese’s Pieces, a candy that E.T. once made unique, were available only in a package with KitKat, Reese’s cups, and two types of Jolly Rancher gummies. Mars has its own versions with Snickers, Milky Way, and two kinds of M&Ms.
The industrial consolidation has turned Halloween candy into an anonymous murk. That mix probably pleases every trick-or-treater a little, but it also risks making every house’s offering essentially identical.
Across the street at Costco, the situation was even more dire. Enormous bags of Kirkland Signature Favorites were on offer, each containing a fusion of the other fusions: Mars and Hershey and Haribo candies all mixed together. The collection was so haphazard and homogeneous, Costco felt obliged to emblazon a large disclaimer on the packaging’s front: “This bag may not include all varieties listed.” A handful of candy an adult might dispense from the Costco superset is identical to one taken from a child’s overall plunder.
The point is clear: Nobody should care what kind of treats they buy or receive anymore. All Halloween candy is what economists call a perfect substitute for other candy. Throw a bag in the cart and get the candy-having done with—then hold out the basket of whatever came in the bag and let the kiddos get the candy-taking done with too.
Perhaps the indifference counts as apathy rather than disdain. Candy might just not be as important as it used to be. Changing tastes have led to an ever-widening set of options for, and conceptions of, treats: sweet beverages, frozen desserts, snack products, yogurt-alikes, breakfast cereals and bars, cookies and cakes. Sweet coffeehouse beverages, limited-edition cookies and snacks, and endless brand collaborations have also created more options and greater surface area for candy alternatives. Competition, combined with a long moral and dental disapproval of sweets, has contracted candy’s charm.
Almost a decade ago, in one of the most unpopular opinions I have ever published, I wrote that full-size candy bars have no place on Halloween. My argument was that big candy bars counted as normal ones, and that Halloween (understood as a holiday of sweets rather than death) was best experienced by amassing and experiencing the treats in a novel and different form. But even all those years ago, I failed to understand how far candy bars had fallen. Full-size ones had already become unfamiliar, such that discovering them on Halloween counted as the novelty that fun-size diversity had represented. Once, a neighborhood’s children could steer one another toward the house with the Reese’s Pieces, or away from the one with the Mounds (sorry, Mounds). But now the only possible take on candy is that yours is better because it is bigger, and you are therefore better because you are more affluent: That house has full-size bars!
So this year, amid the bulging plastic duffels of candy at the Costco, I found myself reconsidering the wickedness of big bars. My wife and I searched our memories of youth, ever more distant, for precedents. Big bars were rarer then, but not unheard-of. When available, we recalled, they did not typically suggest wealth but modesty matched to tribute: mostly big, plain Hershey’s bars, offered by older couples for whom the candy bore particular meaning, because Hershey had transformed chocolate from an unattainable luxury into a cheap, accessible staple.
Then, just beside the anonymous mixed bag, boxes of full-size bars stacked on pallets met my eye. Costco was selling 30 to a box, with Hershey’s, Hershey’s With Almond, the Reese’s Peanut Butter Cup two pack, and the four-fingered KitKat. Even the full-size bars had been relegated to an assortment.
The War on Terror coincided with the rise of the internet, when freshly concerning digital attacks became known as cyberterrorism. Now a new threat has appeared: AI terrorism.
Over the past few years, reports from governments, intergovernmental organizations, and think tanks have warned that AI is likely to fuel terrorist activity, demanding preparation and response. In 2021, the United Nations Office of Counter-Terrorism co-published a report on the “malicious use of artificial intelligence for terrorist purposes.” Cyberattacks such as malware and intrusion featured heavily, as did physical attacks via autonomous vehicles, facial-recognition spoofing, deepfakes, and—the coronavirus pandemic was still raging—engineered biological weapons. Earlier, in 2019 and 2020, the Middlebury Institute of International Studies published two reports on the risks that language models, still mostly unheard-of at the time, would pose to propaganda and radicalization. And just this week, a group of Harvard Kennedy School researchers published a working paper that assumes that the “first major AI-enabled terrorism event” is a credible threat and speculates how it might come about.
Cyberterrorism was already a confusing term, partly because hacking or disabling computer infrastructure might be carried out by state actors and without violence—properties that aren’t normally associated with terrorism. Among the reasons terrorism is bad, a very minor one is that the term has no fixed definition. In revolutionary France, it referred to violence exercised by the Jacobin state, in many cases by guillotine. By the late 19th century, it had shifted to the nonstate violence of anarchist would-be revolutionaries. Terrorism began to overlap with independence movements in the 20th century, drawing it closer to separatist conflict or guerrilla warfare. By the late 20th century, this ideological motivation for terrorism became mandatory. Terrorism amounted to violence designed to produce a psychological effect.
The hijackings and hostage crises of the 1970s exemplify this revised purpose. So do domestic attacks, such as that of Timothy McVeigh, who bombed an Oklahoma City federal building in 1995. Today, a quarter century after the September 11 attacks, terrorism often carries a political judgment instead of describing an act or a type of actor: Nearly any adversary whatsoever might be called a “terrorist” now.
Given the slippage of the term, AI is, in a way, terroristic by nature: It threatens to take your job, steal your money, ruin your children’s education, eradicate art and culture, take away your health care, colonize your town, destroy the climate, make electronics forever unaffordable. You have also been told that AI is inevitable—and that only immensely wealthy Silicon Valley companies, possibly run by sociopathic leaders, have the resources and talent to advance it (which they must do, even if it means the destruction of humankind).
But the older sense of terrorism still persists. The technology could also facilitate, in addition to the familiar chaos that AI threatens to unleash, terrorism in the sense of state- or nonstate-enacted violence or disruption, such as disabling your electric grid and unleashing a bioweapon at your workplace.
The people who study AI terrorism raise different concerns than those raised by whistleblowers inside the AI industry—or AI leaders themselves. Superintelligent-AI-driven extinction events do not top the list of risks, nor do the rogue AI agents that might, in theory, become autonomous. Instead, researchers, intelligence officials, and scenario planners seem principally interested in the ways that AI might allow bad actors to carry out more sophisticated activities with reduced effort. The Las Vegas Cybertruck bomber, for example, asked ChatGPT for advice about how to detonate an explosive, a use that suggests terrorists might be stupid in addition to being enterprising.
Today’s frontier AI models are powerful, cheap, and widely available. They can accomplish (or help accomplish, or appear to accomplish) many types of tasks without having been explicitly designed to do so. Criminals, terrorists, and even governments have tended to use existing communication, transportation, and industrial infrastructure to do bad things. AI offers another such tool. This conclusion is both reasonable and a bit humdrum.
Consequently, most of the reports on AI terrorism that I have read are conservative in their concerns. They stress that carrying out acts of violence or other physical attacks is difficult and that AI can’t magically solve those problems. They are, perhaps, responding to years of cyberterrorism-threat overstatement, the most famous example of which is the idea of an “electronic Pearl Harbor”—a devastating attack on digital infrastructure that incapacitates the nation. That metaphor, which has been in circulation since at least 1999, was made more concrete in 2012, when then–Defense Secretary Leon Panetta warned Congress about a “cyber Pearl Harbor” that “could be as destructive as the terrorist attack on 9/11.”
Many international-policy experts have argued that cyber Pearl Harbor amounts to a mixed metaphor. The attack on Pearl Harbor was an intricately planned act of state warfare, not a rogue operation. More likely, smaller cybersecurity incidents could cascade into bigger ones, or cause unpredictable secondary effects. Real, damaging cyberattacks on physical infrastructure have been possible for many years. AI’s role in cyberterrorism is likely to be the same as the internet’s: an amplified capacity to produce and disseminate propaganda, confusion, and political manipulation.
Panetta justified the warning about a cyber Pearl Harbor on the grounds that the risk was possible even if it wasn’t plausible. A similar justification appears to motivate the Harvard Kennedy School researchers in their paper. The co-authors describe their methodology as a “pre-mortem,” which involves assuming that such an event has taken place and working backwards to understand how it came about.
I asked Stephen Casper, one of the paper’s co-authors, whether artificial intelligence creates new catastrophic capabilities or simply makes existing malice easier, faster, or cheaper. “It’s more the latter,” he told me. Specifically, AI models that possess information about dangerous capabilities, such as how to disrupt infrastructure and construct a chemical weapon, could allow malicious actors to use them to nefarious ends. Max Kamachee, another co-author, told me that new models “can meaningfully assist a novice in creating a known severe biological threat absent safeguards.” Put more simply, bad people with ill intent could use AI to carry out bad acts. The paper goes into much greater detail, including making reasonable suggestions about how different types of models might be controlled or regulated to prevent harm.
The Harvard paper presents two candidates for the first major AI-terrorism event: One is a cyberattack that would cause several billion dollars’ worth of economic damage; the other, a biological-weapon attack that would kill more than 100 people. I asked Casper and Kamachee whether terrorism was the right name for the two scenarios. The paper derives these litmus tests from definitions in recent U.S. laws governing the development and management of models in the AI industry. But Casper acknowledged that those bad outcomes might not be ideologically motivated. “We call it terrorism,” he told me, but he also said that an equivalent name would be “the human-led misuse of AI for malicious ends.” Kamachee called the risk “catastrophic malicious use.”
Predicting the future is foolish, and guessing about whether truly terrible events might or might not come to pass is even worse. On the one hand, Casper and his co-authors seem justified in warning about the worst scenarios that AI could facilitate, even indirectly. On the other hand, AI terrorism risks devolving into the fearmongering rhetoric that cyber Pearl Harbor did.
I pressed Casper to connect the dots between what AI makes possible and which outcomes are likely. He cited the shockingly rapid evolution of capability in the models, along with the fact that some AI companies are directly building AI-driven-biological-research facilities. Anthropic, for example, has built wet labs for Claude-enabled DNA and protein research. Those labs could be breached, or the model weights discovered inside them could make their way out and into the wrong hands. “It’s a big world—lots of people, lots of groups, lots of states, lots of motivations,” Casper said. “The precedented misuse of current AI systems for terrorist-type activities is, you know, a pretty decent sign.” A sign of what? That the AI industry is going to experience what Caspar called “a Challenger or Chernobyl or 9/11 or Enron moment.”
And yet, those historical catastrophes are all quite different. Challenger was doomed by a structural failure that had been overlooked because of organizational mismanagement; the Chernobyl disaster was caused by a design flaw magnified by operator error and stifling bureaucracy; September 11 was an ideologically motivated nonstate attack facilitated by ease of access to expertise and realized thanks to national-intelligence blunders; the Enron scandal was accounting fraud allowed by regulatory failure and auditor complicity. Two of the examples were industrial accidents, one was an unintentional outcome of deliberate misconduct, and only one counts as a terrorist attack.
The actual risks created by AI don’t require terrorism to inspire worry, and thinking about terrorism may confuse or distance us from taking the right preventive actions. Because they act at the fringes of society, terrorists have a tendency to adopt whatever technology and infrastructure that contemporary life makes available. They choose their tools, their weapons, and their targets based on not merely what will cause harm or attract attention but also what they can get hold of. This was true of explosives such as dynamite at the turn of the 20th century and car bombs by mid-century; the newspapers, radio, and television media that would produce notoriety for attackers; commercial aviation once it became mature but not sufficiently secure; and banking exploits and off-books financial networks, among them cryptocurrency. Nobody called McVeigh a rental-truck terrorist or an agricultural-fertilizer terrorist, but those two resources proved convenient and effective for his desired purpose—which was indeed terrorism.
Throughout its history, terrorism has repeatedly appropriated civilian technologies, especially unproven or insufficiently managed ones. Conveniently, such infrastructure also tends to make the public feel danger, which accelerates our fear of vulnerability. Like terrorism that uses railways, cars, airplanes, dense urbanism, and the internet, terrorism that makes use of artificial intelligence is possible and even likely. Preparing for its contingencies is wise, if speculative. In that sense, AI terrorism is not new, even if it is still frightening.
Elon Musk revealed, in a dramatic demonstration on Thursday night at SpaceX’s rocket-testing facility near Waco, Texas, a new “flying” Tesla Roadster prototype. The car hung upside down on a curved track, then performed aerial stunts—including jumping over a Tesla semi, Evel Knievel–style—using air-powered thrusters designed in collaboration with SpaceX. The demo was impressive despite Musk’s awkward manner, which in the context of a supposed literal flying car came off as nerdy charm.
After revealing the car, Musk paraphrased a quote from Peter Thiel, his former PayPal partner: “We wanted flying cars, instead we got 140 characters.” The line, which was part of a 2011 “manifesto” published by Thiel’s venture-capital firm Founders Fund, was intended to indict the feeble ambition of the technology industry. Instead of solving big, important challenges, the world’s smarts—and money—had been devoted to small-potatoes social-internet start-ups such as Twitter.
One obvious oddity of the line—which has since been removed from the manifesto—is that Founders Fund’s most important investment at the time was Facebook, a social-network platform. But another is the idea that anyone actually wanted a flying car for real. That ambition may never have represented an actual dream. Like a jet pack, a flying car represents not just the future but the idea of individual freedom delivered through technological conquest.
Taken at face value, a flying car is not an impressive dream—or at least it hasn’t been since the Wright brothers more or less attained it. Automobiles have always moved on the ground by rolling via motor. The earliest practical airplanes also rolled until they flew (and hopefully landed again). An airplane is, in a way, merely a car with wings.
In 1917, Glenn Curtiss built literally that: His Curtiss Autoplane was an automobile with detachable wings and a propeller. It never really flew effectively, but the idea persisted. Waldo Waterman’s 1937 Arrowbile was more successful at flying, though it wasn’t commercially viable. Many more actual realizations of the supposed dream of flying cars appeared in the ensuing decades. The most famous might be the Taylor Aerocar, which was certified by the federal government in 1956 but was never mass-produced.
Around that time, future-oriented science-factual magazines such as Popular Mechanics frequently rehearsed the vision of the personal aircraft. By then, cars were ordinary and mainstream in America, having displaced collective transportation such as trains and streetcars. The automobile had become a more abstract kind of vehicle, one that embraced and extended personal autonomy through the power to move, without effort, anywhere else—and to do it alone.
But one thing still cramped the automobile’s capacity to deliver that freedom in full: roads. Cars needed them. The open road, of which America had a seemingly endless frontier, somehow was never enough. The best way around those roads was through the air above them; as Emmett Brown told Marty McFly inside one of fiction’s most memorable flying cars, “Roads? Where we’re going, we don’t need roads.”
The stifling oppression of the ground, traversed at any speed, was also an old vexation. In 1923, Hugo Gernsback—sometimes called the father of science fiction thanks to his founding Amazing Stories, an early magazine devoted to the genre—imagined a gyroscopic Helicar that would release travelers from the tyranny of New York City’s newly car-congested streets. A few years later, in 1927, Fritz Lang’s film Metropolis colonized the city’s airspace with small flying vehicles. By 1962, the children’s cartoon The Jetsons had fused those precursors with mid-century Popular Mechanics populism, making the flying car a ubiquitous commuter vehicle.
Over the decades, many attempts to engineer, manufacture, and market a flying car have been undertaken, though none has succeeded. In 1973, Henry Smolinski created the AVE Mizar by attaching Cessna wings to a Ford Pinto; it killed Smolinski during a failed test flight. From the 1960s until the company went dormant in the mid-2010s, Paul Moller made numerous attempts to develop the Skycar, a personal airplane that took off and landed vertically. Around that time, Terrafugia began promising a flying car called the Transition, which remained only a few years away for years—until the company shut down in 2021. “We enter production in 2028,” Musk said of the Tesla Roadster in this week’s Texas demo, before adding, “Actually.”
By all appearances, the Roadster is not a flying car of the same sort as its real and fictional precursors. The vehicle’s air-fan system seems first and principally designed to give the car’s wheels better traction on the ground, which is part of why it can go so fast—it went from zero to 60 miles per hour in less than a second during the unveiling. (Rocket thrusters attached to the back of the thing also help.) The car does not fly so much as it performs aerial acrobatics such as barrel rolls and standing jumps.
Those deeds were impressive as heck. The car certainly appeared to fly, if only for a moment, in the manner of a daredevil. Whether or not it can or is intended to fly in the manner of a traveler or commuter is not yet clear. Watching the Roadster do its thing—unpiloted, to make sure the demonstration didn’t “kill anyone,” as Musk put it—I imagined wealthy gearheads wreaking havoc via ill-fated jumps and spins on suburban blocks instead of salarymen conveying themselves to work and back.
Subdued by decades of failed automobile-airplane hybrids, and perhaps exhausted by Musk’s dubious intentions, some coverage of the Tesla demo has observed, rightly, that substantial regulatory hurdles await any vehicle that leaves the road and enters airspace. But bureaucratic red tape isn’t the main reason the flying-car dream hasn’t come to pass.
Rather, it’s because the dream works better as fantasy than reality. When urban modernism began to feel cramped, the skies rising above it offered the possibility of new freedom. After the continent was settled, the roads that traversed it became new prisons from which airspace might allow escape. Once the individual freedom of the automobile became universal, and the roads clogged with thousands of varmints in the rat race, the blue sky above looked like liberty. George Jetson is a cog in the proverbial machine who rises above insignificance by flying to work unfazed, where he revels in the meaninglessness of his job pressing a button to make actual machine cogs. Even Thiel’s mention of the flying car was symbolic, a paradigmatic example of the kind of dream worth dreaming, not an actual one worth waking up to.
Despite the Tesla Roadster’s truly spectacular capabilities, at least as a one-off demo, it fulfills the century-old flying-car ambition on a technicality—as a vehicle for a trick driver rather than a motorist aviator. Perhaps the dream this car truly realizes is not yours or mine, but Elon Musk’s: to claim to have delivered what a century of his forebears could not.
Have you ever counted the computers in your house? Tally them up now. I’ll wait. You’re probably including your desktop and laptop computers, if you own some. I bet you’re also including the smartphones and tablets. Depending on the size of your family, that might mean 10 at most. Right?
Your count is almost certainly wrong. I’ve got no fewer than 170 computers in my house. That’s because my home, like nearly everyone’s, is now chock-full of devices that a reasonable person might not even know are computers. Mine include speakers and thermostats, wall switches and electric plugs, door locks and cameras, printers and televisions, ceiling fans and window blinds. Robot vacuums, lawnmowers, and pool cleaners also log on. So do some refrigerators, toasters, and washing machines. And that’s just a sample. Together, this family of supposedly smart gadgets is sometimes referred to as the “internet of things,” or IoT. If you’ve got a thing that you interact with via an app, then it’s almost certainly online and chattering in the background.
IoT gadgets have always been insecure, both by design (to simplify operation or allow them to phone home to a manufacturer) and by habit (because normal people are not network administrators who can secure them). If compromised by network exploits, default-password intrusion, or other security vulnerabilities, these devices—these computers—can be harnessed by malicious parties from anywhere in the world to overwhelm a target with a distributed denial-of-service attack; to relay traffic for fraud or hacking; to collect data for sale, use, or ransom; or to spread malware that performs these and other malicious acts. Homes aren’t the only targets, either. Businesses are also at risk, because so many of their operations involve machinery that has been (foolishly, but understandably) connected to the internet. Just this week, hackers attacked hundreds of French supermarkets, spoiling millions of dollars’ worth of food by warming up refrigerators and freezers remotely.
None of this is new. But Anthropic, the company that makes Claude, published a blog post on Monday warning that artificial intelligence could make internet-of-things vulnerabilities worse. Ordinary malware follows the instructions its perpetrators wrote. That means it’s usually designed to exploit a specific vulnerability in a particular device—baby monitors, say—and then to use that vulnerability to run malicious software that does something specific, such as sending unwelcome traffic in a DDoS attack. AI introduces a new risk. Malware could install an AI model on many other nearby devices. Then a hostile program could, in effect, “chat” with that AI to carry out new kinds of pernicious acts. An Anthropic experiment successfully installed a small AI model across 15 Android TV boxes—the type of device that nearly every home contains, and that sends data to and from the internet constantly, even when the set appears to be turned off. Once the AI model is in place, malware programs with access could use it to help the malicious software gain access to even more devices, something that ordinary IoT exploits cannot do. That could help malware spread faster and more effectively.
The idea of a self-replicating AI botnet sounds profoundly dire, especially given the versions of that idea that philosophy and science fiction have imagined: Agent Smith copying himself over other people in The Matrix, the evolutionary swarm of autonomous machines in Stanisław Lem’s The Invincible, Nick Bostrom’s imagined machine so hell-bent on making paper clips that it extinguishes human life along the way. Recent worries about a hypothetically imminent AI-caused apocalypse have surely perked the public’s ear for anxiety about any type of wild AI fecundity. Not just We’re all gonna die, but A swarm of toasters and vacuum cleaners will be the agent of our doom.
But AI-facilitated IoT malware is concerning for the opposite reason: because it is so ordinary and mundane. The millions of small, internet-connected computers that malicious actors might exploit have remained at least somewhat resilient because those attackers usually need to target them explicitly. External changes, such as firmware updates and network alterations, can limit or eliminate the threats. If malware has access to the rough equivalent of a tiny ChatGPT, however, it might be able to make previously difficult or uneconomical attacks newly possible. In the blog post, Anthropic notes that newer devices with built-in AI chips, such as video-conferencing units and security cameras, might become particularly attractive targets, because their hardware makes running a hostile AI model easier. Beyond those obvious marks, previously hard-to-target, low-value computers—including budget televisions, digital picture frames, smart appliances, or older devices left unused but still connected to a network—might become newly penetrable.
The consequences of AI-accelerated malware remain more or less the same as those of the ordinary malware that is very likely running on your home network right now, without your knowledge. Instead of stealing your intimate photos or siphoning off your bank account, it mostly sends traffic to other networks, phones home with data to aggregate and sell, or routes traffic for shady activities to evade detection. That risk could expand should the AI-assisted spread that Anthropic predicts come to pass—a larger number of infected devices issuing greater disruption and facilitating more criminal activity and targeted exploitation.
The self-replicating AI botnet does not issue a substantially new kind of threat—at least for now. But sometimes a quantitative increase in a risk’s exposure can produce a qualitative change in that risk’s consequence. Take antibiotic resistance, for example, or wildfires. When a landscape accrues substantially greater dried materials that can fuel a fire, the same number of successful ignitions can spread much faster and more broadly. Climate change, human expansion into wildlands, and invasive-species infestations, among other factors, have increased both the chances for fire and its consequences. Exposure to wildfire risk becomes much greater than the simple presence of additional kindling might suggest. An AI-assisted IoT malicious-software inundation could increase the impact of malware substantially.
Even then, the outcome would be more witless and banal than the wildfire metaphor suggests. AI-assisted software hacks could, in theory, mess with the actual computers on a network, as in the French-supermarket attack. But instead of shutting off your home’s smart fridge, coordinated, global, AI-enabled malware is more likely to make networked-computing life harder and more miserable for everyone.
Email spam might offer a better metaphor for AI malware’s likely future. Junk email was annoying for years, but it remained manageable for a long time: You deleted the forwards from your weird uncle, and a simple spam filter caught much of the obvious junk. Eventually, though, email automation became cheap enough that it could both target enormous lists of addresses and personalize attacks, for phishing or exploitation. The low success rates of any single spam or phishing email ceased to be a problem once enormous mountains of junk email could be sent cheaply and easily. Now your inbox is ruined. Every email you receive from a stranger feels useless because you have begun to assume that email is all scams and garbage. (Phone calls and text messages are starting to feel the same way.)
AI malware could follow suit, adjusting probes for greater success, slowing down every computer and network by overwhelming it with garbage-processing and traffic, issuing a nuisance arms race of vulnerability-driven updates and patches, and perhaps even assembling a patchwork of previously valueless local-network data into quilts of persistent, individualized deception or harassment. A dumb risk with good odds, if low consequence, might prove to be more awful than a terrifying one that is highly unlikely.
My family and I enjoy the occasional meal of delicious slow-cooked meat, but every time we visit a barbecue joint, we can’t remember whether we like the beans and slaw on offer. Some beans are sweet; others are spicy. Some slaw is vinegar based; others creamy. We should keep track of this somehow, we always say but then never bother. I should make an app, I used to joke. But that was a ridiculous idea.
Thanks to generative artificial intelligence, it became less ridiculous. By last summer, I had been making small programs with AI for some time. AI’s adeptness at programming was helping me, a busy person with a professional programming background but limited reason to program, work creatively with computing for the first time in ages. I wanted to see how far I could push AI as a solo programmer. The barbecue-sides-preference problem, I reasoned, would serve as a perfect test case: building a fully featured iPhone app—Beanslaw, I called it—with esoteric utility that nobody would have bothered commercializing.
Two days of occasional programming were sufficient to make a professional-grade version of Beanslaw—an act that would have taken weeks of full-time dedication not long ago. The app displayed smoked-meat joints on a searchable map. It implemented sophisticated intelligence to filter out Korean barbecue, Brazilian churrasco, and other (no less delectable!) cuisines that entail barbecue of a different, bean-and-slawless variety. A user can assign assessments of beans (such as vegetarian and soupy) and slaw (including chunky and mustard), and filter the map by desired side-dish traits. I even built an onboarding tutorial. I was ready to ship.
Photos courtesy of the author
Excited, I showed a friend. “Where are everyone else’s slaw and bean ratings?” he asked, tapping at his phone. He didn’t get the point of an app that amounted to a personal journal. Almost two decades of smartphone apps had trained him to understand software as a mechanism for collective intelligence. A program that only a few people would use is now alien. App has come to refer to a type of global, leveraged software business, not a computer program.
Because AI is so good at writing software, in only an hour or so, I was able to add community ratings. Now Beanslaw does what my friend would expect it to—connect potentially billions of users. But doing so also turned an idiosyncratic app meant to satisfy me alone into a candidate (a hopeless one, at that) for the software attention economy.
I never released Beanslaw to the public. But during the year since, I have written, with AI’s aid, a dozen or more programs to solve anomalous problems, seeking to understand what AI programming—a practice some people call vibe coding—means for computing culture. AI-assisted coding could revive the old dream of ordinary people making software for themselves. But so far, in practice it amounts to a new trip down an old path that software development has been traversing for decades: one where a privileged few decide what computers are for, and the rest of us fall in line.
In the early 1960s, inspired by Vannevar Bush’s 1945 Atlantic essay “As We May Think,” a recent Swarthmore College philosophy graduate named Ted Nelson began conceptualizing a computer-publishing system for all global knowledge, accessible to anyone—work for which he coined the term hypertext. A decade later, in 1974, Nelson expanded his vision of computing’s populist power in a pamphlet manifesto called Computer Lib/Dream Machines.
At the time, computers were big, expensive machines used by corporations and government offices. Before the personal computer arrived in earnest, Nelson denounced the “appalling gap between public and computer insider.” He characterized the corporate and governmental actors with access to and expertise in computing as a “priesthood.” Nelson advocated for dismantling this priesthood and putting computing in the hands of ordinary people.
Nelson, along with other figures of the personal-computer revolution, saw his work as literally revolutionary: The Lib of Computer Lib referred to liberation, like “women’s lib,” and he and his allies understood their work as a part of the counterculture movement. Seymour Papert, who co-designed the programming language Logo, envisioned programming as a way for kids to control these machines rather than submit to them. Inspired by Papert, Alan Kay imagined the tablet-style Dynabook, a “personal computer for children of all ages.” And Steve Wozniak, who designed the first Apple (and co-founded Apple Computer with Steve Jobs), valued cheap, publicly shared software skills as a way to overthrow the corporate monopoly on computing.
For a time, that liberationist vision became reality. Early personal computers such as the Apple II and Commodore 64 booted up to a command prompt where users could immediately begin programming in BASIC, an intuitive language. Early advertisements pitched them explicitly as devices people could program themselves. Computer magazines of the era published code that readers typed into their machines by hand.
I learned Papert’s Logo as a first grader in the early 1980s, because my private school had invested in personal computers, labs to house them, and teachers to guide us. That head start gave me confidence with computer programming, which was presented as a normal activity akin to making art or reading. I picked up BASIC, assembly, Pascal, C, and other languages naturally. The machines couldn’t do much yet, and I could learn new things as they evolved—which was still slow enough that an ordinary person, even a child, could keep up.
Personal though they might have become, computers remained expensive and largely inaccessible. And the relatively well-off people who bought them were more likely to buy commercial software than to write their own. Within a few years, the graphical user interface popularized by Macintosh and Windows made the machines easier to use yet also more complicated. Business tools such as spreadsheets, word processors, and layout software made new kinds of computer-based creativity possible, but the results were far removed from the visions of Kay and Nelson. Computers, once envisioned as tools of self-actualization, became appliances.
In the 1980s and 1990s, ordinary people’s ability to construct their own software ebbed and flowed. Hypercard users made databases, cookbooks, and games on their Macs; gray-cubicle office workers used Visual Basic to build Windows programs to automate tasks, often for themselves alone. The World Wide Web let ordinary people publish websites for the whole world, rather than just themselves, their friends, or their office mates.
By the mid-2000s, the Web 2.0 economy had begun to roll back the web’s openness, shepherding people into using platforms such as Blogger, YouTube, and Facebook. The priesthood was back, and the wealthy tech giants served as its bishops.
AI programming does not challenge that power. It entrenches it. I encountered a characteristic, if humdrum, example of software authority during what I thought would be a straightforward vibe-coding task last winter. I had long been dissatisfied with the countdown-timer app on the Apple Watch, with its giant list of times and tiny stop button. I mostly time things in 30-second increments: hamstring stretches, grill sears, tea brews. So with AI’s help, I made a watch app I called Taptimer. I’d tap the screen repeatedly to add half minutes; the timer would run, and then it would buzz. Tapping anywhere on the screen would turn it off again. Sounds simple, right?
Except, Apple doesn’t let third-party programs issue the type of repeated-vibration notifications that its special “native” timer program uses. I had to settle on a single-buzz notification. It works—I use Taptimer every day—but the terms of what the software is allowed to do are set by the owners of the platforms that people must use to run it.
With reason, or at least excuse. Under the guises of usability, security, or control, the oligopolistic tech industry dictates what software can be and can do. The software industry began treating a program shared among friends like a product distributed to millions—in part because it cannot imagine why anyone would bother to make a program that didn’t aspire to wealth, power, or prestige. AI programming arrives during the most intensively exclusionary version of the computer priesthood. The software business has been pushing amateurs out of software creation for two decades, forcing us to use its own wares instead.
When I first started writing Mac programs, in the early 1990s, I could compile a program, put it on a disk, and give it to a friend. Apple’s documentation for how to do so came in printed volumes you could buy at a bookstore, and that didn’t change much across an entire decade. By the time I sat down to write Beanslaw, I needed a $99-a-year subscription to the Apple Developer Program, sufficient familiarity with Apple’s development tools to understand how to use ChatGPT’s code, and enough software-publishing experience to get the resulting software off of my computer and onto phones.
I had to know enough about making software that I could reason and negotiate with the AI about how to approach certain tasks. Some of this effort was not technical at all, such as building a whitelist of known barbecue joints (Rendezvous in Memphis, Pecan Lodge in Dallas) that the app would always display. Others required a deeper bureaucratic knowledge of software development as a practice, such as how to include restaurant images via a backend service provided by Yelp without accidentally spending thousands of dollars.
The more I programmed with AI, the more I felt this tug-of-war between my own prior programming knowledge and the exciting, new coding capacities AI unlocked for me. On my own, I could have built the community ratings that my friend who tried Beanslaw had expected. But storing data in a scalable way across an internet service is complicated, and I wouldn’t have bothered for a program that was supposed to be for a handful of people. ChatGPT introduced me to an Apple programming feature called CloudKit, which made the process easy. Then AI made the implementation easy, too, by giving me the code.
In other cases, I had known what I wanted and how to do it, but I didn’t feel like investing time in the doing. AI made minor software excursions both reasonable and delightful. For example, the Ethernet switch in my home office kept dropping to a lower throughput. To monitor it, I made, with AI, a Mac program that performs an internet-speed test and displays the results in the menu bar. Or, when I had to rebuild my personal website before preorders for my latest book opened, I relied on 30 years of web-development experience to ask AI for the features I wanted (redirecting old URLs, presenting different content by geography) but that I couldn’t quite make time for amid the duties of my actual job and life.
To call this process vibe coding—as if one just types a prompt, waves their hands, and gets a complete and working computer program—amounts to a profound misconstrual of what’s going on.
Computer scientists sometimes talk about “high level” and “low level” programming. Low-level programming involves a lot of very small-scale, specific instructions; high-level programming abstracts that effort into simpler methods that are easier and more accessible. Low-level programming is sometimes considered more technical and therefore more righteous, but high-level coding is often understood to produce faster results. The latter value has prevailed, and over the years and decades, software development has moved to higher levels of abstraction because doing so streamlines output. Low-level work is used only when necessary—and often just to build tools or infrastructure to facilitate high-level programming.
Vibe coding continues this tendency. Coding has always involved a repeated process of giving a computer instructions for an intended outcome or experience, making the machine run them, and then revising the instructions based on the result. At first, those instructions were immensely basic: Move a number or perform a mathematical operation. Over time, the instructions became more complex and abstracted: Write a file to disk, draw a circle, plot a location on a map. Under the hood, computer programs often generate and run code at a lower level as a part of their work. AI programming does not disrupt that process at all. It extends the effort.
This makes the aesthetics of vibe coding entirely familiar to programmers. The moment-to-moment experience of coding feels the same, but you do so at a greater remove. Realizing this made me understand why programmer types might find generative AI so immediately compelling, and also why they might not grasp the objections of people in other walks of life who oppose it. Using AI to create prose or illustrations seems to erase the experience of the writer or artist and their participation in the act of creation. It clobbers the process in favor of the outcome. But for a programmer, the outcome and the process improve all at once. Nothing about AI programming feels like a compromise or cheat. It just feels like programming.
Or perhaps it’s programming at its best. I could feel myself becoming a more generous and open-minded programmer when I coded with AI. Feature requests or suggestions seemed not like slights or gripes, but like opportunities worth considering. Likewise, bugs stopped bothering me. A defect no longer represented something I did wrong; it was a problem I could hurl back at the machinery to solve. Even dead ends or bad ideas felt good, both because I enjoyed carrying them out and because the prototyping experience seemed familiar, but faster and more deliberate.
For example, over a few weeks this summer, I became obsessed with a little $30 computer with a round screen and a dial, similar to a Nest thermostat. I was convinced that the thing could host a clever video game. So I bought one and tried a bunch of ideas: circular Pong, round Breakout, marble rolling, safecracking, tetherball, mini golf, driving a car, navigating a spaceship, mowing a lawn. All of them were cute, but none felt interesting enough to play more than once. I stopped trying, at least for the moment. That feeling is familiar: I’ve started and abandoned hundreds of software projects in my amateur and professional programming career. But the vibe-coded failures felt less bad, because the coding had produced instructive results without devouring my life.
All of this still relied on my accidental, if long-in-the-tooth, membership into the computing priesthood. When I first started using ChatGPT to code, the process was almost dialectical. I’d ask AI for something, it would present some code, and I would implement that code in my project. The process involved a lot of step-by-step, back-and-forth discussion and action.
And yet, during my year of AI programming, I could feel my sense of control and competence recede as the technology evolved. By this spring, something had changed. So-called agentic execution in Claude Code had made it possible to give the AI access to a directory and have it create a whole program on its own, even a complex one. This method, which seems more automatic and thus more desirable, has spread, and OpenAI has added a version of it called Codex. AI has shifted from facilitating programming—as a process of iteratively making stuff with computers—to solving a problem on its own. No programmer required.
This might seem to realize vibe coding as automation, and perhaps even defrock the coding priesthood. I doubt it, largely because industrial power has swallowed so many prior attempts to democratize coding. But automation is also the wrong goal for amateur programming. The most productive, interesting, and fulfilling AI-programming experiences I have had are the collaborative ones.
Earlier this year, I became interested in the unexplored design space in software-based reading. So I started making prototypes: an e-reader that meted out a story as if over the 15 hours of a transpacific journey; a similar one that dispensed text about a road trip gone awry as the reader moved through physical space; an aleatory story that subtly changed what happened to its characters every time the reader opened it. In each of these cases, my aesthetic, technical, and writerly goals were iterative: I hatched some plan, ChatGPT helped me implement it, and then I judged the results and repeated the process.
Unfortunately, during the year that I spent programming seriously with AI, that paradigm of creativity has been superseded. When I try to partner with ChatGPT to code, the software now urges me to use its automation tools instead. ChatGPT has made certain kinds of direct file editing, which I found particularly useful for iterative work, hard or impossible. The industry at large now seems to assume that a programmer’s end goal is known in advance, and the work just involves moving toward it: Build a game for me, or make a website, or generate an app. A lot of news coverage of the phenomenon takes this efficiency as an improvement, assuming AI programming to be a way for the software industry to become more productive. But I found AI to be most liberating when I treated it like a partner offering a feedback loop between my intention and the machine’s implementation, rather than a silicon priest delivering answers or a minion carrying out my bidding.
Reflecting on my vibe-coding journey, I now realize that I began the effort insufficiently aware of what I should have known to be true: that software had already mostly been captured by the high-tech industry long before AI-coding tools arrived, and that those tools were doomed to worsen as much as improve the situation. I fell into the ever-enlarging chasm between the small-scale, personal-software fantasy and its scaled-up, big-business reality. Even the AI services I use to write software change so frequently that the methods I used last week to make something might change next week when Claude or ChatGPT updates. The whole structure feels erected on a sheet of ice.
Rather than fight the friction, I’ve sought out a software equivalent of crampons that can bite into it—resilient enough to resist the slide into commercial software, but sufficiently functional to persist beyond curiosity. That charge is difficult but still possible, although I still feel as if I am clinging precariously to a frosty crag when pursuing it.
As part of this clamber, I found an inexpensive, programmable e-ink display that I could hang on a wall. I have a home weather station in my yard, and I liked the idea of building a pretty, inconspicuous display for the hyperlocal data it produces. I wrestled with the new version of ChatGPT, but for the moment collaboration was still possible if I pushed the AI to converse with me as a partner. Even in the more esoteric, open-source electronics platforms I was using for the project, an accidental priesthood has arisen thanks to 20 years of chaotic, incremental work by global homebrew volunteers. AI helped me make sense of it all so that I—so that we—could get started. And then, together, we built the layout, slurped up the data from my weather station, and iterated on the appearance of the weather-condition icons. I got the idea to display the temperature from my pool heater and then the soil moisture from my lawn sensors, silly things that would have felt indulgent if a computer hadn’t made them easy. Now the little screen hangs next to the alarm panel, updating itself every 15 minutes.
Photo courtesy of the author
When I pass by the thing on the way to the kitchen or the driveway, I feel a vibration of delight. The sensation isn’t quite pride, because I can hardly take sole credit for the accomplishment. But neither is it opportunism, as if I might package and resell weather-station displays as some kind of value-additive, hardware-accessory side hustle. Instead, I feel overcome by the modesty of the matter. A little black-and-white display is showing me the weather—one I made, if with AI help. How nice. For once, a computer doesn’t have to be connecting the world or saving it—or ruining it. Instead, it can simply hang humbly on a wall inside that world, doing something cute, and intentional, and perhaps even useful. The revolutionaries of computing’s counterculture, who mostly became its priests, once believed that would be enough.
Login
