Donuts And Fear

I’m back from Europe and traveling in America. See?

Can’t guess the town I visited? Here’s another clue:

It may be the town’s most upright hotel.1

Las Vegas aches with obvious irony and overdone metaphors. Normal people drink cheap vodka at $300 a bottle, so they can feel rich. There’s neon everywhere, alongside shrieking slots, arctic-level air conditioning in 100-degree days, and little cards reminding us to conserve energy. People confidently place money on their highly rational roulette “strategy,” as the fateful ball bounces randomly on its wheel.2

I was there for the tech conferences.3 Brace yourself: Between morning donuts and late night beer, I heard about Artificial Intelligence. Specifically, about the rapid erosion of security and trust because AI makes it easier than ever to fool people. And about threats from malicious AI, which dictates that we must all assume we’re under attack, all of the time, even from people and things that until recently were normal parts of our environment.

Two aspects, one human and one machine, stood out in particular.

I ran into my old friend and colleague Nicole Perlroth, who was a computer security reporter when we worked together at The New York Times. She wrote a great book about the market for cyber weapons, which include tips about vulnerabilities in business computer systems, and ways of spying on people through their own phones. She now invests in security companies and, working with the security company Rubrik, has a podcast about cyber threats. The first season, about Chinese spying, just won a big award.

Her new season is about the North Korean information technology specialists who work remotely for what may be thousands of U.S. companies. This has been going on for a decade, but it was supercharged by the COVID lock down, when companies ditched the office, and needed to put a lot of IT work online. The demand for AI coders has now increased opportunities for the North Koreans like never before. They use AI to produce more convincing fake identities, and chase more jobs.

Once hired, these attackers can tap into all kinds of passwords and secrets, plant malware, and otherwise set traps for future attacks. Meantime, their American wages help fund one of the world’s most hideous regimes.

One sad wrinkle inside this story is how the North Korean remote workers prey on clueless Americans, along with dupes in Ukraine, Argentina, and elsewhere. Since the attackers can’t tunnel directly into a U.S. company from Pyongyang4 without giving themselves away, they con people in the U.S. into hosting in their own homes the company laptops these workers have been issued. Then, the North Koreans tunnel into a seemingly innocent U.S. address, and the company can’t see where they really are. North Korea is exceptionally good at online social engineering, and duping folks who are looking for ways to make easy money. Last year a woman in Arizona was sentenced to 8.5 years for hosting one such laptop farm that touched 300 U.S. companies.

Now, thanks to AI faking, these guys (and they do seem to be almost entirely male) are flooding the market. Nicole appeared onstage with someone who figured out that a North Korean was applying for a job, so he notified the FBI, then sent the man a corporate laptop fully loaded with spyware. Even the delivery box had trackers. What they found was a cell of overworked IT fakers desperately looking for real work in the US, even while they were spying on their current employers. One guy applied for 27,000 jobs over a six-month period. Another actually landed 19 jobs over that period.

What can be done? Fortunately there are some useful tells that companies are starting to recognize. Faked resumes often indicate work experience that goes farther back than the age of the person who shows up for the online interview. Keystroke latency during an online test can show that the applicant is on the other side of the Pacific, and not in Florida.

Now companies are starting to close out interviews with suspected agents by asking them to say, “Kim Jong Un5 is a big ugly pig.” Since these humans are monitored in real time by the secret police, they can’t say that.

Just in case you think your job sucks: Try working in several IT jobs at once, while applying for 16,000 more, under the loving eyes of state security. You could almost feel for the enemy.

Share

Elsewhere, much of the talk was about how Mythos, the powerful Large Language Model from Anthropic, has completely upended computer security. In late May Anthropic said that Mythos has found over 6,200 security vulnerabilities, across some of the most important software we use.

I heard a senior person from Anthropic security say that, possibly as soon as the end of this year, open source LLMs will have that kind of power, meaning bad actors will go on the hunt for new vulnerabilities. There’s now a race on to harden everything ahead of that, and maybe things will get patched in time, as happened with the Y2K threat.6 Still, even when the pros feel good about their system, they worry about flaws in some forgotten computer in, say a local utility or a water treatment plant. We’re all so immensely connected that such flaws could easily touch us.

Security threats used to be something that came to us from elsewhere. Now, as the North Koreans working in our payroll and password management departments show, the threats aren’t coming from outside, they are right at hand.

That is a big deal. Up to now we have kept away threats like hacks and viruses with firewalls and antivirus software. There were typically days, if not weeks, between an initial breach and a disaster, because the bad actors were manually mapping out a corporate system. Now it happens at what they call “machine speed,” meaning an AI might be in and done in less than a minute.

Game over? More likely, we’ll develop new strategies, new ways to work. We’ll assume everything is always under attack, and figure out how to work that problem. It’s likely that the new strategies will take cues from nature, where attack and defense have been refined for billions of years.

The industry has been doing this for more than three decades, exemplified by the seminal antivirus paper, “A biologically inspired immune system for computers.”7 Many elements in security have natural parallels: Firewalls are like the armor of turtles and armadillos; digital identity and authentication are like ants that can only enter their colonies if they emit the right pheromones, or else they get attacked.

If we used to have armor against threats from outside, now we’re like the animals at the watering hole, continuing to drink even as we watch the lion that drinks nearby. Such creatures have exceptionally fast awareness of behavior changes, and react quickly, often in ways that confuse a predator. They zigzag too fast for the attacker to focus, or sacrifice the least able of their group, much the way companies may sacrifice some nonessential data to gain an edge over an attacker.

Where tech itself is concerned, work and identity will look different, perhaps more than in any other sector. Smart people are working on new things all the time. The road to success may become as scary as it was during the worst nuclear days of the Cold War.8 Nicole tells me she’s never been more frightened, but she’s been telling me that for a decade. Historically when we experience shocks, we gain understanding, and learn to live in a changed landscape.

If that sounds like someone rationalizing their participation in a potential catastrophe - well, I have been staring at a man playing roulette. The whole time, he looked like he knew how to win.