After successful autonomous flight tests in December, the military is ramping up its plans to bring artificial intelligence to the skies.

On the morning of December 1, 2022, a modified F-16 fighter jet codenamed VISTA X-62A took off from Edwards Air Force Base, roughly 60 miles north of Los Angeles. Over the course of a short test flight, the VISTA engaged in advanced fighter maneuver drills, including simulated aerial dogfights, before landing successfully back at base. While this may sound like business as usual for the US’s premier pilot training school—or like scenes lifted straight from _Top Gun: Maverick_—it was not a fighter pilot at the controls but, for the first time on a tactical aircraft, a sophisticated AI.

Overseen by the US Department of Defense, VISTA X-62A undertook 12 AI-led test flights between December 1 and 16, totaling more than 17 hours of autonomous flight time. The breakthrough comes as part of a drive by the United States Air Force Vanguard to develop unmanned combat aerial vehicles. Initiated in 2019, the Skyborg program will continue testing through 2023, with hopes of developing a working prototype by the end of the year. 

The VISTA program is a crucial first step toward these goals, M. Christopher Cotting, director of research at USAF Test Pilot School, explains. “This approach, combined with focused testing on new vehicle systems as they are produced, will rapidly mature autonomy for uncrewed platforms and allow us to deliver tactically relevant capability to our warfighter,” he says. 

With Ukraine’s use of semiautonomous drones, the US military’s first autonomous flight of a Black Hawk helicopter last November, and the successful testing of AI algorithms in US U-2 spy planes in 2020, it’s clear that autonomous combat represents the next front in modern warfare. But just how completely will AI take over our skies, and what does it mean for the human pilots left on the ground?

The VISTA X-62A (short for Variable In-flight Simulation Test Aircraft) has always been ahead of its time. Built in the 1980s and based on an F-16D Block 30 Peace Marble Il, the plane previously held the designation NF-16D and became the US Airforce Test Pilot School’s go-to simulation machine in the early 1990s. A versatile and adaptable training tool boasting open systems architecture, the VISTA can be fitted with software that allows it to mimic the performance characteristics of multiple aircraft, from heavy bombers to ultra-light fighter jets. 

Prior to last year’s autonomous flight tests, the VISTA received a much-needed update in the form of a “model following algorithm” (MFA) and a “system for autonomous control of the simulation” (SACS) from Lockheed Martin’s Skunk Works. Combined with the VISTA Simulation System from defense and aerospace company Calspan Corporation, these updates facilitated an emphasis on autonomy and AI integration. 

Utilizing General Dynamics’s Enterprise-wide Open Systems Architecture (E-OSA) to power the Enterprise Mission Computer version 2 (EMC2, or Einstein Box), the SACS system also integrates advanced sensors, a set of Getac tablet displays in both cockpits, and multilevel security features, all of which enhance VISTA’s capabilities, including its rapid-prototyping advantage, which allows for speedy software updates to meet the accelerating pace of AI development.

During testing in December, a pair of AI programs were fed into the system: the Air Force Research Laboratory’s Autonomous Air Combat Operations (AACO) and the Defense Advanced Research Projects Agency’s (DARPA) Air Combat Evolution (ACE). AACO’s AI agents focused on combat with a single adversary beyond visual range (BVR), while ACE focused on dogfight-style maneuvers with a closer, “visible” simulated enemy.

While VISTA requires a certified pilot in the rear cockpit as backup, during test flights, an engineer trained in the AI systems manned the front cockpit to deal with any technical issues that arose. In the end, these issues were minor. While not able to elaborate on the intricacies, DARPA program manager Lt. Col. Ryan Hefron explains that any hiccups were “to be expected when transitioning from virtual to live.” All in all, it was a significant step toward realizing Skyborg’s aim of getting autonomous aircraft off the ground as soon as possible.

The Department of Defense stresses that AACO and ACE are designed to supplement human pilots, not replace them. In some instances, AI copilot systems could act as a support mechanism for pilots in active combat. With AACO and ACE capable of parsing millions of data inputs per second, and having the ability to take control of the plane at critical junctures, this could be vital in life-or-death situations. For more routine missions that do not require human input, flights could be entirely autonomous, with the nose-section of planes being swapped out when a cockpit is not required for a human pilot.

“We’re not trying to replace pilots, we’re trying to _augment_ them, give them an extra tool,” Cotting says. He draws the analogy of soldiers of bygone campaigns riding into battle on horses. “The horse and the human had to work together,” he says. “The horse can run the trail really well, so the rider doesn’t have to worry about going from point A to B. His brain can be freed up to think bigger thoughts.” For example, Cotting says, a first lieutenant with 100 hours of experience in the cockpit could artificially gain the same edge as a much higher-ranking officer with 1,000 hours of flight experience, thanks to AI augmentation.

For Bill Gray, chief test pilot at the USAF Test Pilot School, incorporating AI is a natural extension of the work he does with human students. “Whenever we \[pilots\] talk to engineers and scientists about the difficulties of training and qualifying AI agents, they typically treat this as a new problem,” he says. “This bothers me, because I have been training and qualifying highly non-linear and unpredictable natural intelligence agents—students—for decades. For me, the question isn’t, ‘Can we train and qualify AI agents?’ It’s, ‘Why can we train and qualify humans, and what can this teach us about doing the same for AI agents?’

Gray believes AI is “not a wonder tool that can solve all of the problems,” but rather that it must be developed in a balanced approach, with built-in safety measures to prevent costly mishaps. An overreliance on AI—a “trust in autonomy”—can be dangerous, Gray believes, pointing out failures in Tesla’s autopilot program despite Tesla asserting the need for the driver to be at the wheel as a backup. Cotting agrees, calling the ability to test AI programs in the VISTA a “risk-reduction plan.” By training AI on conventional systems such as the VISTA X-62—rather than building an entirely new aircraft—automatic limits and, if necessary, safety pilot intervention can help prevent the AI from endangering the aircraft as it learns.

The USAF’s technology is advancing rapidly. This past December, trial flights for ACE and ACCO were often completed within hours of each other, with engineers switching autonomy algorithms onboard the VISTA in minutes, without safety or performance issues, according to Cotting. In one instance, Cotting describes uploading new AI at 7:30 am and the plane being ready to test by 10 am.

“Once you get through the process of connecting an AI to a supersonic fighter, the resulting maneuvering is endlessly fascinating,” says Gray. “We have seen things that make sense, and completely surprising things that make no sense at all. Thanks to our safety systems, programmers are changing their models overnight, and we’re engaging them the next morning. This is unheard of in flight control system development, much less experimentation with unpredictable AI agents.”

Despite these successes, it will take some time before the curriculum at the USAF Test Pilot School undergoes an AI overhaul. Cotting explains that the newness of the AACO and ACE platforms means students will require a greater level of understanding before trying them out in the cockpit of the VISTA. “We’re basically building the bridge as we’re driving over,” Cotting says.

In the meantime, students will undergo a broader test this fall in which they’re exposed to a set of AI and have to figure out how to test it, then execute that test.

As for wider military applications, Cotting says that while he has no visibility into these areas, AI is already ubiquitous in image recognition technology used across the military. While AI-driven tanks may not be on the horizon just yet, the skies, it seems, are set to be home to a new kind of intelligence.

The US Air Force Is Moving Fast on AI-Piloted Fighter Jets

Once praised for its generous social safety net, the country now collects troves of data on welfare claimants.

ILLUSTRATION: KATHERINE LAM

Once praised for its generous social safety net, the country now collects troves of data on welfare claimants.

**I Am Not a Number**

Unprecedented access to one of the world’s most sophisticated welfare fraud algorithms has revealed a deeply flawed and discriminatory system.

In a sparsely decorated corner office of the Danish Public Benefits Administration sits one of Denmark’s most quietly influential people. Annika Jacobsen is the head of the agency’s data mining unit, which, over the past eight years, has conducted a vast experiment in automated bureaucracy. Blunt, and with a habit of completing others’ sentences, Jacobsen is clear about her mission: “I’m here to catch cheaters.”

Denmark’s Public Benefits Administration employs hundreds of people who oversee one of the world's most well-funded welfare states. The country spends 26 percent of its GDP on benefits—more than Sweden, the United States, and the United Kingdom. It’s been hailed as a leading example of how governments can support their most vulnerable citizens. Bernie Sanders, the US senator, called the Nordic nation of 6 million people a model for how countries should approach welfare.

But over the past decade, the scale of Denmark’s benefits spending has come under intense scrutiny, and the perceived scourge of welfare fraud is now at the top of the country’s political agenda. Armed with questionable data on the amount of benefits fraud taking place, conservative politicians have turned Denmark’s famed safety net into a polarizing political battleground. 

It has become an article of faith among the country’s right-wing politicians that Denmark is losing hundreds of millions of euros to benefits fraud each year. In 2011, KMD, one of Denmark’s largest IT companies, estimated that up to 5 percent of all welfare payments in the country were fraudulent. KMD’s estimates would make the Nordic nation an outlier, and its findings have been criticized by some academics. In France, it’s estimated that fraud amounts to 0.39 percent of all benefits paid. A similar estimate made in the Netherlands in 2016 by broadcaster RTL found the average amount of fraud per benefit payment was €‎17 ($18), or just 0.2 percent of total benefits payments.

The perception of widespread welfare fraud has empowered Jacobsen to establish one of the most sophisticated and far-reaching fraud detection systems in the world. She has tripled the number of state databases her agency can access from three to nine, compiling information on people’s taxes, homes, cars, relationships, employers, travel, and citizenship. Her agency has developed an array of machine learning models to analyze this data and predict who may be cheating the system.

Documents obtained by Lighthouse Reports and WIRED through freedom-of-information requests show how Denmark is building algorithms to profile benefits recipients based on everything from their nationality to whom they may be sleeping next to at night. They reveal a system where technology and political agendas have become entwined, with potentially dangerous consequences.

Danish human rights groups such as Justitia describe the agency’s expansion as “systematic surveillance” and disproportionate to the scale of welfare fraud. Denmark's system has yet to be challenged under EU law. Whether the country’s experiments with machine learning cross a legal line is a question that could be answered by the European Union’s landmark Artificial Intelligence Act, proposed legislation that aims to safeguard human rights against emerging technologies.

The debate about welfare in Denmark changed in October 2012, when officials asked residents to send in photos of suspected welfare cheats in their local area. The call led some left-leaning commentators to warn of a “war on welfare,” and arrived as the far-right Danish People’s Party—which criticized the government for “luring” immigrants with welfare benefits—rose up in opinion polls. 

Within a year, consulting firm Deloitte released an audit of welfare fraud controls in Denmark, finding them inadequate to detect fraud in an increasingly digitized welfare system. The auditors, commissioned by the Danish finance ministry, estimated the “short-term savings” of a new “risk-scoring infrastructure” to be €126 million.

Deloitte’s vision was realized in February 2015 with a bill that overhauled the Danish welfare state. It proposed a massive expansion of the Public Benefit Administration’s powers, including the ability to store and collect data on millions of people, access other authorities’ databases, and even request data from foreign governments. Largely unnoticed at the time, it also called for the creation of a “data mining unit” to “control for social benefits fraud.”

The bill was backed by all of the major political parties in Denmark and became law in April 2015. That month, Jacobsen left an IT job in the financial sector to become Denmark’s first head of data mining and fraud detection. 

As Jacobsen got to work, conservative politician Troels Lund Poulsen took office in June 2015 as Denmark’s new employment minister. He implemented random airport checks to catch welfare recipients taking undeclared vacations, and proposed giving the new data mining unit access to welfare recipients’ electricity and water bills in order to detect where they were living. He was joined by a growing chorus of supporters, with one municipality reportedly asking for data from cell towers to track where welfare recipients were staying. “It’s about politics,” Poulsen said in March 2018. “It is important for me to send a clear signal that we will not accept social cheating and fraud.”

Jacobsen’s critics have accused her unit of conducting mass surveillance, but she argues that there are clear safeguards that prevent overreach. Jacobsen says her algorithms don’t actually cancel benefits—they only flag people as suspicious. Ultimately, it is up to a human fraud investigator to make the final call, and citizens have the right to appeal their decisions. “You are not guilty just because we point you out. There will always be a person that looks into your data,” she says.

The majority of Danish residents flagged for investigation are found innocent. Of the nearly 50,000 cases selected by the data mining unit in 2022, 4,000, or 8 percent, resulted in some form of punishment. In the cases where wrongdoing was found, the data mining unit has managed to recover €‎23.1 million—a significant return on its annual budget of €‎3.1 million.

But the scale and reach of Denmark's data collection has been criticized by the Danish Institute of Human Rights, an independent human rights watchdog, and the Danish Data Protection Authority, a public body that enforces privacy regulations. Justitia has compared the Public Benefits Administration to the National Security Agency in the US, and claimed that its digital monitoring of millions of Danish residents violates their privacy rights.

Jacobsen says the agency’s use of data is proportional under European data protection laws, and that preventing error and fraud is important to maintain trust in the welfare state. The Public Benefits Administration is also looking to have its algorithms check citizens earlier in the process, when they first apply for benefits, to avoid situations where they have to repay large sums of money. “Most citizens are honest; however, there will always be some citizens who try to get welfare benefits that they are not entitled to,” Jacobsen says. 

Jacobsen also argues that machine learning is fairer than analog methods. Anonymous tips about potential welfare cheats are unreliable, she claims. In 2017, they made up 14 percent of the cases selected for investigation by local fraud officials, whereas cases from her data unit amounted to 26 percent. That means her unit is more effective than anonymous tips, but nearly half of the cases local investigators decide to take on come from their own leads. Random selection is also unfair, she claims, because it means burdening people when there are no grounds for suspicion. “\[Critics\] say that when the machine is looking at data, it is violating the citizen, \[whereas\] I might think it’s very violating looking at random citizens,” Jacobsen says. “What is a violation of the citizen, really? Is it a violation that you are in the stomach of the machine, running around in there?”

Denmark isn’t alone in turning to algorithms amid political pressure to crack down on welfare fraud. France adopted the technology in 2010, the Netherlands in 2013, Ireland in 2016, Spain in 2018, Poland in 2021, and Italy in 2022. But it’s the Netherlands that has provided the clearest warning against technological overreach. In 2021, a childcare benefits scandal, in which 20,000 families were wrongly accused of fraud, led to the resignation of the entire Dutch government. It came after officials interpreted small errors, such as a missing signature, as evidence of fraud, and forced welfare recipients to pay back thousands of euros they’d received as benefits payments.

As details of the Dutch scandal emerged, it was found that an algorithm had selected thousands of parents—nearly 70 percent of whom were first or second generation migrants—for investigation. The system was abandoned after the Dutch Data Protection Authority found that it had illegally used nationality as a variable, which Amnesty International later compared to “digital ethnic profiling.” 

The EU’s AI Act would ban any system covered by the legislation that “exploits the vulnerabilities of a specific group,” including those who are vulnerable because of their financial situation. Systems like Jacobsen’s, which affect citizens’ access to essential public services, would also likely be labeled as “high risk” and subject to stringent requirements, including transparency obligations and a requirement for “high levels of accuracy.”

The documents obtained by Lighthouse Reports and WIRED appear to show that Denmark’s system goes beyond the one that brought down the Dutch government. They reveal how Denmark’s algorithms use variables like nationality, whose use has been equated with ethnic profiling.

One of Denmark's fraud detection algorithms attempts to work out how someone might be connected to a non-EU country. Heavily redacted documents show that, in order to do this, the system tracks whether a welfare recipient or their “family relations” have ever emigrated from Denmark. Two other variables record their nationality and whether they have ever been a citizen of any country other than Denmark.

Jacobsen says that nationality is only one of many variables used by the algorithm, and that a welfare recipient will not be flagged unless they live at a “suspicious address” and the system isn’t able to find a connection to Denmark. 

The documents also show that Denmark’s data mining unit tracks welfare recipients’ marital status, the length of their marriage, who they live with, the size of their house, their income, whether they’ve ever lived outside Denmark, their call history with the Public Benefits Administration, and whether their children are Danish residents. 

Another variable, “presumed partner,” is used to determine whether someone has a concealed relationship, since single people receive more benefits. This involves searching data for connections between welfare recipients and other Danish residents, such as whether they have lived at the same address or raised children together. 

“The ideology that underlies these algorithmic systems, and \[the\] very intrusive surveillance and monitoring of people who receive welfare, is a deep suspicion of the poor,” says Victoria Adelmant, director of the Digital Welfare and Human Rights Project. 

For all the complexity of machine learning models, and all the data amassed and processed, there is still a person with a decision to make at the hard end of fraud controls. This is the fail-safe, Jacobsen argues, but it’s also the first place where these systems collide with reality.

Morten Bruun Jonassen is one of these fail-safes. A former police officer, he leads Copenhagen's control team, a group of officials tasked with ensuring that the city’s residents are registered at the correct address and receive the correct benefits payments. He's been working for the city’s social services department for 14 years, long enough to remember a time before algorithms assumed such importance—and long enough to have observed the change of tone in the national conversation on welfare.

While the war on welfare fraud remains politically popular in Denmark, Jonassen says only a “very small” number of the cases he encounters involve actual fraud. For all the investment in it, the data mining unit is not his best source of leads, and cases flagged by Jacobsen’s system make up just 13 percent of the cases his team investigates—half the national average. Since 2018, Jonassen and his unit have softened their approach compared to other units in Denmark, which tend to be tougher on fraud, he says. In a case documented in 2019 by DR, Denmark’s public broadcaster, a welfare recipient said that investigators had trawled her social media to see whether she was in a relationship before wrongfully accusing her of welfare fraud.

While he gives credit to Jacobsen’s data mining unit for trying to improve its algorithms, Jonassen has yet to see significant improvement for the cases he handles. “Basically, it’s not been better,” he says. In a 2022 survey of Denmark’s towns and cities conducted by the unit, officials scored their satisfaction with it, on average, between 4 and 5 out of 7.

Jonassen says people claiming benefits should get what they’re due—no more, no less. And despite the scale of Jacobsen’s automated bureaucracy, he starts more investigations based on tips from schools and social workers than machine-flagged cases. And, crucially, he says, he works hard to understand the people claiming benefits and the difficult situations they find themselves in. “If you look at statistics and just look at the screen,” he says, “you don’t see that there are people behind it.” 

_Additional reporting by Daniel Howden, Soizic Penicaud, Pablo Jiménez Arandia, and Htet Aung. Reporting was supported by the Pulitzer Center’s AI Accountability Fellowship and the Center for Artistic Inquiry and Reporting._

*   The lie detector was never good at telling the truth
    

*   China is relentlessly hacking its neighbors
    

*   Amazon’s HQ2 is on pause
    

Gabriel Geiger is an investigative reporter at Lighthouse Reports, covering technology and surveillance.

How Denmark’s Welfare State Became a Surveillance Nightmare

A system used by the Dutch city of Rotterdam ranked people based on their risk of fraud. The results were troubling.

From the outside, Rotterdam’s welfare algorithm appears complex. The system, which was originally developed by consulting firm Accenture before the city took over development in 2018, is trained on data collected by Rotterdam’s welfare department. It assigns people risk scores based on 315 factors. Some are objective facts, such as age or gender identity. Others, such as a person’s appearance or how outgoing they are, are subjective and based on the judgment of social workers.

In Hoek van Holland, a town to the west of Rotterdam that is administratively part of the city, Pepita Ceelie is trying to understand how the algorithm ranked her as high risk. Ceelie is 61 years old, heavily tattooed, and has a bright pink buzz cut. She likes to speak English and gets to the point quickly. For the past 10 years, she has lived with chronic illness and exhaustion, and she uses a mobility scooter whenever she leaves the house. 

Ceelie has been investigated twice by Rotterdam’s welfare fraud team, first in 2015 and again in 2021. Both times investigators found no wrongdoing. In the most recent case, she was selected for investigation by the city’s risk-scoring algorithm. Ceelie says she had to explain to investigators why her brother sent her €150 ($180) for her sixtieth birthday, and that it took more than five months for them to close the case.

Sitting in her blocky, 1950s house, which is decorated with photographs of her garden, Ceelie taps away at a laptop. She’s entering her details into a reconstruction of Rotterdam’s welfare risk-scoring system created as part of this investigation. The user interface, built on top of the city’s algorithm and data, demonstrates how Ceelie’s risk score was calculated—and suggests which factors could have led to her being investigated for fraud.

All 315 factors of the risk-scoring system are initially set to describe an imaginary person with “average” values in the data set. When Ceelie personalizes the system with her own details, her score begins to change. She starts at a default score of 0.3483—the closer to 1 a person’s score is, the more they are considered a high fraud risk. When she tells the system that she doesn’t have a plan in place to find work, the score rises (0.4174). It drops when she enters that she has lived in her home for 20 years (0.3891). Living outside of central Rotterdam pushes it back above 0.4. 

Switching her gender from male to female pushes her score to 0.5123. “This is crazy,” Ceelie says. Even though her adult son does not live with her, his existence, to the algorithm, makes her more likely to commit welfare fraud. “What does he have to do with this?” she says. Ceelie’s divorce raises her risk score again, and she ends with a score of 0.643: high risk, according to Rotterdam’s system.

“They don’t know me, I’m not a number,” Ceelie says. “I’m a human being.” After two welfare fraud investigations, Ceelie has become angry with the system. “They’ve only opposed me, pulled me down to suicidal thoughts,” she says. Throughout her investigations, she has heard other people’s stories, turning to a Facebook support group set up for people having problems with the Netherlands’ welfare system. Ceelie says people have lost benefits for minor infractions, like not reporting grocery payments or money received from their parents.

“There are a lot of things that are not very clear for people when they get welfare,” says Jacqueline Nieuwstraten, a lawyer who has handled dozens of appeals against Rotterdam’s welfare penalties. She says the system has been quick to punish people and that investigators fail to properly consider individual circumstances.

The Netherlands takes a tough stance on welfare fraud, encouraged by populist right-wing politicians. And of all the country’s regions, Rotterdam cracks down on welfare fraud the hardest. Of the approximately 30,000 people who receive benefits from the city each year, around a thousand are investigated after being flagged by the city's algorithm. In total, Rotterdam investigates up to 6,000 people annually to check if their payments are correct. In 2019, Rotterdam issued 2,400 benefits penalties, which can include fines and cutting people’s benefits completely. In 2022 almost a quarter of the appeals that reached the country’s highest court came from Rotterdam.

This Algorithm Could Ruin Your Life

Peter Eckersley did groundbreaking work to encrypt the web. After his sudden death, a new organization he founded is carrying out his vision to steer artificial intelligence toward “human flourishing.”

About a week before the privacy and technology luminary Peter Eckersley unexpectedly died last September, he reached out to artificial intelligence entrepreneur Deger Turan. Eckersley wanted to persuade Turan to be the president of Eckersley's brainchild, a new institute that aimed to do nothing less ambitious than course-correct AI's evolution to safeguard the future of humanity.

Eckersley explained he couldn't run this project himself: He was having serious health issues due to colon cancer, and the organization's co-founder, Brittney Gallagher, was very pregnant and about to go on maternity leave. But Turan wouldn't be alone, Eckersley assured him—as soon as his illness was resolved, he'd be back to serve as the group's chief scientist. They agreed to meet in San Francisco a few days later to hash out a plan.

By the time Turan's plane landed in San Francisco, Eckersley had died—a tragedy that has sent still-resonating shockwaves through his friends, family, and the tech world. Instead of a meeting with Eckersley to draw up the institute's roadmap, Turan found himself attending his friend's funeral.

In the preceding days, while still fully expecting to recover, Eckersley had already told the board of his nascent organization—the AI Objectives Institute, or AOI—that Turan would be its president. The 44-year-old technologist and activist had also written a rough, incomplete will in Google Docs, in the unlikely event of his death. It began by naming AOI as the inheritor of all his US-based assets. “We started something important,” Eckersley wrote. “I'd want to see whether the people involved could get it a little further.”

Turan had never actually had the chance to tell Eckersley he accepted his request. But as soon as he learned of Eckersley's death, he knew that the role at AOI was not only the most important work he could be doing but also a way to help establish a central pillar of his friend's legacy. “So I said yes,” Turan says. “Let's do this—not a little further, but all the way.”

Peter Eckersley on an evening bike ride in San Francisco in 2021.

Photograph: Laura Helen Winn

Yesterday, hundreds in Eckersley's community of friends and colleagues packed the pews for an unusual sort of memorial service at the church-like sanctuary of the Internet Archive in San Francisco—a symposium with a series of talks devoted not just to remembrances of Eckersley as a person but a tour of his life's work. Facing a shrine to Eckersley at the back of the hall filled with his writings, his beloved road bike, and some samples of his Victorian goth punk wardrobe, Turan, Gallagher, and 10 other speakers gave presentations about Eckersley's long list of contributions: his years pushing Silicon Valley towards better privacy-preserving technologies, his co-founding of a groundbreaking project to encrypt the entire web, and his late-life pivot to improving the safety and ethics of AI.

The event also served as a kind of soft launch for AOI, the organization that will now carry on Eckersley's work after his death. Eckersley envisioned the institute as an incubator and applied laboratory that would work with major AI labs to that take on the problem Eckersley had come to believe was, perhaps, even more important than the privacy and cybersecurity work to which he'd devoted decades of his career: redirecting the future of artificial intelligence away from the forces causing suffering in the world, toward what he described as “human flourishing.”

“We need to make AI not just who we are, but what we aspire to be,” Turan said in his speech at the memorial event, after playing a recording of the phone call in which Eckersley had recruited him. “So it can lift us in that direction.”

The mission Eckersley conceived of for AOI emerged from a growing sense over the last decade that AI has an “alignment problem”: That its evolution is hurtling forward at an ever-accelerating rate, but with simplistic goals that are out of step with those of humanity's health and happiness. Instead of ushering in a paradise of superabundance and creative leisure for all, Eckersley believed that, on its current trajectory, AI is far more likely to amplify all the forces that are already wrecking the world: environmental destruction, exploitation of the poor, and rampant nationalism, to name a few.

AOI's goal, as Turan and Gallagher describe it, is not to try to restrain AI's progress but to steer its _objectives_ away from those single-minded, destructive forces. They argue that's humanity's best hope of preventing, for instance, hyperintelligent software that can brainwash humans through advertising or propaganda, corporations with god-like strategies and powers for harvesting every last hydrocarbon from the earth, or automated hacking systems that can penetrate any network in the world to cause global mayhem. “AI failures won't look like nanobots crawling all over us all of the sudden,” Turan says. “These are economic and environmental disasters that will look very recognizable, similar to the things that are happening right now.”

Gallagher, now AOI's executive director, emphasizes that Eckersley's vision for the institute wasn't that of a doomsaying Cassandra, but of a shepherd that could guide AI toward his idealistic dreams for the future. “He was never thinking about how to prevent a dystopia. His eternally optimistic way of thinking was, ‘how do we make the utopia?’” she says. “What can we do to build a better world, and how can artificial intelligence work toward human flourishing?”

To that end, AOI is already working on a handful of example projects to push AI onto that path, now with the help of nine core contributors and a handful of grants including $485,000 from the Survival and Flourishing fund. The pilot project that's furthest along, called “Talk to the City,” is designed to use a ChatGPT-like interface to survey millions of people in a city to both understand their needs and to advocate for them in discussions with policymakers, journalists, and other citizens. Turan describes the experiment as a tool for collective organizing and for governments, enabling a form of democracy more nuanced than simple elections or referendums. He says interested beta testers for the project include everyone from the organizers of Burning Man's Black Rocky City to staffers at the United Nations.

Another prototype, called “Mindful Mirror,” will serve as a kind of personal interactive journal, a chatbot that converses with its user to help them process the events of their daily life. A third, called “Lucid Lens,” will function as a browser plugin that highlights content it detects as being designed to cause outrage or “dopamine loops” that manipulate users in ways they'd rather be aware of or avoid.

Those initial projects might sound modest compared to AOI's lofty futurist goals. But the story of Eckersley's long, renowned career in cybersecurity and privacy was one of building similarly simple-sounding tools that could serve as levers to effect profound changes. Working for the Electronic Frontier Foundation (EFF) for a dozen years, eventually as its chief scientist, Eckersley helped build projects like Privacy Badger, a browser plugin to stop web trackers, and HTTPS Everywhere, another plugin that made your browser navigate to the HTTPS-encrypted version of a website whenever possible. He co-created the SSL Observatory, which scanned the entire internet to determine how much of it was encrypted. Another Eckersley project, a site called Panopticlick, audited a user's browser protection against tracking. And the EFF's Secure Messaging Scorecard rated messaging apps on their privacy and security features, helping usher in a world where billions of WhatsApp users' messages are end-to-end encrypted by default.

“His genius was finding the tiny little hack that would open up a big story,” EFF executive director Cindy Cohen said in her speech at Eckersley's memorial service. “The demonstration that would make manifest what people needed to see about how technology was working.”

Perhaps Eckersley's most celebrated work of all was his cofounding of Let's Encrypt, a free alternative to the certificate authority companies that enable website owners to use HTTPS encryption. By removing a key hurdle to switching on a site's encryption, Let's Encrypt measurably transformed the web: Let's Encrypt has so far issued free certificates to more than 300 million websites. Today, more than 90 percent of the web is encrypted, compared to less than an estimated 40 percent when Let's Encrypt launched in 2015. “We have Peter Eckersley to thank for that,” Let's Encrypt co-founder Alex Halderman said at Eckersley's memorial.

Even as Eckersley was in the midst of launching those influential projects, he was already thinking about an entirely different area of technology where he felt he might make an even bigger long-term impact. By 2013, Eckersley was talking to computer scientists like Anders Sandberg, Stuart Russell, and Nick Bostrom who were focused on “civilizational risk” from AI, says Brian Christian, a scientist and author of the AI-focused books _The Most Human Human_ and _Algorithms to Live By_, and who served as the master of ceremonies for Eckersley's memorial event.

By the time he left the EFF in 2018, Christian says, Eckersley had decided it was time to re-focus his efforts on shaping AI's future. “He saw it has higher stakes, in a way,” says Christian. “He ultimately ended up concluding that the gravity of AI, even in its more hypothetical harms, was so great that it felt urgent, that it was the most important thing to him.” Christian says Eckersley was so persuasive about the magnitude of the problem that he transformed Christian's thinking about AI's future, too. He dedicated his 2020 book on the topic, _The Alignment Problem_, “to Peter, who convinced me.”

Eckersley's sister Nicole, who gave the final speech of the evening, said that she had begun to hear from her brother about his vision for something like AOI in 2020. And even when his health suddenly took a precipitous decline in the late summer of 2022, the institute remained his focus. “Even from his hospital bed, he was charging full speed ahead on AOI. All of his last wishes and instructions were about the survival of this incredibly important project,” she said. “We want to see Peter's plans come to fruition. We want to keep engaged with this incredible community. We want to stop the robots from eating us and crapping out money.”

“So I hope you'll all see this not just as a memorial,” she concluded, “but as the start of an incredible living legacy.”

A Privacy Hero's Final Wish: An Institute to Redirect AI's Future

Plus: The US Marshals disclose a “major” cybersecurity incident, T-Mobile has gotten pwned so much, and more.

Chinese hackers proved themselves to be as prolific and invasive as ever this week with new findings revealing that in February 2022, Beijing-backed hackers compromised the email server of the Association of Southeast Asian Nations, an intergovernmental body of 10 Southeast Asian countries. The security alert, first reported by WIRED, comes as China has escalated its hacking in the region amidst rising tensions.

Meanwhile, with Russia facing economic sanctions over its invasion of Ukraine, the Kremlin has been trying to address gaps in its tech sector. Now, we've learned, it's scrambling to get a home-brewed Android phone off the ground this year. The National Computer Corporation company, a Russian IT giant, says it will somehow produce and sell 100,000 smartphones and tablets by the end of 2023. Though Android is an open-source platform, there are steps Google could take to restrict the license for the new Russian phone that could ultimately force the project to seek a different mobile operating system.

At the Network and Distributed System Security Symposium in San Diego this week, researchers from Ruhr University Bochum and the CISPA Helmholtz Center for Information Security presented findings that popular DJI quadcopters communicate using unencrypted radio signals that can be intercepted to determine where the drones are, as well as the GPS coordinates of their operators. The researchers discovered the exposed communications by reverse engineering DJI's radio protocol, DroneID.

In the US, a long-awaited national cybersecurity plan from the White House finally debuted on Thursday. In focuses in part on familiar priorities like hardening defenses for critical infrastructure and and expanding efforts to disrupt cybercriminal activity. But the plan also includes a proposal to shift legal liability for vulnerabilities and security failures onto the companies who cause them, like software makers or institutions that don't make a reasonable effort to protect sensitive data.

If you want to do something good for your cyber hygiene this weekend, we've got a roundup of the most pressing software patches to download ASAP. Seriously, go install them now, we'll wait here.

And there's more. Each week, we round up the security news we didn’t cover in-depth ourselves. Click the headlines to read the full stories, and stay safe out there.

In December, the password-manager maker LastPass revealed that an August breach it had disclosed at the end of November was worse than the company originally thought, compromising encrypted copies of some users’ password vaults, on top of other personal information. Now, the company has disclosed a second incident that began in mid-August and allowed attackers to rampage through the company's cloud storage and exfiltrate sensitive data. Attackers gained such extraordinary access by targeting a specific LastPass employee with deep system privileges 

“This was accomplished by targeting \[a\] DevOps engineer’s home computer and exploiting a vulnerable third-party media software package, which enabled remote code execution capability and allowed the threat actor to implant keylogger malware,” LastPass wrote in an account of the situation. “The threat actor was able to capture the employee’s master password as it was entered, after the employee authenticated with MFA, and gain access to the DevOps engineer’s LastPass corporate vault.”

To target the LastPass employee, attackers exploited a Plex Media Server software vulnerability that had already been long-patched at the time. The company issued a fix for the bug in May 2020, “roughly 75 versions ago,” Plex said.

US law enforcement officials said on Monday that a stand-alone US Marshals Service network suffered a data exfiltration and ransomware attack in mid-February. “The affected system contains law enforcement sensitive information, including returns from legal process, administrative information, and personally identifiable information pertaining to subjects of USMS investigations, third parties, and certain USMS employees,” Marshals Service spokesperson Drew Wade said in a statement. The impacted data seemingly did not include information from the Witness Security Program or witness protection database. Nonetheless, Wade said that officials had “determined that it constitutes a major incident.”

Three cybercriminal groups that conduct SIM-swapping attacks have claimed that they repeatedly hacked T-Mobile last year as part of their scams. The groups would target T-Mobile employees with phishing attacks to gain access to internal company systems. Then they would sell this access to other cybercriminals to intercept individual T-Mobile customers’ SMS text messages and calls on attacker-controlled devices. The findings come from an analysis by Krebs on Security of Telegram chat activity of the three SIM-swapping gangs.

T-Mobile declined to confirm or deny the claims to Krebs on Security. “We have continued to drive enhancements that further protect against unauthorized access, including enhancing multi-factor authentication controls, hardening environments, limiting access to data, apps or services, and more,” the telecom said in a statement. “We are also focused on gathering threat intelligence data, like what you have shared, to help further strengthen these ongoing efforts.”

A bill filed last week in Texas by Representative Steve Toth would mandate that Texas internet service providers block websites that offer information about receiving abortion care. The bill would also outlaw domain registration and hosting for websites that help Texas residents obtain abortions, either through fundraising, procuring abortifacient drugs, or sharing resources. The proposal lists specific examples of websites that would have to be blocked, including aidaccess.org, heyjane.co, plancpills.org, mychoix.co, justthepill.com, and carafem.org.

The LastPass Hack Somehow Gets Worse

Employee monitoring increased with Covid-19’s remote work—and stuck around for back-to-the-office.

You're not being paranoid. If you always feel like somebody's watching you, as the song goes, you're probably right. Especially if you're at work.

Over the course of the Covid-19 pandemic, as labor shifted to work-from-home, a huge number of US employers ramped up the use of surveillance software to track employees. The research firm Gartner says 60 percent of large employers have deployed such monitoring software—it doubled during the pandemic—and will likely hit 70 percent in the next few years.

That's right—even as we've shifted toward a hybrid model with many workers returning to offices, different methods of employee surveillance (dubbed "bossware" by some) aren't going away; it's here to stay and could get much more invasive. 

As detailed in the book _Your Boss Is an Algorithm_, authors Antonio Aloisi and Valerio de Stefano describe "expanded managerial powers" that companies have put into place over the pandemic. This includes the adoption of more tools, including software and hardware, to track worker productivity, their day-to-day activities and movements, computer and mobile phone keystrokes, and even their health statuses. 

This can be called "datafication" or "informatisation," according to the book, or "the practice by which every movement, either offline or online, is traced, revised and stored as necessary, for statistical, financial, commercial and electoral purposes."

Ironically, experts point out that there's not sufficient data to support the idea that all this data collection and employee monitoring actually increases productivity. But as the use of surveillance tech continues, workers should understand how they might be surveilled and what, if anything, they can do about it.

What Kind of Monitoring Is Happening?

Using surveillance tools to monitor employees is not new. Many workplaces continue to deploy low-tech tools like security cameras, as well as more intrusive ones, like content filters that flag content in emails and voicemails or unusual activity on work computers and devices. The workplace maxim has long been that if you're in the office and/or using office phones or laptops, then you should never assume any activity or conversation you have is private.

But the newer generation of tools goes beyond that kind of surveillance to include monitoring through wearables, office furniture, cameras that track body and eye movement, AI-driven software that can hire as well as issue work assignments and reprimands automatically, and even biometric data collection through health apps or microchips implanted inside the body of employees.

Some of these methods can be used to track where employees are, what they’re doing at any given moment, what their body temperature is, and what they’re viewing online. Employers can collect data and use it to score workers on their individual productivity or to track data trends across an entire workforce.

These tools aren't being rolled out only in office spaces, but in work-from-home spaces and on the road to mobile workers such as long-haul truck drivers and Amazon warehouse workers.

Is This Legal?

As you might imagine, the laws of the land have had a hard time keeping up with the quick pace of these new tools. In most countries, there are no laws specifically forbidding employers from, say, video-monitoring their workforce, except in places where employees should have a “reasonable expectation of privacy,” such as bathrooms or locker rooms.

In the US, the 1986 Electronic Communications Privacy Act laid out the rule that employees should not intercept employee communication, but its exceptions—that they can be intercepted to protect the privacy and rights of the employer or if business duties require it, or if the employee granted prior permission—make the law toothless and easy to get around.

What to Do When Your Boss Is Spying on You

The Biden administration’s new strategy would shift the liability for security failures to a controversial target: the companies that caused them.

In the endless fight to improve cybersecurity and encourage investment in digital defenses, some experts have a controversial suggestion. They say the only way to make companies take it seriously is to create real economic incentives—by making them legally liable if they have not taken adequate steps to secure their products and infrastructure. The last thing anyone wants is more liability, so the idea has never exploded in popularity, but a national cybersecurity strategy from the White House this week is giving the concept a prominent boost.

The long-awaited document proposes stronger cybersecurity protections and regulations for critical infrastructure, an expanded program to disrupt cybercriminal activity, and a focus on global cooperation. Many of these priorities are widely accepted and build on national strategies put out by past US administrations. But the Biden strategy expands significantly on the question of liability.

“We must begin to shift liability onto those entities that fail to take reasonable precautions to secure their software while recognizing that even the most advanced software security programs cannot prevent all vulnerabilities,” it says. “Companies that make software must have the freedom to innovate, but they must also be held liable when they fail to live up to the duty of care they owe consumers, businesses, or critical infrastructure providers.”

Publicizing the strategy is a way of making the White House's priorities clear, but it does not in itself mean that Congress will pass legislation to enact specific policies. With the release of the document, the Biden administration seems focused on promoting discussion about how to better handle liability as well as raising awareness about the stakes for individual Americans.

“Today, across the public and private sectors, we tend to devolve responsibility for cyber risk downwards. We ask individuals, small businesses, and local governments to shoulder a significant burden for defending us all. This isn’t just unfair, it’s ineffective,” acting national cyber director Kemba Walden told reporters on Thursday. “The biggest, most capable, and best-positioned actors in our digital ecosystem can and should shoulder a greater share of the burden for managing cyber risk and keeping us all safe. This strategy asks more of industry, but also commits more from the federal government.”

Jen Easterly, director of the US Cybersecurity and Infrastructure Security Agency, had a similar sentiment for an audience at Carnegie Mellon University earlier this week. “We often blame a company today that has a security breach because they didn’t patch a known vulnerability,” she said. “What about the manufacturer that produced the technology that required too many patches in the first place?”

The goal of shifting liability to large companies has certainly started a conversation, but all eyes are on the question of whether it will actually result in change. Chris Wysopal, founder and CTO of the application security firm Veracode, provided input to the Office of the National Cyber Director for the White House strategy.

“Regulation in this area is going to be complicated and tricky, but it can be powerful if done appropriately,” he says. Wysopal likens the concept of security liability laws to environmental regulations. “You can’t simply pollute and walk away; businesses will need to be prepared to clean up their mess.”

The comparison underscores how resistant businesses will likely be to such a transition, though, particularly large, legacy tech companies whose products are used widely around the US and the world.  “Some companies will welcome the strategy more than others,” Wysopal concedes.

Shawn Tuma, a partner in the law firm Spencer Fane who specializes in cybersecurity and data privacy issues, emphasizes that from an industry perspective, “the devil is in the details” on all these proposals. On legal liability, he says the debate comes down to what exactly is meant by “reasonable.”

“We all see the extremes in the continuum—we see the providers that are doing a poor job, that are just throwing stuff out there,” he says. “I’m fine for liability on them, but what about those that are trying to do their best but are engaged in an unwinnable war with well-resourced hackers? What’s ‘reasonable’?”

One point from the strategy that might see more movement is the Biden administration's proposal for some sort of federal backstop to help stabilize the cybersecurity insurance market. If liability for cybersecurity failures were to shift in any meaningful way, cybersecurity insurance would become even more vital than it already is for tech companies and others who hold sensitive data, like health care firms. But that's assuming insurance companies will cover cybersecurity incidents at all.

In late December, Mario Greco, CEO of the massive European insurer Zurich, told the _Financial Times_, “What will become uninsurable is going to be cyber.” The comment, made a day after Christmas, added an edge to an already tense climate in which companies grasp for safeguards and solutions as cybercriminal and nation-state attacks impose rapidly rising costs.

A government backstop like the one the national cybersecurity strategy is proposing could provide crucial reassurances, but Tuma points out that it could also come with strings attached for the insurance industry and its clients. He suggests the US government could mandate that, in exchange for its support, anyone who makes cybersecurity insurance claims would be required to report the incident to the FBI's Internet Crime Complaint Center. “They need more cooperation from the private sector in reporting these events,” Tuma says.

And this question of how to incentivize all different facets of cybersecurity investment is at the core of what the new White House strategy is grappling with.

“I feel the White House is very serious about this,” Veracode's Wysopal says. “The public-private partnership around cybersecurity is quite real in the federal government today. That is a welcome change from just a few years ago.”

The High-Stakes Blame Game in the White House Cybersecurity Plan

Amid isolating sanctions, a Russian tech giant plans to launch new Android phones and tablets. But experts are skeptical the company can pull it off.

The fresh wave of sanctions after the invasion of Ukraine has given new wind to the concept of technological self-sufficiency, with Russia’s government launching multiple initiatives to create domestic substitutes for foreign electronics, online platforms, and software, on which many Russian companies are dependent.

Over a thousand tech companies stopped or curtailed their operations in Russia after the invasion of Ukraine. In just a month, Cisco, SAP, Oracle, IBM, TSMC, Nokia, and Ericsson, as well as Samsung and Apple, left the market, affecting entire industries, including mobile operators, factories, startups, and large state-owned companies. According to IDC, a global market-analysis firm, the Russian IT market in 2022 shrank by $12.1 billion, or 39 percent.

Russian prime minister Mikhail Mishustin said in February that Russia wants to replace 85 percent of foreign software with Russian substitutes, opening dozens of so-called import substitution centers. Among them is a project to create a national operating system for devices. The plan, however, is at an early stage with no road map in sight, says the Internet Research Institute’s Kazaryan.

“Currently, a few big players are trying to woo the government for subsidies to create devices on ‘national mobile OS,’ whatever that might be,” he says.

One of the more promising alternatives to Android is Aurora OS, a Linux-based smartphone operating system made by the Russian state-owned telecommunications firm Rostelecom. But Aurora was primarily made for government use and does not support Android apps. In December, the Russian government refused to allocate any of the 22 billion rubles ($292.1 million) earmarked for the development of the Russian operating system.

Other Russian smartphone makers, such as BQ, have promised to adapt Huawei’s HarmonyOS for its handsets. But there's been no news of progress since BQ’s announcement in September. Huawei, which is based in China, developed its own operating system in 2019 after Google stopped providing its suite of mobile software services to the company because of US trade sanctions. The Chinese IT giant has said it has no plans to launch HarmonyOS-equipped mobile phones outside of China.

Huawei’s struggle to compete outside China shows that it’s hard for a smartphone brand to gain buyers without access to Google services. Huawei lost almost a third of its revenue in 2020, the year after sanctions cut its access to Google Maps, Gmail, and other common Google apps. The biggest hurdle NCC’s new smartphone may face, however, is cheap and readily available phones from China. 

Counterpoint’s data shows that phones from Xiaomi, Realme, and Honor, a budget brand previously owned by Huawei, have replaced once best-selling iPhones and Samsung Galaxy devices, accounting for 95 percent of the market last year.

“There's still a lot of competition,” says Counterpoint’s Stryjak. “I don't think there's a huge gap in the market for a new player.”

Other Russian phones, most famously Yotaphone, have tried to capture the domestic market, but they remained at a very small scale, says Stryjak. Russians prefer brands they already know. Thanks to parallel trading—importing goods without the permission of the manufacturer—even Samsungs and iPhones are still available in the country. NCC says it aims to price its smartphones between 10,000 and 30,000 rubles ($132 and $398). 

“We are considering various options. This could be production at Russian factories or contract manufacturing at Chinese enterprises,” NCC’s Kalinin told local media. NCC did not respond to WIRED’s request for comment.

Other Russian smartphone makers such as Smartekosistema, owned by state giant Rostec, have found that they were unable to procure the necessary chips from TSMC for the second iteration of their handset, the AYYA T2. All of this may make creating Russian challengers to Samsung or Apple very expensive.

“You probably can make a smartphone in Russia with Chinese parts, but it's not very efficient,” says Kazaryan. “And why would anyone buy a Russian phone that is more expensive than Xiaomi?”

The Sketchy Plan to Build a Russian Android Phone

Every DJI quadcopter broadcasts its operator's position via radio—unencrypted. Now, a group of researchers has learned to decode those coordinates.

There’s a reason consumer drones have evolved from an expensive toy into a tool of war: They can perform high-altitude surveillance, carry out reconnaissance, or even deploy weapons, with their operator safely hidden as far as miles away. But hackers are revealing that for quadcopters sold by the world’s biggest drone manufacturer, operators aren’t nearly as hidden as they might think. In fact, these small flying machines are continually broadcasting their pilots’ exact locations from the sky, and anyone with some cheap radio hardware and a newly released software tool can eavesdrop on those broadcasts and decode them to extract their coordinates.

At the Network and Distributed System Security Symposium (NDSS) in San Diego this week, researchers from Ruhr University Bochum and the CISPA Helmholtz Center for Information Security demonstrated that they were able to reverse engineer the radio signals of drones sold by DJI, the leading manufacturer of consumer quadcopter drones, to decode a radio protocol they use called DroneID. By deconstructing this signal, the researchers could see that every DJI drone’s DroneID communications transmit not only its own GPS location and a unique identifier for that drone, but also the GPS coordinates of its operator.

A screenshot of the German researchers’ tool shows how it decodes the radio broadcasts of a DJI drone to extract both the drone and operator locations.

Courtesy of Ruhr University Bochum and CISPA Helmholtz Center for Information Security

That DroneID system was designed to allow governments, regulators, and law enforcement to monitor drones and prevent their abuse. But hackers and security researchers have warned for the past year that DroneID is unencrypted and open to anyone who can receive its radio signals. The German researchers, as well as another researcher working separately at the University of Tulsa, have now shown just how completely that signal can be decoded and read, allowing any hacker who can eavesdrop on DroneID to pinpoint a drone’s hidden operator, even if that drone pilot is miles away.

The German group has released a prototype tool to receive and decode that DroneID data here.

The researchers’ findings—and their public tool—provide new evidence of the serious privacy and operational security concerns DroneID presents for operators, especially considering that DJI drones are now often used in war zones, where revealing a drone operator’s location can draw enemy fire. And while DJI has an enormous majority share of the consumer drone market, the problem will only grow when new US Federal Aviation Administration regulations go into effect in September, mandating that all consumer drones implement systems similar to DroneID.

“This is a big problem, right?” says Moritz Schloegel, one of the Ruhr University graduate researchers presenting the DroneID findings at NDSS. “You might think your drone transmits its position. But suddenly, it’s transmitting _your_ position as well. Whether you’re privacy-minded or you’re in a conflict zone, nasty stuff can happen.”

DJI’s DroneID became the subject of controversy last spring when the Ukrainian government criticized the company because Russian military forces were using DJI drones for their missile targeting and using the radio signals broadcast from Ukraine’s own DJI drones to locate Ukrainian military personnel. China-based DJI has long sold a suitcase-sized device called Aeroscope to government regulators and law enforcement agencies that allows them to receive and decode DroneID data, determining the location of any drone and its operator from as far as 30 miles away.

DJI’s DroneID and Aeroscope devices are advertised for civilian security uses, like preventing disruptions of airport runways, protecting public events, and detecting efforts to smuggle cargo into prisons. But Ukraine’s vice minister of defense wrote in a letter to DJI that Russia had repurposed Aeroscope devices from Syria to track Ukrainian drones and their operators, with potentially deadly consequences.

DJI responded by warning against any military use of its consumer drones and later cutting off all sales of its drones to both Ukraine and Russia. It also initially claimed in response to the Verge’s reporting on the controversy that DroneID was encrypted, and thus inaccessible to anyone who didn’t have its carefully controlled Aeroscope devices. But DJI later admitted to the Verge that the transmissions were _not_ in fact encrypted, after security researcher Kevin Finisterre showed that he could intercept some DroneID data with a commercially available Ettus software-defined radio.

The German researchers—who also helped debunk DJI’s initial encryption claim—have gone further. By analyzing the firmware of a DJI drone and its radio communications, they’ve reverse engineered DroneID and built a tool that can receive DroneID transmissions with an Ettus software-defined radio or even the much cheaper HackRF radio, which sells for just a few hundred dollars compared to over $1,000 for most Ettus devices. With that inexpensive setup and their software, its possible to fully decode the signal to find the drone operator’s location, just as DJI’s Aeroscope does.

While the German researchers only tested their radio eavesdropping on a DJI drone from ranges of 15 to 25 feet, they say they didn’t attempt to optimize for distance, and they believe they could extend that range with more engineering. Another hacker, University of Tulsa graduate researcher Conner Bender, quietly released a pre-publication paper last summer with similar findings that will be presented at the CyCon cybersecurity conference in Estonia in late May. Bender found that his HackRF-based system with a custom antenna could pick up DroneID data from hundreds or thousands of feet away, sometimes as far as three-quarters of a mile.

WIRED reached out to DJI for comment in multiple emails, but the company hasn’t responded. The former DJI executive who first conceived of DroneID, however, offered his own surprising answer in response to WIRED’s query: DroneID is working exactly as it’s supposed to.

Brendan Schulman, DJI’s former VP of policy and legal affairs, says he led the company’s development of DroneID in 2017 as a direct response to US government demands for a drone-monitoring system, and that it was never intended to be encrypted. The  FAA, federal security agencies, and Congress were strongly pushing at the time for a system that would allow anyone to identify a drone—and its operator’s location—as a public safety mechanism, not with hacker tools or DJI’s proprietary ones, but with mobile phones and tablets that would allow for easy citizen monitoring.

“As we were told in 2017 during a summer-long FAA advisory committee process, the location of the operator is an essential aspect of remote identification for US government security purposes,” Schulman says. “And the US government _wanted_ members of the public to have access to that information, just like how a car’s license plate is accessible to everyone who can see it, so they can file a report with authorities if they have concerns about how a drone is being used.”

Schulman notes that he advocated for that broadcasting system over what he saw as a far more invasive suggestion from the government, that drone makers should both broadcast operators’ locations _and_ connect all drones to a network of drone-monitoring services that would record every operator’s detailed flight records in government-accessible databases. He also notes that the DroneID issue isn’t unique to DJI:  He expects that all consumer drones will have a function similar to DroneID when the new FAA regulations take effect later this year.

But none of that changes the fact that DJI drone operators don’t expect to have their locations revealed by their drone’s radio broadcasts, says University of Tulsa’s Bender. “The average drone user definitely doesn’t know that their location is being broadcasted in a way anyone with a cheap receiver can view in real time,” Bender says. He adds that DJI’s handling of the issue—claiming last year that the broadcast was encrypted when it wasn’t—further confused users. “I don’t know if they intentionally marketed Aeroscope this way, but they made it seem like you could really only intercept DroneID with this one device. And that wasn’t the case.”

Regardless of DJI’s motives in including drone pilots’ location in the data their drones continually transmit, the fact that this location data can be intercepted—not just with DJI’s Aeroscope devices but by any knowledgeable hacker—will have a significant impact on how the world’s most common quadcopter drones are used in war zones and other adversarial settings, says August Cole, a futurist and fellow at the Scowcroft Center for Strategy and Security at the Atlantic Council.

“The ability to ID an operator of a drone is sort of the holy grail right now in terms of targeting,” Cole says. “And to be able to do this so easily, when a drone maker adds that through either intentional or unintentional engineering, it’s a pretty profound revelation for this new kind of warfare.”

This Hacker Tool Can Pinpoint a DJI Drone Operator's Exact Location

New details reveal that Beijing-backed hackers targeted the Association of Southeast Asian Nations, adding to a string of attacks in the region.

In May 2022, Joe Biden was on a charm offensive. The US president invited the leaders of 10 Southeast Asian nations to the White House for the first time for talks about the region, which is home to more than 600 million people. High on the agenda was China—a key trading partner for all the countries, but also a potential threat to their stability. Biden promised $150 million in extra support for the nations to help improve their security, infrastructure, and ongoing pandemic response. 

However, in the weeks leading up to the meeting, according to a cybersecurity alert seen by WIRED, hackers working on behalf of China were stealing thousands of emails and sensitive details from the Southeast Asian nations. The cyberespionage, which has not been previously reported, is the latest in a string of incidents where Chinese-linked hackers have quietly compromised neighboring countries, looking to gain political and economic information.

According to the cybersecurity alert, Chinese-linked hackers were able to break into mail servers operated by the Association of Southeast Asian Nations (ASEAN) in February 2022 and steal a trove of data. The ASEAN organization is an intergovernmental body made up of 10 Southeast Asian countries, including Singapore, Malaysia, and Thailand. This was the third time the organization has been compromised since 2019, the document says.

The hackers were able to steal “gigabytes” of emails sent by ASEAN countries, and the data was stolen “daily,” according to the cybersecurity alert. It’s believed that the attackers stole more than 10,000 emails, making up more than 30 GB of data. The incident “impacts all ASEAN members due to correspondence that was compromised,” the alert says. The notification was sent to cybersecurity agencies, foreign affairs ministries, and other governmental organizations in all 10 of the ASEAN member countries. 

Haji Amirudin Abdul Wahab, the CEO of CyberSecurity Malaysia, an agency under the country’s Ministry of Science, Technology, and Innovation, says it received the alert in 2022, notified officials within the country, and generally condemns hacking. Other nations impacted declined to comment or did not respond to WIRED’s request for comment. The ASEAN group itself did not respond to repeated requests for comment. 

China's embassy in the US did not immediately respond to a request for comment.

Amplified Voices, Quiet Theft

“ASEAN is really important as the key regional grouping, not just in Southeast Asia but beyond,” says Susannah Patton, director of the Southeast Asia Program at Australian think tank the Lowy Institute. Patton explains that ASEAN helps coordinate Southeast Asian policies across a number of different areas. “Even beyond Southeast Asia, ASEAN has an important role because it convenes or organizes other big regional summits,” Patton says. As a result, the data it holds could be useful for understanding political feelings in the region.

ASEAN helps to “amplify” the voices of the 10 member countries that are involved in it, says Scot Marciel, an Oksenberg-Rohlen Fellow at Stanford University and former US ambassador to Indonesia and Myanmar. The group holds both formal meetings and informal conversations, Marciel says, and will discuss everything from economic integration and infrastructure plans to trade negotiations and geopolitics. “That would all be stuff that I would think Beijing would be interested in,” Marciel says.

Source

Wired