Mark Zuckerberg personally promised that the privacy feature would launch by default on Messenger and Instagram chat. WIRED goes behind the scenes of the company’s colossal effort to get it right.

Since 2016, the social behemoth now known as Meta has been working to deploy end-to-end encryption in its communication apps. CEO Mark Zuckerberg even promised in 2019 that the data privacy protection would roll out by default across all of the company's chat apps. In practice, though, it was a wildly ambitious goal fraught with technical and political challenges, and Meta has only been able to move toward it in gradual, incremental steps. But this week the company is finally starting its full rollout.

“It's been a wild ride," says Jon Millican, a software engineer within Meta's messenger privacy team. “I suspect this is the first time that something’s been end-to-end encrypted with all of the constraints that we’re working with. It’s not just that we’re migrating people’s data, but it’s actually that we're having to fundamentally change a bunch of the assumptions that they work with when they’re using the product.”

Meta has had to stake out a position as a committed proponent of end-to-end encryption amid pressure from law enforcement and victim advocacy groups that the privacy feature—which makes data unintelligible everywhere except on the devices of the sender and recipient—limits necessary oversight and impedes crucial police investigations. Meanwhile, the company has spent the past four years, not to mention the better part of a decade, developing the technology to retrofit two massive communication platforms—Messenger and Instagram chat—such that they could still offer the features and general experience users expect under the technical constraints and usability challenges of end-to-end encryption.

“I understand that many people don't think Facebook can or would even want to build this kind of privacy-focused platform—because frankly, we don't currently have a strong reputation for building privacy-protective services, and we've historically focused on tools for more open sharing,” Zuckerberg memorably wrote in his 2019 treatise. But he added that there was a clear desire from users to have access to private and secure encrypted communication services. “This is the future I hope we will help bring about,” he wrote.

Meta says that it will take some time for the rollout of full default end-to-end encryption to reach all Messenger and Instagram chat users, and the feature is still only launching for direct messages between two accounts. End-to-end encryption for group chats will continue to be opt-in for now. But these final delays have to do with gradually converting billions of accounts to run the cryptography and encrypted storage schemes that underly the effort. And while the infrastructure is new and had to be painstakingly tailored to Meta's services, the company says it built the system on the Signal Protocol and thoroughly vetted the implementation both internally and with independent experts. In the lead-up to this announcement, the company did a final round of outreach to privacy groups and cryptographers to show them the documentation and have them test the feature.

“It looks just like Messenger, except that under the hood it has really strong encryption,” says Matt Green, a Johns Hopkins cryptographer who previewed the launch a few weeks ago. “Getting things to work on the web seems like it was the hard part, but they pulled it off.”

The challenge of building end-to-end encrypted services has to do with the fact that such systems inherently blind the servers that enable them to the activity they are facilitating. In other words, these systems have to somehow stand in the hallway on the first day of school and tell each student which classes to go to and how to get there without knowing who any of the kids are or what their course schedule is.

This especially poses challenges for syncing people's messages across multiple devices or repopulating their messages on a new device. Some encrypted chat apps like Signal address this issue by storing all your messages locally on your device and then providing a tool that helps you transfer that data trove from one device to the next over Bluetooth when you, say, get a new phone and want to switch everything over. This approach doesn't preserve your history if you lose the device where the data is saved, and many users around the world don't have resources or access to devices with enough storage space to preserve messages locally. People may want to auto-delete messages anyway, but for users who want to save their history, such schemes can be impractical.

With these considerations in mind, Meta engineers developed an encrypted storage protocol, dubbed Labyrinth, that allows the company to store users' chat histories and other communication data on its servers for ease of use, but in a form that is always encrypted and inaccessible to the company.

“There are two things users will experience” in the transition, Gail Kent, Messenger's global policy director, tells WIRED. “They will be asked to create secure storage and create a pin that will then enable them to add the data and the messages onto other devices and restore if they lose their device. Yet nobody apart from them has access to that message content. It seems like a tiny thing, but the innovation behind it is pretty extraordinary. And then the second thing that they’ll see is a line in their conversations that says this message is now end-to-end encrypted. And when they start a new conversation, it will say that at the start of the conversation.”

Turning on end-to-end encryption for cloud services makes it harder for companies to assist their users with data recovery. It also requires thoughtful strategies for keeping systems usable while striking a balance with security. The challenge is one that companies like Apple have grappled with as well. Like many of its peers, Meta will offer multiple options for protecting and accessing Labyrinth secure storage, including setting a pin, storing a code in a third-party service like iCloud, or saving a 40-character recovery code.

Meta is also providing an option for people to use end-to-end encrypted communication without turning on secure storage at all if they want to store messages locally on one or all devices. Currently, the company isn't offering a mechanism to transfer this local data onto other devices. A core component of Labyrinth, though, is what's known in cryptography as a “key rotation” system for revoking a device's access and keeping it from viewing or sending messages on an account if a user wants to remove it.

Meta's engineers have spent a huge amount of time over the past four years re-architecting more than 100 Messenger and Instagram chat features so they interoperate with end-to-end encryption and seem to function the same way they always have—even if they're working differently on a technical level under the hood. Extra features like themes, custom reactions, and emojis are all preserved. The company also announced new features this week like the ability to edit a message and control whether other users see a read receipt once you've viewed a message.

Initial reactions to the launch have already begun to mirror the broader controversy around end-to-end encryption, with privacy advocates lauding the move and law enforcement and victim defense groups, particularly those that aim to address online child sexual abuse, condemning the step. After years of receiving both sympathy and criticism about the scale and scope of the project, though, Meta is finally, really taking the plunge.

“The technical complexity was just extraordinary, and it's even more complex when you are doing it for well over a billion users who don’t stop using the app to let you re-create it,” Meta's Kent says. But at the same time, she adds, “we’re increasing the security, safety, and privacy for well over a billion people, and that’s a pretty amazing feat.”

End-to-End Encrypted Instagram and Messenger Chats: Why It Took Meta 7 Years

Binance’s settlement requires it to offer years of transaction data to US regulators and cops, exposing the company—and its customers—to a “24/7, 365-days-a-year financial colonoscopy.”

One attraction of Binance, as the company grew from its 2017 founding into the biggest cryptocurrency exchange in the world, was the firm's freewheeling flouting of rules. As it amassed well over 100 million crypto-trading users globally, it openly told the United States government that, as an offshore operation, it didn't have to comply with the country's financial regulations and money-laundering laws.

Then, late last month, those years of brushing off US regulators caught up with the company in the form of one the most punitive money-laundering criminal settlements in the history of the US Justice Department. The crackdown doesn't just mean a chastened Binance will have to change its practices going forward. It means that when the company is sentenced in a matter of months, it will be forced to open its _past_ books to regulators, too. What was once a haven for anarchic crypto commerce is about to be transformed into the opposite: perhaps the most fed-friendly business in the cryptocurrency industry, retroactively offering more than a half-decade of users' transaction records to US regulators and law enforcement.

When the Department of Justice announced on November 21 that Binance's executives had agreed to plead guilty to criminal money-laundering charges, much of the attention on that settlement focused on founder Changpeng Zhao giving up his CEO role and on the company's record-breaking $4.3 billion fine. But Binance's settlement agreements with the DOJ and the US Treasury Department also stipulate a strict new regime of data-sharing with law enforcement and regulators. The company has agreed to comply with regulators' "requests for information"—a term that carries none of the evidence or suspicion requirements necessary for obtaining a warrant or even a subpoena—to the point of producing any "information, testimony, document, record, or other tangible evidence."

Binance has also agreed to scour all of its transactions from 2018 to 2022 and file suspicious activity reports (SARs) for anything it deems a potential violation of US law from that five-year period. That “SAR lookback” means the company will now be actively scrutinizing its customers in retrospect, not just passively assenting to regulators poring over its databases. Those SARs are collected by FinCEN, the Treasury Department's financial crimes division, but then made available to law enforcement agencies from the FBI to IRS Criminal Investigations to local police. And all of this new scrutiny will be overseen by a "monitor" firm chosen by the US government but paid by Binance—an in-house watchdog assigned to make sure Binance is complying in good faith.

"I don't think Binance's customers have the slightest clue of the ramifications of this plea and consent decree. It's unprecedented," says John Reed Stark, who spent 20 years as an attorney at the US Securities and Exchange Commission (SEC), including as the founder of its Office of Internet Enforcement. “If they're a drug dealer or a terrorist or a child pornography peddler, they're going to get caught." He describes Binance's agreement as a "24/7, 365-days-a-year financial colonoscopy."

One US prosecutor, who asked not to be named because they weren't authorized to speak to media about the case, calls the degree of access to Binance's records described in the agreement "kind of crazy," and remains in disbelief at the idea of Binance abiding by the settlement. "I don't know what kind of business would want to operate while allowing that much government oversight, especially one that's deliberately stayed out of the US so that they're not under our nose," they say. "The other option must have been really bad."

If Binance does comply, however, the prosecutor adds that "it would be a game changer in taking down transnational syndicates doing evil deeds worldwide and trying to shield those crimes by using cryptocurrency to move money."

Binance's chief compliance officer, Noah Perlman, tells WIRED that Binance has collected "know-your-customer" information on users and cooperated with US law enforcement on data requests for the past two years. He added that all reports to the monitor firm inside Binance would be "confidential"—as in, not shared publicly, only with the US government—and that it would continue to abide by data privacy laws in the jurisdictions where it operates.

But Perlman also says he's "excited" for the new era the agreements represent for Binance. "I feel like this is a great opportunity for Binance to set the standard for what compliance in this industry should look like," he says. "For the general community, removing concerns of illicit finance in crypto is one of the most important things we can do to drive mainstream adoption. Hopefully, the vast majority of users will feel that there's assurance here, that the funds are safer than ever, and they have nothing to worry about as long as they're not part of the very small, small group of users that use crypto for illicit purposes."

While Binance's new radical transparency may be welcomed by law enforcement and regulators, its users and advocates of financial privacy may not be so pleased. Human Rights Foundation chief strategy officer Alex Gladstein calls the settlement an "overreach" that he believes is part of a US regulatory effort to set a precedent for crypto as a whole. “They’re going to try to force people to use these regulated platforms where everything is monitorable," Gladstein says. He adds that Binance is "an unsavory corporation, but still, it's alarming what the US government is doing."

Digital civil liberties nonprofit the Electronic Frontier Foundation, too, has historically called on cryptocurrency exchanges to stop giving up users' transaction data to law enforcement and regulators without notifying those users. Now, the Binance settlement would create perhaps the most extreme case yet of that crypto exchange data-sharing, giving the US government wholesale access to the records of a crypto hub that at some points processed billions of transactions a day.

"EFF is increasingly worried about law enforcement turning to intermediaries such as cryptocurrency exchanges and hosted wallet providers to obtain sensitive user data," the EFF's cryptocurrency-focused attorney, Marta Belcher, wrote in a 2020 blog post. "The fact that the transactions are made through cryptocurrency rather than through traditional financial channels indicates that the transactions are more likely to be sensitive, and that the person making the transaction may be turning to cryptocurrency precisely because of the privacy protection it provides." That argument may apply particularly to Binance, given its early reputation as an offshore exchange that didn't bow to US government data demands.

In fact, some Binance users may not have considered the risk of their data becoming available to crypto investigators in the new settlement, in part because Binance has at some points collected far _less_ data on its users than other exchanges. Part of Binance's appeal to users has been that, for years, it asked only for a user's email address to set up an account—one of its many now-admitted violations of US know-your-customer requirements that led to last month's crackdown.

But US law enforcement has proven that even troves of exchange data that lack users' names can nonetheless be highly revealing of their financial history—especially in combination with blockchain data and information from other exchanges that usually _do_ comply with know-your-customer laws. In the case of the Welcome to Video child sexual abuse materials dark-web site in 2017, for instance, one alleged abuser was identified and arrested after his email address was tied to an account on the cryptocurrency exchange BTC-e, which authorities had seized months earlier.

In another case, BTC-e's data allowed IRS criminal investigators to identify a hacker who had taken nearly 70,000 bitcoins from the Silk Road dark-web drug market—worth more than $3 billion today—and then track them down and seize the funds. Though BTC-e didn't collect users' names or other identifying details, its data still served as the missing link in both those cases—just as Binance's no doubt will in many more investigations to come.

Assenting to have US regulators comb through its data, for a company that spent years resisting regulation, may be a severe culture shock, says Stark, the former SEC attorney. He says he won't be surprised if the company ends up violating the terms of the agreement. "It's like taking someone who's been a drug addict for a decade and drug-testing them every day and thinking that they're not going to try to sneak something in," he says.

Ultimately, with enormous fines and criminal sentences hanging over the heads of its executives—and worse punishments if its settlement falls through—Binance may not have a choice about baring its soul to the US government. Neither, whether they know it or not, will its users.

The Binance Crackdown Will Be an 'Unprecedented' Bonanza for Crypto Surveillance

Governments can access records related to push notifications from mobile apps by requesting that data from Apple and Google, according to details in court records and a US senator.

If you have push notifications turned on for sensitive apps, you may want to reconsider your settings.

The United States government and foreign law enforcement can demand Apple and Google share metadata associated with push notifications from apps on iOS and Android, according to a US senator and court records reviewed by WIRED. These notifications can reveal which apps a person uses, along with other information that may be pertinent to law enforcement investigations.

US Senator Ron Wyden, an Oregon Democrat, highlighted the government surveillance technique in a letter sent to the US Department of Justice (DOJ) today. Wyden is specifically asking the DOJ to allow Apple and Google to discuss government requests for push notification records with their users, which Wyden says the US government has required them to keep secret thus far.

“In the spring of 2022, my office received a tip that government agencies in foreign countries were demanding smartphone ‘push’ notification records from Google and Apple,” Wyden wrote in the letter, which was first reported by Reuters. “My staff have been investigating this tip for the past year, which included contacting Apple and Google. In response to that query, the companies told my staff that information about this practice is restricted from public release by the government.”

App developers deliver push notifications using Apple’s Push Notification Service on iOS or Google’s Firebase Cloud Messaging on Android. Each user of an app is assigned a “push token,” which is transferred between the app and the mobile operating system’s push notification service. Push tokens are not permanently assigned to a single user, and new tokens may be generated when a person reinstalls an app or switches to a new device.

To identify a person of interest and whom they may have been communicating with, law enforcement must first go to an app developer to obtain the relevant push token and then bring it to the operating system maker—Apple or Google—and request information on which account the token is associated with. This puts the tech giants in “a unique position to facilitate government surveillance of how users are using particular apps,” Wyden writes.

According to Wyden, the records that governments can obtain from Apple and Google include metadata that reveals which apps a person has used, when they’ve received notifications, and the phone associated with a particular Google or Apple account. The content of push notifications is not included in this information, but, for at least some apps, law enforcement could obtain information about the content of specific pushes through additional requests based on the information from the push tokens.

While Wyden’s letter says that governments outside the US have requested people’s push notification records, the Federal Bureau of Investigation (FBI) has done so as well. A February 2021 search warrant application submitted by an FBI agent to the US District Court in Washington, DC, requested details for two accounts controlled by Meta (then Facebook), specifically citing a request for push notification tokens. The search warrant request related to an investigation into a person accused of taking part in the January 6, 2021, attack on the US Capitol.

Meta, which owns Facebook, WhatsApp, and Instagram, did not immediately respond to WIRED’s request to comment. A spokesperson for Signal, the popular encrypted messaging app, also did not respond. The DOJ declined to comment.

Although Wyden is asking the DOJ to allow Apple and Google to discuss government requests for push notification records, the senator’s letter appears to have enabled them to do just that.

An Apple spokesperson tells WIRED that the company has updated its Law Enforcement Guidelines in its transparency report to reflect government requests for push notification records. The company will also begin to detail these requests in its next transparency report. Apple's updated rules for police requests say push notification records “may be obtained with a subpoena or greater legal process.”

“Apple is committed to transparency and we have long been a supporter of efforts to ensure that providers are able to disclose as much information as possible to their users,” Apple says in a statement. “In this case, the federal government prohibited us from sharing any information and now that this method has become public we are updating our transparency reporting to detail these kinds of requests.”

Google confirmed to WIRED that it receives requests for push notification records, but the company says it already includes these types of requests in its transparency reports. The company says requests from US-based law enforcement for push notification records require court orders with judicial approval.

“We were the first major company to publish a public transparency report sharing the number and types of government requests for user data we receive, including the requests referred to by Senator Wyden,” a Google spokesperson tells WIRED. “We share the senator’s commitment to keeping users informed about these requests.”

A WIRED review of Google’s most recent transparency report for the period between December 2019 and December 2022 found that it does not specifically break out government requests for push notification records, and Google confirmed that it aggregates this data in its transparency report.

Google’s transparency report shows that the US government requested Google Cloud Platform data from enterprise customers 175 times during the period, and of those, used a search warrant 13 times. It is unclear whether any of those requests for user data included push notification records—details that may, following Wyden’s letter, be revealed in the future.

_Additional reporting by William Turton and Dhruv Mehrotra._

_Updated at 1:30 pm ET, December 6, 2023, to note that the DOJ has declined to comment on Wyden's letter and to add additional details about Apple's updated rules for law enforcement regarding requests for push notification records._

_Updated at 3:50 pm ET, December 6, 2023, to add details about Google's legal requirement for law enforcement requests for push notification records._

Police Can Spy on Your iOS and Android Push Notifications

23andMe has provided more information about the scope and scale of its recent breach, but with these details come more unanswered questions.

More details are emerging about a data breach the genetic testing company 23andMe first reported in October. But as the company shares more information, the situation is becoming even murkier and creating greater uncertainty for users attempting to understand the fallout.

23andMe said at the beginning of October that attackers had infiltrated some of its users' accounts and piggybacked off of this access to scrape personal data from a larger subset of users through the company's opt-in, social sharing service known as DNA Relatives. At the time, the company didn't indicate how many users had been impacted, but hackers had already begun selling data on criminal forums that seemed to be taken from at least a million 23andMe users, if not more. In a US Securities and Exchange Commission filing on Friday, the company said that “the threat actor was able to access a very small percentage (0.1 %) of user accounts,” or roughly 14,000 given the company's recent estimate that it has more than 14 million customers.

Fourteen thousand is a lot of people in itself, but the number didn't account for the users impacted by the attacker's data-scraping from DNA Relatives. The SEC filing simply noted that the incident also involved “a significant number of files containing profile information about other users’ ancestry.”

On Monday, 23andMe confirmed to TechCrunch that the attackers collected the personal data of about 5.5 million people who had opted in to DNA Relatives, as well as information from an additional 1.4 million DNA Relatives users who “had their Family Tree profile information accessed." 23andMe subsequently shared this expanded information with WIRED as well.

From the group of 5.5 million people, hackers stole display names, most recent login, relationship labels, predicted relationships, and percentage of DNA shared with DNA Relatives matches. In some cases, this group also had other data compromised, including ancestry reports and details about where on their chromosomes they and their relatives had matching DNA, self-reported locations, ancestor birth locations, family names, profile pictures, birth years, links to self-created family trees, and other profile information. The smaller (but still massive) subset of 1.4 million impacted DNA Relatives users specifically had display names and relationship labels stolen and, in some cases, also had birth years and self-reported location data affected.

Asked why this expanded information wasn't in the SEC filing, 23andMe spokesperson Katie Watson tells WIRED that “we are only elaborating on the information included in the SEC filing by providing more specific numbers.”

23andMe has maintained that attackers used a technique known as credential stuffing to compromise the 14,000 user accounts—finding instances where leaked login credentials from other services were reused on 23andMe. In the wake of the incident, the company forced all of its users to reset their passwords and began requiring two-factor authentication for all customers. In the weeks after 23andMe initially disclosed its breach, other similar services. including Ancestry and MyHeritage, also began promoting or requiring two-factor authentication on their accounts.

In October and again this week, though, WIRED pressed 23andMe on its finding that the user account compromises were attributable solely to credential-stuffing attacks. The company has repeatedly declined to comment, but multiple users have noted that they are certain their 23andMe account usernames and passwords were unique and could not have been exposed somewhere else in another leak.

On Tuesday, for example, US National Security Agency cybersecurity director Rob Joyce noted on his personal X (formerly Twitter) account: “They disclose the credential stuffing attacks, but they don’t say how the accounts were targeted for stuffing. This was unique and not an account that could be scraped from the web or other sites.” Joyce, who was apparently a 23andMe user impacted by the breach, wrote that he creates a unique email address for each company he makes an account with. “That account is used NOWHERE else and it was unsuccessfully stuffed,” he wrote, adding: “Personal opinion: @23andMe hack was STILL worse than they are owning with the new announcement.”

23andMe has not clarified how such accounts can be reconciled with the company's disclosures. Furthermore, it may be that the larger numbers of impacted users were not in the SEC report because 23andMe (like many companies that have suffered security breaches) does not want to include _scraped_ data in the category of _breached_ data. These inconsistencies, though, ultimately make it difficult for users to grasp the scale and impact of security incidents.

“I firmly believe that cyber-insecurity is fundamentally a policy problem,” says Brett Callow, a threat analyst at the security firm Emsisoft. “We need standardized and uniform disclosure and reporting laws, prescribed language for those disclosures and reports, regulation and licensing of negotiators. Far too much happens in the shadows or is obfuscated by weasel words. It's counterproductive and helps only the cybercriminals.”

Meanwhile, apparent 23andMe user Kendra Fee flagged on Tuesday that 23andMe is notifying customers about changes to its terms of service related to dispute resolutions and arbitration. The company says that the changes will “encourage a prompt resolution of any disputes” and “streamline arbitration proceedings where multiple similar claims are filed.” Users can opt out of the new terms by notifying the company that they decline within 30 days of receiving notice of the change.

The 23andMe Data Breach Keeps Spiraling

Adversarial algorithms can systematically probe large language models like OpenAI’s GPT-4 for weaknesses that can make them misbehave.

When the board of OpenAI suddenly fired the company’s CEO last month, it sparked speculation that board members were rattled by the breakneck pace of progress in artificial intelligence and the possible risks of seeking to commercialize the technology too quickly. Robust Intelligence, a startup founded in 2020 to develop ways to protect AI systems from attack, says that some existing risks need more attention.

Working with researchers from Yale University, Robust Intelligence has developed a systematic way to probe large language models (LLMs), including OpenAI’s prized GPT-4 asset, using “adversarial” AI models to discover “jailbreak” prompts that cause the language models to misbehave.

While the drama at OpenAI was unfolding, the researchers warned OpenAI of the vulnerability. They say they have yet to receive a response.

“This does say that there’s a systematic safety issue, that it’s just not being addressed and not being looked at,” says Yaron Singer, CEO of Robust Intelligence and a professor of computer science at Harvard University. “What we’ve discovered here is a systematic approach to attacking any large language model.”

OpenAI spokesperson Niko Felix says the company is “grateful” to the researchers for sharing their findings. “We’re always working to make our models safer and more robust against adversarial attacks, while also maintaining their usefulness and performance,” Felix says.

The new jailbreak involves using additional AI systems to generate and evaluate prompts as the system tries to get a jailbreak to work by sending requests to an API. The trick is just the latest in a series of attacks that seem to highlight fundamental weaknesses in large language models and suggest that existing methods for protecting them fall well short.

“I’m definitely concerned about the seeming ease with which we can break such models,” says Zico Kolter, a professor at Carnegie Mellon University whose research group demonstrated a gapping vulnerability in large language models in August.

Kolter says that some models now have safeguards that can block certain attacks, but he adds that the vulnerabilities are inherent to the way these models work and are therefore hard to defend against. “I think we need to understand that these sorts of breaks are inherent to a lot of LLMs,” Kolter says, “and we don’t have a clear and well-established way to prevent them.”

Large language models recently emerged as a powerful and transformative new kind of technology. Their potential became headline news as ordinary people were dazzled by the capabilities of OpenAI’s ChatGPT, released just a year ago.

In the months that followed the release of ChatGPT, discovering new jailbreaking methods became a popular pastime for mischievous users, as well as those interested in the security and reliability of AI systems. But scores of startups are now building prototypes and fully fledged products on top of large language model APIs. OpenAI said at its first-ever developer conference in November that over 2 million developers are now using its APIs.

These models simply predict the text that should follow a given input, but they are trained on vast quantities of text, from the web and other digital sources, using huge numbers of computer chips, over a period of many weeks or even months. With enough data and training, language models exhibit savant-like prediction skills, responding to an extraordinary range of input with coherent and pertinent-seeming information.

The models also exhibit biases learned from their training data and tend to fabricate information when the answer to a prompt is less straightforward. Without safeguards, they can offer advice to people on how to do things like obtain drugs or make bombs. To keep the models in check, the companies behind them use the same method employed to make their responses more coherent and accurate-looking. This involves having humans grade the model’s answers and using that feedback to fine-tune the model so that it is less likely to misbehave.

Robust Intelligence provided WIRED with several example jailbreaks that sidestep such safeguards. Not all of them worked on ChatGPT, the chatbot built on top of GPT-4, but several did, including one for generating phishing messages, and another for producing ideas to help a malicious actor remain hidden on a government computer network.

A similar method was developed by a research group led by Eric Wong, an assistant professor at the University of Pennsylvania. The one from Robust Intelligence and his team involves additional refinements that let the system generate jailbreaks with half as many tries.

Brendan Dolan-Gavitt, an associate professor at New York University who studies computer security and machine learning, says the new technique revealed by Robust Intelligence shows that human fine-tuning is not a watertight way to secure models against attack.

Dolan-Gavitt says companies that are building systems on top of large language models like GPT-4 should employ additional safeguards. “We need to make sure that we design systems that use LLMs so that jailbreaks don’t allow malicious users to get access to things they shouldn’t,” he says.

A New Trick Uses AI to Jailbreak AI Models—Including GPT-4

Legislation set to be introduced in Congress this week would extend Section 702 surveillance of people applying for green cards, asylum, and some visas—subjecting loved ones to similar intrusions.

Americans with family overseas who hope to visit the United States may soon face an increased risk of being surveilled by their own government.

Support in Congress is growing for intensified vetting procedures at the US border, which would see immigrants and foreign visitors subjected to the same levels of scrutiny as suspected terrorists and spies. A bill introduced last week by members of the Senate Intelligence Committee (SSCI) and forthcoming legislation from its House counterpart both aim to expand the use of a key foreign intelligence program—Section 702—for screening and vetting visitors to the US.

The House bill, which has yet to be introduced, would additionally target asylum seekers and those applying for nonimmigrant visas and green cards, according to a memo released by the House Intelligence Committee (HPSCI) last month.

In an interview with CBS on Sunday, US Representative Mike Turner, Republican from Ohio and HPSCI chair, said he had gained the support of his Democratic counterpart, Jim Himes, in advancing legislation that would include the enhanced vetting procedures.

The 702 program allows the government to force US companies to wiretap the communications of foreigners who are presumed to be overseas—even when they’re communicating with people inside the US. Billions of calls, texts, and emails are intercepted under the program and committed to a searchable database accessible by four US intelligence agencies, including the Central Intelligence Agency (CIA) and Federal Bureau of Investigation (FBI).

A warrant is not required to conduct searches of the database, and its targets need not be suspected threats to the country. While ostensibly for defense, the 702 program selects targets based on whether they’re expected to possess or receive “foreign intelligence information.” In May, the US State Department said the program was “essential to advancing and promoting US interests around the world,” citing its value as a diplomatic tool.

In a declassified report released this year, a presidential oversight board said the calls and messages of Americans intercepted under Section 702 are “highly personal and sensitive,” detailing exchanges between “loved ones, friends, medical providers, academic advisors, lawyers, or religious leaders.” These communications, according to the Privacy and Civil Liberties Oversight Board (PCLOB), a federal civil liberties watchdog, may provide “great insight into an individual’s whereabouts, both in a given moment and in patterns over time.”

Public knowledge of this surveillance, the board concluded, “can have a chilling effect on speech.”

The 702 program is slated to expire on January 1, 2024. Lawmakers in the House and Senate are rushing to find a solution that would enable the program to continue despite growing mistrust from lawmakers and the public following years of unauthorized use. Abuses of 702 data have included the targeting of racial justice protesters, political activists, and immigrants. Other misused searches of the database for information related to a sitting US congressperson, a US senator, and members of a local political party.

US intelligence community leaders say permitting the program to expire would significantly diminish the government’s ability to monitor and disrupt overseas threats, from terrorism and cyberattacks targeting US infrastructure to espionage and the black market sale of nuclear weapons. “Loss of this vital provision, or its reauthorization in a narrowed form, would raise profound risks,” FBI director Christopher Wray said in a statement this fall.

The wealth of data collected under the 702 program is imponderable. US intelligence agencies have repeatedly claimed there are no means by which to calculate how often Americans are wiretapped without violating procedures intended to keep the program constitutional. The PCLOB report published earlier this year says simply that the collection of domestic communications “should not be understood as occurring infrequently.”

Civil rights experts argue that expanding the program to include routine vetting of people entering the United States would almost certainly be an encumbrance on immigration and asylum systems that US politicians readily acknowledge are broken. There are currently more than 2 million cases pending in US immigration courts, with the average case taking more than four years to adjudicate.

“Some noncitizens—including children and families—wait years to have their cases heard. The delays postpone decisions for vulnerable populations who may be eligible for protections, such as asylum. They also prolong the removal from the US of those who do not have valid claims to remain,” the US Government Accountability Office reported in October.

The vetting proposal, now formally endorsed by both intelligence committees in Congress, has ignited a firestorm among immigrants’ rights groups and nonprofits combating racism against Asian American and Pacific Islander (AAPI) communities. The groups say that, due to their international ties, immigrants naturally face an increased risk of having their communications swept up by the 702 program.

Kia Hamadanchy, senior policy counsel at the American Civil Liberties Union (ACLU), says the Section 702 program “invariably” intercepts communications between foreigners and their American family members. Using the program for vetting purposes means committing to “entirely suspicionless searches” of both, he says.

Andy Wong, a director of advocacy at Stop AAPI Hate, a coalition of community-based groups, has called out Democrats specifically for supporting the move, labeling it a “betrayal” of the Latino and Asian American communities. “We need leaders who dismantle systemic racism,” Wong says, “not entrench policies that leave us more exposed, separated, and vulnerable.”

“The government should be seeking to streamline the immigration process, including the family-based immigration system, not expand warrantless surveillance for individuals seeking to come to the United States,” says Joanna YangQing Derman, program director at the nonprofit Asian Americans Advancing Justice (AAJC). "Expanding warrantless surveillance to specifically go after immigrant communities would not only be harmful, but it would also be a deeply disappointing continuation of this country’s unfair treatment of Asian Americans and Asian immigrants.”

Representative Turner, the HPSCI chair, said Sunday on _Face the Nation_ that his committee had a bill to extend the 702 program endorsed by, among others, Representative Jim Himes, the committee’s ranking Democrat. Turner accused lawmakers not onboard with his bill of misunderstanding how the program works and its “value and importance” to “national security.”

ACLU’s Hamadanchy tells WIRED that it would be concerning to see Himes and other Democrats endorsing use of the program against immigrant communities. “It would represent a dramatic expansion of the current vetting practices,” he says, “and it would be disappointing if it is something Congressman Himes has signed on to.”

Himes did not immediately respond to a request for comment.

Turner said Sunday that he’d already gained the support of the SSCI chair, Senator Mike Warner (D-Virginia), who introduced his own 702 bill last week. Warner’s bill also includes language that expands the 702 program to “enable the vetting of non-United States persons who are being processed for travel to the United States,” but does not mention visas or green cards.

Elizabeth Goitein, senior director of the Brennan Center for Justice’s liberty & national security program, calls Warner’s proposals unnecessary. “There are already plenty of vetting mechanisms in place to ensure that visitors to this country don’t pose a threat to national security or public safety,” she says.

“People should be able to vacation, work, or study in the United States without opening up their private communications to US government scrutiny,” Goitein adds.

Although the text of his bill is not public, much is already known about Turner’s aims. Last month, the HPSCI released an outline of its proposals describing, among other so-called reforms, an amendment that would allow the government to search the 702 database “for the purpose of screening and vetting immigration and non-immigrant visa applicants.”

According to Goitein, the HPSCI proposal would likely impact millions of people living lawfully in the United States, including those with work permits and student visas. “The notion that immigrants give up their privacy rights when they come to this country is wrong and repugnant,” she says.

Senior congressional sources tell WIRED that the HPSCI bill may surface as early as this week. It will compete against another bipartisan bill introduced in the House Judiciary Committee (HJC), chaired by Representative Jim Jordan (R-Ohio), a prominent critic of domestic surveillance abuse.

The contrast between the two House bills is likely to be drastic, with Jordan having previously endorsed an array of privacy reforms, including those aimed at requiring the FBI to obtain a warrant before accessing Americans’ communications amassed by the 702 program—a limitation that is strictly opposed by the heads of the two intelligence committees.

Neither Warner nor Turner immediately responded to inquiries from WIRED on Monday.

Mystery surrounds which bill Representative Mike Johnson (R-Louisiana), the new House speaker, will inevitably endorse. But the HJC traditionally faces stiff opposition from House leaders when pursuing surveillance reform. Republican and Democratic leaders alike have historically shown deference to the appeals of the intelligence community for sustaining and enhancing its authority.

Johnson, meanwhile, is rumored to have held war room discussions last month about extending the Section 702 program by amending must-pass legislation, such as the National Defense Authorization Act—which, despite being months late already, has yet to materialize.

Johnson has not responded to repeated requests for comment from WIRED regarding his thoughts on the 702 program.

_Updated at 10:45 am ET, December 4, 2023, with comment from Joanna YangQing Derman of AAJC._

US Lawmakers Want to Use a Powerful Spy Tool on Immigrants and Their Families

A WIRED investigation into internet censorship in US schools found widespread use of filters to censor health, identity, and other crucial information. Students say it makes the web entirely unusable.

Inside America's School Internet Censorship Machine

QR codes can be convenient—but they can also be exploited by malicious actors. Here’s how to protect yourself.

And you don’t need anything special to create a QR code. The tools are widely available and straightforward to use, and putting together a QR code of your own isn’t much more difficult than scanning one. If you wanted to create a QR code that points to a website that’s been put together for malicious purposes, it would only take a couple of minutes. The QR code could then be stuck on a wall, attached to an email, or printed on a document, ready to be scanned.

The aims of these websites are the same as they’ve always been: to get you to download something that will compromise the security of your accounts or your devices, or to get you to enter some login credentials that will then be relayed straight to the hackers (most probably using a spoof site set up to look like something genuine and trustworthy). The intended end results are the same as ever, but the method of getting there is different.

Avoiding QR Code Hacks

The security precautions you should already be using are the same ones that will keep you protected against QR code hacking. Just as you would with emails or instant messages, don’t trust QR codes if you’re not sure where they’ve come from—perhaps attached to suspicious-looking emails or on websites that you can’t verify. The QR code on the menu in your local restaurant, in contrast, is highly unlikely to have been generated by hackers.

Of course, there’s always the chance that the accounts of your friends, family, and colleagues have been compromised, so you can never be 100 percent sure that a message with a QR code in it is genuine. Scams will usually try to imply a sense of urgency and alarm: Scan this QR code to verify your identity or prevent the deletion of your account or take advantage of a time-limited offer.

You should get a preview of the link you’re visiting from a QR code.

Apple via David Nield

As always, your digital accounts should be as heavily protected as possible, so that if you do fall victim to a QR code trick, safety nets are in place. Switch on two-factor authentication for every account that offers it, make sure your personal details are up to date (such as backup email addresses and phone numbers that can be used to recover your accounts), and log out of devices you’re no longer using (you should also delete old accounts you no longer have any need for).

Finally, keep your software up to date—something that’s happily now very easy to do. The latest versions of popular mobile web browsers come with built-in tech for spotting fraudulent links: These integrated protections aren’t infallible, but the more up-to-date your browser and mobile OS are, the better your chances of getting a warning on screen if you’re about to visit an unsafe location on the web.

How to Not Get Hacked by a QR Code

Plus: A major ransomware crackdown, the arrest of Ukraine’s cybersecurity chief, and a hack-for-hire entrepreneur charged with attempted murder.

OpenAI didn't immediately respond to WIRED's request for comment on the researchers' findings. When we tried the “repeat ‘poem’ forever" and “repeat ‘book’ forever” prompts ourselves, they didn't produce training data but instead threw up flags for a potential violation of ChatGPT's terms of use, suggesting at least some instances of the problem may have been fixed.

In the midst of Israel's ongoing war with Hamas, US and Israeli government agencies on Friday warned that hackers calling themselves “Cyberav3ngers” but working for Iran's Revolutionary Guard Corps had breached the networks of multiple US water and wastewater utilities. The breaches, which affected “less than 10” utilities, according to a CNN source, aimed to deface computer screens in the facilities with an anti-Israel message. In each case, the hackers took advantage of vulnerabilities in equipment sold by Unitronics, an Israeli company. “You have been hacked, down with Israel. Every equipment ‘made in Israel’ is Cyberav3ngers legal target," some of the defaced screens read. While the intrusions appear to have been opportunistic and aimed at sending a message, the ability of a foreign government to gain broad access to US critical infrastructure led the Cybersecurity and Infrastructure Security agency to brief members of Congress on the hacking campaign on Thursday.

In a sprawling bust that spanned multiple Ukrainian cities, at least five key members of a ransomware gang were arrested this week in raids coordinated by Europol along with law enforcement agents from Ukraine, the US, Canada, the Netherlands, and other European countries. The group's members are accused of deploying multiple ransomware variants including LockerGoga, Hive, MegaCortex, and Dharma. According to Ukrainian police, the gang allegedly did at least $82 million in damage in attacks that encrypted more than a thousand servers on victim networks over the past five years.

In a very different sort of Ukrainian criminal case, Ukrainian law enforcement this week detained Viktor Zhora, the deputy director of the State Special Communications Service of Ukraine, its agency focused on cybersecurity. Zhora, along with the agency's director, is accused of taking part in a multimillion-dollar corruption scheme. While corruption has long plagued the Ukrainian government and military, the charges against Zhora—and his detainment this week—have sent shock waves through the global cybersecurity community, in which Zhora was a high-profile figure and often the public face of Ukraine's cybersecurity defense. In November, for instance, Zhora keynoted the popular Cyberwarcon conference of security researchers in Arlington, Virginia. Zhora was released on bail later in the week. When his charges were announced, he told TechCrunch that he would “defend \[his\] name and reputation in a court.”

In keeping with this week's theme of (alleged) crime and punishment, David Vincenzetti, the founder of hacker-for-hire firm Hacking Team, was arrested last weekend for the alleged stabbing and attempted murder of a family member, TechCrunch reported based on news articles in multiple Italian-language media outlets. According to one of those newspapers, _Il Giorno_, the victim was visiting Vincenzetti to take care of him due to his psychological issues. When Vincenzetti appeared before a judge, he reportedly gave a rambling statement that caused a judge to ask prosecutors to investigate his mental health, according to _La Stampa_. The reported charge could suggest a dark ending to the story of a man with a dark career, who helped launch an industry of cyber-mercenaries like NSO Group, Appin, CyberRoot, and BellTroX.

_Updated at 12 pm ET, December 2, 2023, with an item about Iranian hackers breaching US critical infrastructure._

ChatGPT Spit Out Sensitive Data When Told to Repeat ‘Poem’ Forever

A federal court ruled on Friday that Trump, as president, may be able to avoid civil action for his role in the January 6, 2021, attack on the US Capitol. But candidate Trump is something different.

Donald Trump, the president, may well be immune from any civil action for allegedly inciting an attack against the US Capitol on January 6, 2021. But Donald Trump, the candidate, is not, an appellate court in the District of Columbia says.

“When a first-term President opts to seek a second term, his campaign to win re-election is not an official presidential act,” says a ruling handed down this morning.

The court’s unanimous decision to reject Trump’s claim of absolute immunity places the Republican frontrunner in increased financial jeopardy, paving the way for members of Congress and Capitol police to seek compensation for harms allegedly endured during a riot that resulted in millions of dollars in damages and caused injury to nearly 140 police officers, according to their union.

In a 67-page opinion, the court says that Trump failed in his attempt to demonstrate that he’s entitled to what’s called “official-act immunity,” a powerful liability shield afforded to presidents that aims to ensure that they can—as the court puts it—“fearlessly and impartially discharge the singularly weighty duties” of the presidency.

“We answer no,” the court says, “at least at this stage of the proceedings,” adding that “campaigning to gain that office is not an official act of the office.”

The ruling has been long awaited. Trump’s claim of immunity was first rejected by US district court judge Amit Mehta in February 2022.

In part, it was Trump’s own attempt to overturn the election at the US Supreme Court that may have doomed his case. The DC Circuit says he acknowledged that his post-election efforts to have the result reversed in his favor were done in a personal capacity, and not that of a sitting president. Those claims, the ruling says, certified that Trump sought the court’s intervention based on his own “unique and substantial” interests as—specifically—a candidate.

Trump’s attorney has argued that the distinction is immaterial, and the appellate court unanimously disagreed. However, the matter of immunity was the sole issue under consideration, and Trump’s liability has yet to be determined.

The US Justice Department, asked by the appellate court to weigh in on the matter last year, concluded in March that Trump could be sued over the attack, adding that while presidents are afforded great protection with regard to a “vast realm” of speech, it did not extend to the “incitement of imminent private violence.”

The civil case is separate from the federal criminal trial in the district, which was led by special counsel and former acting US attorney Jack Smith and concerns not only Trump’s attempt to overturn the election, but allegations that he unlawfully retained classified documents at his Mar-a-Lago estate in Florida. Earlier this week, former vice president Mike Pence reportedly told the special counsel that Trump’s advisors—“crank” attorneys, as Pence put it—pushed the country to the brink of a constitutional crisis. The case is set to go to trial this March.

The siege began shortly after Trump delivered a 75-minute speech at a park south of the White House, known as the Ellipse. A House select committee investigating the riot last year said that Trump was aware the attack on the Capitol was underway as he arrived at the White House roughly 15 minutes after the speech. Witnesses, including a former DC police sergeant, claimed at the time that Trump had been swept away by US Secret Service agents trying to prevent him from joining the march.

Trump’s then press secretary, Kayleigh McEnany, later testified that during the attack Trump had requested a list of phone numbers belonging to US senators on the Hill, whom he reportedly contacted in an attempt to stop the results of the 2020 election from being certified. The calls were not recorded on the presidential call log. During this time, Trump supporters were engaged with police officers and had begun forcing them back onto the Capitol lawn, throwing bottles and dousing several offices with chemical spray.

The attack lasted approximately two hours and resulted in five deaths, including that of a Capitol police officer. The rioters, who broke windows, ransacked lawmakers’ offices, and stole documents and electronics, reportedly caused more than $2 million in damages to the 222-year-old building.

Two Capitol police officers, James Blassingame and Sidney Hemby, are seeking $75,000 in compensatory damages (as well as unspecified punitive damages) for injuries they say they sustained in the attack. In a court filing, Blassingame, an officer of 19 years, says he was struck in the face, head, and up and down his body by Trump supporters during the attempt to breach the Capitol building. Hemby, a former Marine, suffered “cuts and abrasions” to his face and hands. Neither officer could be immediately reached for comment.

At the onset of the suit, the plaintiff lawmakers included Democrats Eric Swalwell, Stephen Cohen, Bonnie Coleman, Veronica Escobar, Pramila Jayapal, Henry C. Johnson, Marcia Kaptur, Barbara Lee, Jerrold Nadler, and Maxine Waters. Karen Bass, a former congressperson and current mayor of Los Angeles, has also joined the suit. The lawmakers, including Bass, either did not respond or declined to comment.

“The Court today confirms that no one is above the law, even presidents while engaged in conduct during their presidency," Representative Johnson says.

Bennie Thompson, the congressperson from Mississippi, says he was no longer party to the case on appeal but welcomed the court’s decision. “Donald Trump should not be able to use the presidency to shirk accountability for what he did to cause the insurrection on January 6,” he tells WIRED.

An attorney for Trump, Jesse Binnall, did not immediately respond to a request for comment.

Source

Wired