Artificial intelligence capabilities are coming to a desktop near you — with Microsoft 365 Copilot, Google Gemini with Project Jarvis, and Apple Intelligence all arriving (or having arrived). But what are the risks?

Source: 'Who is Danny' via Shutterstock

Artificial intelligence has come to the desktop.

Microsoft 365 Copilot, which debuted last year, is now widely available. Apple Intelligence just reached general beta availability for users of late-model Macs, iPhones, and iPads. And Google Gemini will reportedly soon be able to take actions through the Chrome browser under an in-development agent feature dubbed Project Jarvis.

The integration of large language models (LLMs) that sift through business information and provide automated scripting of actions — so-called "agentic" capabilities — holds massive promise for knowledge workers but also significant concerns for business leaders and chief information security officers (CISOs). Companies already suffer from significant issues with the oversharing of information and a failure to limit access permissions — 40% of firms delayed their rollout of Microsoft 365 Copilot by three months or more because of such security worries, according to a Gartner survey.

The broad range of capabilities offered by desktop AI systems, combined with the lack of rigorous information security at many businesses, poses a significant risk, says Jim Alkove, CEO of Oleria, an identity and access management platform for cloud services.

"It's the combinatorics here that actually should make everyone concerned," he says. "These categorical risks exist in the larger \[native language\] model-based technology, and when you combine them with the sort of runtime security risks that we've been dealing with — and information access and auditability risks — it ends up having a multiplicative effect on risk."

Related:Citizen Development Moves Too Fast for Its Own Good

Desktop AI will likely take off in 2025. Companies are already looking to rapidly adopt Microsoft 365 Copilot and other desktop AI technologies, but only 16% have pushed past initial pilot projects to roll out the technology to all workers, according to Gartner's "The State of Microsoft 365 Copilot: Survey Results." The overwhelming majority (60%) are still evaluating the technology in a pilot project, while a fifth of businesses haven't even reached that far and are still in the planning stage.

Most workers are looking forward to having a desktop AI system to assist them with daily tasks. Some 90% of respondents believe their users would fight to retain access to their AI assistant, and 89% agree that the technology has improved productivity, according to Gartner.

**Bringing Security to the AI Assistant**

Unfortunately, the technologies are black boxes in terms of their architecture and protections, and that means they lack trust. With a human personal assistant, companies can do background checks, limit their access to certain technologies, and audit their work — measures that have no analogous control with desktop AI systems at present, says Oleria's Alkove.

Related:Cleo MFT Zero-Day Exploits Are About to Escalate, Analysts Warn

AI assistants — whether they are on the desktop, on a mobile device, or in the cloud — will have far more access to information than they need, he says.

"If you think about how ill-equipped modern technology is to deal with the fact that my assistant should be able to do a certain set of electronic tasks on my behalf, but nothing else," Alkove says. "You can grant your assistant access to email and your calendar, but you cannot restrict your assistant from seeing certain emails and certain calendar events. They can see everything."

This ability to delegate tasks needs to become part of the security fabric of AI assistants, he says.

**Cyber-Risk: Social Engineering Both Users & AI**

Without such security design and controls, attacks will likely follow.

Earlier this year, a prompt injection attack scenario highlighted the risks to businesses. Security researcher Johann Rehberger found that an indirect prompt injection attack through email, a Word document, or a website could trick Microsoft 365 Copilot into taking on the role of a scammer, extracting personal information, and leaking it to an attacker. Rehberger initially notified Microsoft of the issue in January and provided the company with information throughout the year. It's unknown whether Microsoft has a comprehensive fix for the issue.

Related:Generative AI Security Tools Go Open Source

The ability to access the capabilities of an operating system or device will make desktop AI assistants another target for fraudsters who have been trying to get a user to take actions. Instead, they will now focus on getting an LLM to take actions, says Ben Kilger, CEO of Zenity, an AI agent security firm.

"An LLM gives them the ability to do things on your behalf without any specific consent or control," he says. "So many of these prompt injection attacks are trying to social engineer the system — trying to go around other controls that you have in your network without having to socially engineer a human."

**Visibility Into AI's Black Box**

Most companies lack visibility into and control of the security of AI technology in general. To adequately vet the technology, companies need to be able to examine what the AI system is doing, how employees are interacting with the technology, and what actions are being delegated to the AI, Kilger says.

"These are all things that the organization needs to control, not the agentic platform," he says. "You need to break it down and to actually look deeper into how those platforms actually being utilized, and how do people build and interact with those platforms."

The first step to evaluating the risk of Microsoft 365 Copilot, Google's purported Project Jarvis, Apple Intelligence, and other technologies is to gain this visibility and have the controls in place to limit an AI assistant's access on a granular level, says Oleria's Alkove.

Rather than a big bucket of data that a desktop AI system can always access, companies need to be able to control access by the eventual recipient of the data, their role, and the sensitivity of the information, he says.

"How do you grant access to portions of your information and portions of the actions that you would normally take as an individual, to that agent, and also only for a period of time?" Alkove asks. "You might only want the agent to take an action once, or you may only want them to do it for 24 hours, and so making sure that you have those kind of controls today is critical."

Microsoft, for its part, acknowledges the data-governance challenges, but argues that they are not new, just made more apparent due to AI’s arrival.

"AI is simply the latest call to action for enterprises to take proactive management of controls their unique, respective policies, industry compliance regulations, and risk tolerance should inform – such as determining which employee identities should have access to different types of files, workspaces, and other resources," a company spokesperson said in a statement.

The company pointed to its Microsoft Purview portal as a way that organizations can continuously manage identities, permission, and other controls. Using the portal, IT admins can help secure data for AI apps and proactively monitor AI use though a single management location, the company said. Google declined to comment about its forthcoming AI agent.

**About the Author**

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline Journalism (Online) in 2003 for coverage of the Blaster worm. Crunches numbers on various trends using Python and R. Recent reports include analyses of the shortage in cybersecurity workers and annual vulnerability trends.

Does Desktop AI Come With a Side of Risk?

While low-code/no-code tools can speed up application development, sometimes it's worth taking a slower approach for a safer product.

Source: Westend61 GmbH via Alamy Stock Photo

COMMENTARY

Say you're working on an important financial report for your company, with a strict deadline. You need to share it with external financial advisers, but security restrictions are preventing you from adding them directly. You grab the report, open your personal email, upload the report — and just before you hit send, you realize this is probably not a wise decision. You delete your draft.

I'm sure you can think of many other examples where you got into a similar situation in the heat of the moment; hopefully you bumped into a security guardrail that made you think twice. Sometimes some friction is needed to slow us down and get us to rethink.

**Low-Code/No-Code Makes Things Too Easy**

Business units can't wait around for IT and development units to get to their items on an ever-growing backlog. Low-code/no-code platforms have really made a difference in large enterprises in the past few years, and generative artificial intelligence has turbocharged this trend. Nontechnical users are empowered to create applications by describing them to a chatbot that does everything from generate the database to the user interface. They are also creating automations to streamline business processes, either by chatting with a chatbot or using drag-and-drop. This is all happening at the heart of the enterprise and is wonderful for productivity.

Security controls provided by low-code/no-code platforms typically focus on the point that an application inherits its user's permissions. That means that, theoretically, a user could manually do everything the application or automation does on their behalf. So what's the problem?

People are not robots. We don't move the same amount of data, we are not consistent when we do something again and again, and — most importantly — we have common sense. A human can understand that sharing a financial report externally is not a good idea, while sharing nonsensitive files might be all right. But if an automation is set up to sync data between you and your external vendors, with the intent of sharing nonsensitive files, no one is going to be there to flag it or second-guess when sensitive files are also transferred unintentionally.

You could say that the person who created the automation should have thought about it, and you're right. But that requires them to stop and think. If you can create an automation by talking to a chatbot, then you quickly get into a situation where you're creating automations left and right without fully thinking through the consequences. Low-code/no-code platforms are lowering the bar to be creative within the enterprise, which is wonderful but also dangerous.

**Tapping the Brakes, Not Taking the Keys**

Some friction could make all the difference in the world, if carefully used. Allowing citizen developers to create automations and applications is great, but perhaps if there are external data sources or vendors, somebody needs to take a second look. Low-code/no-code doesn't really follow the software development life cycle process, but notifying the security team or center of excellence for selective reviews where it matters is feasible. We must be careful not to add too much friction, however, or we'll lose the productivity benefits that citizen development brings — or people are going to find ways around our controls.

To hit the right balance, we should let citizen developers build freely but intervene where needed. We should set up automated guardrails that catch when developers go outside of our approved risk zone and intervene — even if just by nudging them to stop and rethink.

**About the Author**

CTO & Co-Founder, Zenity

Michael Bargury is an industry expert in cybersecurity focused on cloud security, SaaS security, and AppSec. Michael is the CTO and co-founder of Zenity.io, a startup that enables security governance for low-code/no-code enterprise applications without disrupting business. Prior to Zenity, Michael was a senior architect at Microsoft Cloud Security CTO Office, where he founded and headed security product efforts for IoT, APIs, IaC, Dynamics, and confidential computing. Michael holds 15 patents in the field of cybersecurity and a BSc in Mathematics and Computer Science from Tel Aviv University. Michael is leading the OWASP community effort on low-code/no-code security.

Citizen Development Moves Too Fast for Its Own Good

A 20-year-old Tucson man was arrested for horrific CSAM and cyberstalking linked to the dangerous online extremist group 764.

****SUMMARY****

*   **Arrest and Charges**: Baron Martin, 20, arrested for producing child sexual abuse material and cyberstalking, linked to extremist networks 764 and CVLT.

*   **Disturbing Crimes**: Accused of coercing minors into self-mutilation, creating abuse material, and plotting violence against victims and their families.

*   **764 Network**: The extremist group uses child exploitation to normalize violence and promote societal collapse, flagged as a major threat by the DoJ.

*   **Potential Penalties**: Martin faces up to 40 years in prison for his crimes, alongside fines and possible lifetime supervised release.

*   **Law Enforcement Response**: FBI and DoJ emphasize vigilance and programs like Project Safe Childhood to combat online child exploitation.

A 20-year-old Tucson man, Baron Martin, has been arrested for producing child sexual abuse material (CSAM) and cyberstalking offences linked to his involvement in online violent extremist networks known as 764 and CVLT, reveals an unsealed complaint by the Department of Justice (DoJ).

The complaint details a disturbing pattern of behaviour by 764 members. They allegedly target vulnerable youth online, force them into self-harm, and spread the abuse to desensitize victims and normalize violence. Martin, who allegedly used the online alias “Convict,” is accused of:

*   **Creating a guide on how to identify, groom, and extort children:** Martin is accused of creating child sex abuse content on Discord in September 2022 by having two minors self-mutilate for him, with one child having his name cut into their chest, stomach, and thighs, and directing them to cut swastikas and satanic symbols.

*   **Forcing two minors (aged 13 and 16) to self-mutilate on Discord:** Martin allegedly directed a 16-year-old victim on Discord to self-injury, drawing blood, and pouring alcohol over the wounds. Martin sent three videos of the abuse to another user

*   **Cyberstalking a 13-year-old by threatening her grandmother and soliciting her murder**: Martin is accused of cyberstalking a 13-year-old girl in September 2022, threatening her grandmother, and soliciting her murder. He agreed to pay $3,000 for the kidnapping and murder, and in a separate chat, posted the girl’s and her grandmother’s phone numbers for harassment.

If convicted, Martin faces up to 30 years in prison for producing CSAM and up to 10 years for cyberstalking, along with significant fines and potential lifetime supervised release.

****764 Network Described as “Dangerous” by DoJ****

According to the DoJ’s press release, it considers the 764 network a huge threat. “764 is a dangerous network of violent extremists who systematically target children and use child sexual abuse material to further their agenda of societal collapse,” said Assistant Attorney General for National Security Matthew G. Olsen.

The FBI is leading the investigation. The case is being prosecuted by the U.S. Attorney’s Office for the District of Arizona and the National Security Division’s Counterterrorism Section. 

It is worth noting that Martin is not the only alleged member of the notorious 764 network to face legal action. In November 2024, Richard Anthony Reyna Densmore, 47, of Kaleva, Michigan, was **sentenced** to 30 years in prison on charges related to child sexual abuse material (CSAM).

The arrest of Baron Martin shows the problematic reality of online extremism and grooming and its devastating impact on vulnerable youth. This issue extends beyond the U.S., with the U.K. facing similar challenges, particularly on platforms like Snapchat. According to a **report** by a U.K. charity, half of all online child grooming cases now occur on Snapchat.

The 764 network’s calculated targeting of children and their use of child sexual abuse material to further their extremist ideology emphasises the urgent need for increased vigilance and law enforcement efforts to combat such threats. 

US Attorney Gary Restaino for the District of Arizona stressed the importance of **Project Safe Childhood** (PCS), a US-based initiative targeting child sexual exploitation, and encouraged parents and children to seek help if they encounter online predators.

1.  **Man sought dark web hitman to murder child abuse victim**
2.  **Op protected childhood: 113 online child predators arrested**
3.  **How Facebook Helped FBI Capture a Notorious Child Abuser**
4.  **9 risky apps that you must monitor on your kids’ smartphone**
5.  **Microsoft’s tool detects, reports pedophiles from online chats**

FBI Targets 764 Network: Man Faces 30 Years for Cyberstalking, CSAM

A thwarted attack demonstrates that threat actors using yet another delivery method for the malware, which already has been spread using phishing emails, malvertising, hijacking of instant messages, and SEO poisoning.

Source: Brian Jackson via Alamy Stock Photo

The DarkGate remote access Trojan (RAT) has a new attack vector: A threat actor targeted a Microsoft Teams user via a voice call to gain access to their device.

The attack adds to the other methods for spreading the RAT, which previously has been propagated using phishing emails, malvertising, hijacking of Skype and Teams messages, and search engine optimization (SEO) poisoning, researchers said.

Researchers at Trend Micro discovered the voice phishing, or vishing, attack, in which an attacker initially tried to install a Microsoft remote support application to gain access to the user's device, they revealed in a recent blog post. While this failed, the cyberattackers then used social engineering to convince the victim to download the AnyDesk tool for remote access, which they eventually achieved.

The attacker loaded multiple "suspicious files" onto the victim's machine via a connection that was established to a command-and-control (C2) server, one of which was DarkGate, according to Trend Micro. The RAT, distributed as usual via an AutoIt script, enabled remote control over the user's machine, executed malicious commands, gathered system information, and connected to a command-and-control (C2) server.

**A Multistage Vishing Cyberattack**

The multistage attack started off in a more typical DarkGate way, through a flood of thousands of phishing emails sent to the victim's inbox. The emails were followed up with a Microsoft Teams call purportedly for technical support, which kicked off the vishing attack.

The caller claimed to be an employee of an external supplier of the victim's company needing assistance, and instructed the victim to download the Microsoft Remote Support application.

"However, the installation via the Microsoft Store failed," Trend Micro researchers Catherine Loveria, Jovit Samaniego, and Gabriel Nicoleta wrote in the post. "The attacker then instructed the victim to download AnyDesk via browser and manipulate the user to enter her credentials to AnyDesk."

The attacker used AnyDesk to set up a communication channel to C2 and initiate various malicious scripts and eventually a PowerShell command to drop DarkGate using the Autoit legitimate Windows automation and scripting tool favored by attackers for obfuscation and defense evasion. After installation, the attack also loaded files and a registry entry for persistence.

**Another Channel for Spreading DarkGate Malware**

While ultimately the attack was stopped before data could be exfiltrated from the victim's machine, it demonstrates DarkGate actors using yet another means to spread the formidable RAT, adding to a long list of previously used delivery methods, the researchers said.

DarkGate has been used to target users around the world since at least 2017 and integrates multiple diverse and malicious functions. Among its capabilities are executing commands for gathering system information, mapping networks, and doing directory traversal, as well as launching Remote Desktop Protocol (RDP), hidden virtual network computing, AnyDesk, and other remote access software.

DarkGate also has features to support cryptocurrency mining, keylogging, privilege escalation, and stealing information from browsers, and is even known to carry additional payloads, including other RATs like Remcos.

**How to Protect Against Sophisticated Vishing Attacks**

Vishing attacks are becoming ever more psychologically sophisticated, with attackers even resorting to physical intimidation to coerce victims into complying with demands. Training employees on signs of a vishing attack, including staying up to date on the latest tactics, is becoming increasingly important as these attacks escalate.

"Well-informed employees are less likely to fall victim to social engineering attacks, strengthening the organization’s overall security posture," the researchers wrote.

Organizations also should "thoroughly vet third-party technical support providers" to "ensure that any claims of vendor affiliation are directly verified before granting remote access to corporate systems, the researchers wrote. Moreover, they should establish cloud-vetting processes to evaluate and approve remote access tools, such as AnyDesk to assess security compliance and vendor reputation before putting them in use.

Whitelisting approved remote access tools and blocking any unverified applications as well as integrating multifactor authentication (MFA) on remote access tools also reduce "the risk of malicious tools being used to gain control over internal machines," the researchers wrote.

**About the Author**

Elizabeth Montalbano is a freelance writer, journalist, and therapeutic writing mentor with more than 25 years of professional experience. Her areas of expertise include technology, business, and culture. Elizabeth previously lived and worked as a full-time journalist in Phoenix, San Francisco, and New York City; she currently resides in a village on the southwest coast of Portugal. In her free time, she enjoys surfing, hiking with her dogs, traveling, playing music, yoga, and cooking.

Microsoft Teams Vishing Spreads DarkGate RAT

Plus: The US indicts North Koreans in fake IT worker scheme, file-sharing firm Cleo warns customers to patch a vulnerability amid live attacks, and more.

What a week! On Monday, police arrested 26-year-old Luigi Mangione and charged him in the murder of UnitedHealthcare CEO Brian Thompson. Mangione’s five-day run from authorities ended after he was spotted eating at a McDonald’s in Altoona, Pennsylvania, about 300 miles from Manhattan, where Thompson was gunned down on the morning of December 4. Authorities say they found Mangione carrying fake IDs and a 3D-printed “ghost gun,” the model of which is known as the FMDA, or “Free Men Don’t Ask.”

Meanwhile, a flood of mysterious drone sightings across New Jersey and neighboring states caused so much havoc, it quickly gained federal attention. While many people wondered why the US military couldn’t just shoot down the drones, the FBI, Department of Homeland Security, and independent experts say the drone mystery may not be much of a mystery, and the drones are probably mostly just airplanes.

As for more terrestrial threats, we dove into the far-right realm of “Active Clubs,” small groups of young, fitness-focused men who are steeped in extremist ideology and linked to several violent attacks. While the man who helped invent the Active Club network, Robert Rundo, was sentenced in federal court this week, Active Clubs around the world are proliferating.

Finally, we investigated cheating schemes that use tiny cameras to gain an illicit edge in poker, and we interrogated the ways humans will use generative AI to make the world a more dangerous place.

But that’s not all. Each week, we round up the privacy and security news we didn’t cover in depth ourselves. Click the headlines to read the full stories. And stay safe out there.

Back in May, Microsoft jubilantly announced Recall, an AI feature for some Windows PCs that silently takes screenshots every five seconds and then allows you to easily search through the resulting digital footprint. Forgotten where you saw a recipe online? Tapping a couple of keywords into Recall could, in theory, find the dish again. It didn’t take long for the privacy and security community to find gaping holes in the feature.

In response, Microsoft delayed Recall’s launch and eventually made some significant changes—such as making Recall opt-in rather than on by default, better encrypting information captured by Recall, and adding authentication to access data that it stored. Recall finally launched for some users this month.

However, this week, testing of Recall by Tom’s Hardware demonstrated that a key safeguard put in place by Microsoft can still fail. With a Recall setting called “filter sensitive information” turned on, Tom’s Hardware’s tests found that it still took screenshots of some sensitive information—such as credit card numbers and Social Security numbers. When the publication typed a credit card number and a username and password into a Notepad window, they were gathered in the screenshots. “Similarly, when I filled out a loan application PDF in Microsoft Edge, entering a social security number, name and DOB, Recall captured that,” Avram Piltch writes. The tool, however, didn’t record details when they were entered on a couple of online stores.

Microsoft pointed to its product information that says the system will get better over time and that people should report if sensitive information is captured by Recall. While that may be the case, it’s unlikely that any system can correctly determine what is and isn’t sensitive every time. And that could make it a bigger target for hackers.

**14 North Koreans Identified and Indicted as Fraudulent IT Workers**

For years, North Korean nationals posing as tech workers have tried to get hired by global businesses so they can send their wages back to help the Hermit Kingdom pay for its nuclear programs. Increasingly, some companies are revealing they were targeted, and investigators are unravelling the schemes. This week, the US government indicted 14 North Koreans for their alleged role in generating $88 million, stealing sensitive business information, and attempting to use that information to extort more payments from companies. To get hired, the FBI alleges, the IT workers stole real identities, paid people in the US to use their home Wi-Fi connections, or paid them to attend job interviews. Researchers from Google-owned cybersecurity firm Mandiant who focus on North Korea say that in recent months they’ve seen IT workers following through on leaking sensitive data and demanding more cryptocurrency than ever before—although, they say this desperation could be a sign of the schemes becoming less effective.

**Cleo File-sharing Software Exploited to Spread Cybercriminal Malware**

File-sharing software firm Cleo this week warned customers to implement a new security patch to prevent a wave of intrusions by cybercriminals actively exploiting a vulnerability in its code. Researchers at security firm Huntress Labs told news outlet Recorded Future that at least two dozen organizations have already been breached by the hackers exploiting the Cleo flaw. Huntress has found a sample of malware found on those victims’ networks it calls Malichus, and says it appears to have been used by a sophisticated hacker group. Huntress also noted that Blue Yonder, a software firm breached by the Termite ransomware gang in November, had a vulnerable instance of Cleo’s software exposed on its network, and some evidence suggests the same cybercrime group may now be using the software’s vulnerability to hit other victims. Cleo first released a patch for the bug currently being exploited in October, but hackers appear to have circumvented it, and the company is urging customers to apply its new patch now.

**US Sanctions Chinese Hackers Who Allegedly Hijacked Thousands of Firewalls**

For five years, UK cybersecurity firm Sophos engaged in a cat-and-mouse game with a mysterious group of Chinese hackers targeting the company’s firewalls as a vector to break into its customers’ networks, as WIRED chronicled in October. Sophos went so far as to download surveillance “implants” to its devices that the hackers were testing to better monitor and preempt their intrusion techniques—and to gain more information about who they were and where they operated. Now, following Sophos’s revelations, those hackers have been hit with sanctions from the US government, and one of them has been indicted by name. Sichuan Silence Information Technology, based in Chengdu, and an alleged hacker named Guan Tianfeng are accused of hijacking 81,000 firewalls by exploiting a zero-day vulnerability that Guan is said to have discovered. Sichuan Silence, a known seller of disinformation tools to the Chinese government, and Guan are accused of targeting 23,000 firewalls in the US specifically, 36 of which were used in US critical infrastructure networks. The US State Department also issued a $10 million bounty for information about the company or Guan.

Microsoft’s AI Recall Tool Is Still Sucking Up Credit Card and Social Security Numbers

PRESS RELEASE

BOSTON — December 12, 2024 — Zerto, a Hewlett Packard Enterprise company, today announced the launch of the Zerto Cloud Vault, which delivers Zerto’s best-in-class cyber resilience capabilities as a service through managed service providers (MSPs). Zerto’s security-focused MSP partners at launch include Assurestor, Converge, LincolnIT, and Verinext. Built on the capabilities of the Zerto Cyber Resilience Vault, the Zerto Cloud Vault is a cloud-based, fully managed solution that offers logical air-gapping, immutability, and clean room recovery.

Ransomware attacks remain a harsh fact of life for most businesses with collective ransomware losses totaling over $1 billion in 2023 alone. This financial toll is exacerbated by the resulting data loss and downtime, with some estimates suggesting upwards of $1 million in losses per hour of downtime for larger businesses.

With Zerto Cloud Vault, customers can leverage MSPs that deliver tailored cyber resilience strategies to meet the specific requirements of their organizations. Cloud Vault is the latest offering in HPE’s comprehensive cyber resilience portfolio and complements the existing Zerto Cyber Resilience Vault, which is deployed as a self-hosted, on-premises stack.

Zerto Cloud Vault capabilities include:

*   Managed Cyber Services: Take advantage of MSP experts who know how to prevent and mitigate attacks and have built their services on top of Zerto’s award-winning technology.
    
*   Real-Time Encryption Detection: Detect encryption anomalies in real-time and be alerted within seconds to potential issues through integration with cybersecurity dashboards.
    
*   Immutable Data Copies: Retain data for up to 12 months and protect against ransomware attacks through immutable copies, all without impacting production workloads, without any agents or snapshots.
    
*   Clean Room Recovery: Leverage the elasticity of the cloud to create clean environments on demand with isolated networks that are protected from attackers. Use it to validate data, scrub malware, and perform forensics before recovering back into production.
    
*   Non-Disruptive Testing: With Zerto’s non-disruptive solutions, businesses can test more frequently and comprehensively, including conducting cyber recovery tests at any time on entire sites, multiple sites, or individual VMs. These tests can be used to validate recovery plans and train incident response teams.
    
*   Fully isolated from production environments: Ability to run different security postures within product and vault environments.
    

“Zerto’s disaster recovery and cyber resilience solutions offer peace of mind to businesses struggling to combat ransomware attacks,” said Jim O’Dorisio, senior vice president and general manager, HPE Storage. “Leveraging our cyber resilience capabilities, MSPs will be able to bring fully managed Cloud Vault services to even more organizations, helping them thwart the plans of attackers and keep precious business assets safe.”

Coupled with the expertise of Zerto’s vetted MSP partners, the hosted Zerto Cloud Vault mitigates the most devastating ransomware scenarios while keeping businesses in compliance with state and federal regulations — reducing the risk of substantial fines or prosecution. Where other solutions offer detrimental lengthy recovery point objectives (RPOs) and recovery time objectives (RTOs), Zerto Cloud Vault slashes both, minimizing the risks of disruption and allowing businesses to get back on their feet as fast as possible after the inevitable happens. Zerto is a part of HPE’s hybrid cloud business, helping HPE customers protect workloads and data across hybrid IT environments.

Partner Quotes

“Our Gold Standard for cyber recovery considers a product’s recoverability readiness, non-disruptive testing capability, and speed of data recovery; we found that Zerto substantially delivered on all these points. Combined with Cloud Vault, the protection provided from ransomware attacks was robust and allowed pinpoint recovery faster than any other products evaluated,” said Stephen Young, executive director, Assurestor.

“Ransomware attacks are a real threat to the data and operations of organizations of all sizes. Having the ability to offer a cloud vault with Zerto technology gives our clients added protection against ransomware and peace of mind that they can recover successfully,” said John Antimisiaris, executive vice president, LincolnIT.

“Some of our clients’ infrastructure protection needs to be kept with a very tight RPO, but they still need some deeper recovery options than simple replication can provide. A cyberattack is seldom clearly understood, even when recovery efforts have begun. Zerto Cloud Vault provides immutable protection history that can be leveraged to find the latest clean point in time for replicated hosts,” said Jeremy Brovage, product engineer and solutions architect, Converge Enterprise Cloud.

“Cyberattacks that encrypt data are one of the primary disruptors requiring data recovery. Zerto provides the best tools to recover quickly and with the least data loss in a cloud vault,” said Nick Martino, product manager, managed services, Verinext.

Explore the Zerto Cloud Vault and discover how it empowers businesses to safeguard their data and operations: Learn More. 

About Zerto

Zerto, a Hewlett Packard Enterprise company, empowers customers to run an always-on business by simplifying the protection, recovery, and mobility of on-premises and cloud applications. Zerto eliminates the risk and complexity of modernization and cloud adoption across private, public, and hybrid deployments. The simple, software-only solution uses continuous data protection at scale to solve for ransomware resilience, disaster recovery, and multi-cloud mobility. Zerto is trusted by over 9,500 customers globally, and is powering offerings for Microsoft Azure, IBM Cloud, Google Cloud, Oracle Cloud, and more than 350 managed service providers.

Zerto Introduces Cloud Vault Solution for Enhanced Cyber Resilience Through MSPs

Businesses deploying large language models and other GenAI systems have a growing collection of open source tools for testing AI security.

Source: Olena Ivanova via Shutterstock

Companies deploying generative artificial intelligence (GenAI) models — especially large language models (LLMs) — should make use of the widening variety of open source tools aimed at exposing security issues, including prompt-injection attacks and jailbreaks, experts say.

This year, academic researchers, cybersecurity consultancies, and AI security firms released a growing number of open source tools, including more resilient prompt injection tools, frameworks for AI red teams, and catalogs of known prompt injections. In September, for example, cybersecurity consultancy Bishop Fox released Broken Hill, a tool for bypassing the restrictions on nearly any LLM with a chat interface.

The open source tool can be trained on a locally hosted LLM to produce prompts that can be sent to other instances of the same model, causing those instances to disobey their conditioning and guardrails, according to Bishop Fox.

The technique works even when companies deploy additional guardrails — typically, simpler LLMs trained to detect jailbreaks and attacks, says Derek Rush, managing senior consultant at the consultancy.

"Broken Hill is essentially able to devise a prompt that meets the criteria to determine if \[a given input\] is a jailbreak," he says. "Then it starts changing characters and putting various suffixes onto the end of that particular prompt to find \[variations\] that continue to pass the guardrails until it creates a prompt that results in the secret being disclosed."

The pace of innovation in LLMs and AI systems is astounding, but security is having trouble keeping up. Every few months, a new technique appears for circumventing the protections used to limit an AI system's inputs and outputs. In July 2023, a group of researchers used a technique known as "greedy coordinate gradients" (GCG) to devise a prompt that could bypass safeguards. In December 2023, a separate group created another method, Tree of Attacks with Pruning (TAP), that also bypasses security protections. And two months ago, a less technical approach, known as Deceptive Delight, was introduced that uses fictionalized relationships to fool AI chatbots to violate their systems restrictions.

The rate of innovation in attacks underscores the difficulty of securing GenAI systems, says Michael Bargury, chief technology officer and co-founder of AI security firm Zenity.

"It's an open secret that we don't really know how to build secure AI applications," he says. "We are all trying, but we don't know how to yet, and we are basically figuring that out while building them with real data and with real repercussions."

**Guardrails, Jailbreaks, and PyRITs**

Companies are erecting defenses to protect their valuable business data, but whether those defenses are effective remains a question. Bishop Fox, for example, has several clients using programs such as PromptGuard and LlamaGuard, which are LLMs programmed to analyze prompts for validity, says Rush.

"We're seeing a lot of clients \[adopting\] these various gatekeeper large language models that try to shape, in some manner, what the user submits as a sanitization mechanism, whether it's to determine if there's a jailbreak or perhaps it's to determine if it's content-appropriate," he says. "They essentially ingest content and output a categorization of either safe or unsafe."

Now researchers and AI engineers are releasing tools to help companies determine whether such guardrails are actually working.

Microsoft released its Python Risk Identification Toolkit for generative AI (PyRIT) in February 2024, for example, an AI penetration testing framework for companies that want to simulate attacks against LLMs or AI services. The toolkit allows red teams to build an extensible set of capabilities for probing various aspects of an LLM or GenAI system.

Zenity uses PyRIT regularly in its internal research, says Bargury.

"Basically, it allows you to encode a bunch of prompt-injection strategies, and it tries them out on an automated basis," he says.

Zenity also has its own open source tool, PowerPwn, a red-team toolkit for testing Azure-based cloud services and Microsoft 365. Zenity's researchers used PowerPwn to find five vulnerabilities in Microsoft Copilot.

**Mangling Prompts to Evade Detection**

Bishop Fox's Broken Hill is an implementation of the GCG technique that expands on the original researchers' efforts. Broken Hill starts with a valid prompt and begins changing some of the characters to lead the LLM in a direction that is closer to the adversary's objective of disclosing a secret, Rush says.

"We give Broken Hill that starting point, and we generally tell it where we want to to end up, like perhaps the word 'secret' being within the response might indicate that it would disclose the secret that we're looking for," he says.

The open source tool currently works on more than two dozen GenAI models, according to its GitHub page.

Companies would do well to use Broken Hill, PyRIT, PowerPwn, and other available tools to explore their AI applications vulnerabilities because the systems will likely always have weaknesses, says Zenity's Bargury.

"When you give AI data — that data is an attack vector — because anybody that can influence that data can now take over your AI if they are able to do prompt injection and perform jailbreaking," he says. "So we are in a situation where, if your AI is useful, then it means it's vulnerable because in order to be useful, we need to feed it data."

**About the Author**

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline Journalism (Online) in 2003 for coverage of the Blaster worm. Crunches numbers on various trends using Python and R. Recent reports include analyses of the shortage in cybersecurity workers and annual vulnerability trends.

Generative AI Security Tools Go Open Source

Learn how blockchain and smart contracts improve cybersecurity factors in online transactions, remove the element of fraud, and…

Learn how blockchain and smart contracts improve cybersecurity factors in online transactions, remove the element of fraud, and promote trust with automation and transparency.

In this connected age, where data breaches and online fraud are rising concerns, blockchain technology and smart contracts have become powerful solutions for secure transactions. Their ability to provide transparency, remove middlemen, and enable fraud-resistant mechanisms has made them a key part of today’s financial systems and more.

****How Blockchain Builds a Secure Foundation****

Blockchain is best known as the technology behind cryptocurrencies like Bitcoin and Ethereum, but its uses go well beyond digital currencies. It works as a decentralized ledger, recording transactions across multiple points in a network, which makes it incredibly secure and nearly impossible for cybercriminals to tamper with or hack.

When it comes to digital payments, this decentralization is key. Traditional payment systems rely on central authorities, which are vulnerable to malware attacks, breaches and fraud.

Blockchain, on the other hand, distributes the transaction record across a network, reducing the risk of a single point of failure. The stability of blockchain also means that once a transaction is recorded, it cannot be changed, adding another layer of trust and accountability.

****The Role of Smart Contracts****

Smart contracts take blockchain’s security a step further. These self-executing agreements operate based on established criteria rules written into code. Once the conditions are met, the contract automatically executes, removing the need for intermediaries.

For example, imagine buying a digital service. A smart contract can be programmed to release payment only when the service is delivered and verified. This reduces the risk of fraud and disputes, as the terms are enforced automatically.

Another benefit is efficiency. By automating processes, smart contracts not only save time but also reduce costs associated with manual verification and third-party oversight. This makes them especially appealing for industries like supply chain management, insurance, and real estate, where trust and speed are paramount.

However, security vulnerabilities have also affected smart contracts, which need to be addressed by skilled blockchain developers or specialized auditing teams. According to Onchainpay, a European secure cryptocurrency payment gateway, smart contract vulnerabilities are one single major factor that continues to pose security risks within the blockchain infrastructure.

A recent systematic literature review identified 101 distinct vulnerabilities in Ethereum smart contracts, categorized into ten groups. In particular, in the first quarter of 2024 alone, exploits targeting these vulnerabilities resulted in nearly $45 million in losses across 16 incidents, averaging $2.8 million per exploit.

****Real-World Applications****

Nevertheless, despite ups and downs, Blockchain and smart contracts are already being used to secure payments and transactions globally. As noticed by Halborn, Microsoft, JPMorgan Chase, Walmart, IBM and Amazon are just some major examples.

In international payments, they enable faster and more transparent transfers without the hefty fees associated with traditional banking. In e-commerce, smart contracts can automate escrow services, protecting both buyers and sellers.

Even outside of payments, blockchain is enhancing security. From safeguarding healthcare records to verifying the authenticity of luxury goods, its applications are vast.

****What This Means for Businesses****

As cybersecurity threats grow, businesses opting for blockchain and smart contracts gain a competitive advantage in security and efficiency. For consumers, this means safer and more reliable transactions.

While the technology isn’t without challenges, such as regulatory hurdles and technical complexity, its benefits are clear. Businesses looking to adopt secure payment systems should consider solutions that leverage blockchain and smart contracts to protect their data and enhance trust in their transactions.

1.  **6 of the Best Crypto Bug Bounty Programs**
2.  **5 Platforms for Identifying Smart Contract Vulnerabilities**
3.  **Growing Importance of Secure Crypto Payment Gateways**
4.  **Best Crypto Marketing Agencies for Web3 Security Brands**
5.  **Why Hosting firms have started accepting crypto payments**
6.  **Top 5 Platforms for Identifying Smart Contract Vulnerabilities**

The Role of Blockchain and Smart Contracts in Securing Digital Transactions

As the adoption of LCNC grows, so will the complexity of the threats organizations face.

Source: ArtemisDiana via Alamy Stock Photo

COMMENTARY

As organizations lean into low-code/no-code (LCNC) platforms to streamline development and empower citizen developers, security risks become increasingly challenging to manage. One of the more under-the-radar LCNC threats is OData injection, an attack vector that can expose sensitive corporate data and is predominant on the Microsoft Power Platform. This new vulnerability is poorly understood by security professionals in LCNC environments, where traditional safeguards are lacking.

**What Is OData? **

OData, or Open Data Protocol, is an OASIS standard that has gained traction in LCNC platforms as a way to manage and deliver data through REST APIs. It's widely adopted because it allows seamless communication between applications and data sources, regardless of the underlying data storage model. In LCNC environments, it is commonly used as a query language to retrieve data from a variety of sources, such as SQL databases, SharePoint, or Dataverse.

OData is particularly valuable in LCNC platforms because of its simplicity — developers don't need to be database experts to use it, and the same query language can be used for very different data sources. 

**The OData Injection Threat**

OData injection manipulates user input that is later used by an application or automation to form an OData query. The query is then applied to an enterprise data source. This allows an attacker to gain unauthorized access to manipulate or exfiltrate sensitive user and corporate data. 

While SQL injection (SQLi) is generally understood by security professionals, OData injection poses a different set of challenges, especially in LCNC environments, where multiple data sources are often connected and managed by citizen developers with minimal security training. Unlike SQLi, which is confined to relational databases, OData can connect to a wide array of data sources, including custom applications and third-party services, broadening the potential impact of an attack. 

OData also lacks the well-established security practices that have been developed for SQL. For example, SQLi can typically be mitigated with parameterized queries, a practice that has become standard over the years. OData injection, however, doesn't have a similar one-size-fits-all solution. Developers must create custom input validation mechanisms — a manual and error-prone process. In addition, the general lack of awareness of OData injection techniques further reduces the likelihood that custom validation methods will be implemented. 

**A New External Attack Surface**

OData vulnerabilities in LCNC environments often stem from the unrecognized risks associated with external data inputs. These are frequently integrated into workflows that manipulate critical enterprise data, including Web forms, email messages, social media, and external Web applications. These inputs typically are accepted without stringent validation, leaving the attack surface vulnerable and often undefended, as developers and security teams may overlook these sources as potential risks.  

This oversight allows attackers to exploit these inputs by injecting malicious OData queries. For instance, a simple product feedback form could be exploited to extract sensitive data or modify stored information. 

**Security Challenges **

Because most citizen developers don't have formal security training and are often unfamiliar with the dangers of accepting unchecked external inputs in their workflows, OData Injection vulnerabilities can flourish undetected.

Also, unlike SQL injection, validating user inputs in OData queries requires a more hands-on approach. Developers must manually sanitize inputs — removing harmful characters, ensuring proper formatting, and guarding against common injection techniques. This process takes time, effort, and more advanced programming knowledge that most LCNC developers lack.

Furthermore, in traditional development environments, security vulnerabilities are often tracked and remediated through ticketing systems or backlog management tools like Jira. This formal process does not exist in most LCNC development environments, where developers may not be full-time coders and have no formalized way to handle bug tracking or vulnerability management.

**Mitigation Best Practices**

Combating OData injection requires a proactive security strategy. Ideally, LCNC developers should be trained on OData query risks and how external inputs could be exploited. This is unrealistic, since citizen developers aren't full-time coders. 

Instead, automation can play a significant role in monitoring and detecting OData injection vulnerabilities. Security teams should deploy tools that continuously assess LCNC environments for potential vulnerabilities, especially as new applications and workflows are created. This will help identify weaknesses early and quickly provide developers with actionable insights into how to fix them.

Collaboration between security teams and LCNC developers is another essential piece of the puzzle. Security teams should be granted access to monitor the development process in real-time, particularly in environments where critical corporate data is being processed. When vulnerabilities are identified, security must communicate clearly with developers, offering specific guidance on how to remediate issues. This could include best practices for input validation and sanitation, as well as tools for automating the process where possible.

Lastly, security should be integrated into the LCNC development life cycle. Much like the "shift-left" movement in traditional software development, security checks should be built into the LCNC workflow from the outset. Automated testing tools can be leveraged to scan for vulnerabilities as applications are being built, reducing the likelihood of OData injection vulnerabilities slipping through the cracks.

As the adoption of LCNC continues to grow, so will the complexity of the threats organizations face. Addressing LCNC vulnerabilities like OData injection now will help keep enterprises safe in the long run.

**About the Author**

CTO & Co-Founder, Nokod Security

Amichai Shulman, chief technology officer (CTO) and co-founder of Nokod Security, is an internationally recognized expert and researcher in data and application security. He was previously co-founder, CTO, and chief scientist at Imperva for more than 15 years. Amichai is a frequent conference speaker and industry authority on cybersecurity. As an investor and adviser, he has guided successful startups, including Intsights and SkyFence. Amichai is also an adjunct professor at the Technion – Israel Institute of Technology.

OData Injection Risk in Low-Code/No-Code Environments

AI creates what it’s told to, from plucking fanciful evidence from thin air, to arbitrarily removing people’s rights, to sowing doubt over public misdeeds.

OpenAI CEO Sam Altman expects AGI, or artificial general intelligence—AI that outperforms humans at most tasks—around 2027 or 2028. Elon Musk’s prediction is either 2025 or 2026, and he has claimed that he was "losing sleep over the threat of AI danger." Such predictions are wrong. As the limitations of current AI become increasingly clear, most AI researchers have come to the view that simply building bigger and more powerful chatbots won’t lead to AGI.

However, in 2025, AI will still pose a massive risk: not from artificial superintelligence, but from human misuse.

These might be unintentional misuses, such as lawyers over-relying on AI. After the release of ChatGPT, for instance, a number of lawyers have been sanctioned for using AI to generate erroneous court briefings, apparently unaware of chatbots’ tendency to make stuff up. In British Columbia, lawyer Chong Ke was ordered to pay costs for opposing counsel after she included fictitious AI-generated cases in a legal filing. In New York, Steven Schwartz and Peter LoDuca were fined $5,000 for providing false citations. In Colorado, Zachariah Crabill was suspended for a year for using fictitious court cases generated using ChatGPT and blaming a "legal intern" for the mistakes. The list is growing quickly.

Other misuses are intentional. In January 2024, sexually explicit deepfakes of Taylor Swift flooded social media platforms. These images were created using Microsoft’s “Designer” AI tool. While the company had guardrails to avoid generating images of real people, misspelling Swift's name was enough to bypass them. Microsoft has since fixed this error. But Taylor Swift is the tip of the iceberg, and non-consensual deepfakes are proliferating widely—in part because open-source tools to create deepfakes are available publicly. Ongoing legislation across the world seeks to combat deepfakes in hope of curbing the damage. Whether it is effective remains to be seen.

In 2025, it will get even harder to distinguish what’s real from what’s made up. The fidelity of AI-generated audio, text, and images is remarkable, and video will be next. This could lead to the "liar's dividend": those in positions of power repudiating evidence of their misbehavior by claiming that it is fake. In 2023, Tesla argued that a 2016 video of Elon Musk could have been a deepfake in response to allegations that the CEO had exaggerated the safety of Tesla autopilot leading to an accident. An Indian politician claimed that audio clips of him acknowledging corruption in his political party were doctored (the audio in at least one of his clips was verified as real by a press outlet). And two defendants in the January 6 riots claimed that videos they appeared in were deepfakes. Both were found guilty.

Meanwhile, companies are exploiting public confusion to sell fundamentally dubious products by labeling them “AI.” This can go badly wrong when such tools are used to classify people and make consequential decisions about them. Hiring company Retorio, for instance, claims that its AI predicts candidates' job suitability based on video interviews, but a study found that the system can be tricked simply by the presence of glasses or by replacing a plain background with a bookshelf, showing that it relies on superficial correlations.

There are also dozens of applications in health care, education, finance, criminal justice, and insurance where AI is currently being used to deny people important life opportunities. In the Netherlands, the Dutch tax authority used an AI algorithm to identify people who committed child welfare fraud. It wrongly accused thousands of parents, often demanding to pay back tens of thousands of euros. In the fallout, the Prime Minister and his entire cabinet resigned.

In 2025, we expect AI risks to arise not from AI acting on its own, but because of what people do with it. That includes cases where it _seems_ to work well and is over-relied upon (lawyers using ChatGPT); when it works well and is misused (non-consensual deepfakes and the liar's dividend); and when it is simply not fit for purpose (denying people their rights). Mitigating these risks is a mammoth task for companies, governments, and society. It will be hard enough without getting distracted by sci-fi worries.

Tag

#microsoft