Researchers measured a threefold increase in credential stealing between 2023 and 2024, with more than 11.3 million such thefts last year.

Source: Artur Marciniec via Alamy Stock Photo

NEWS BRIEF

After analyzing more than a million pieces of malware collected in 2024, researchers have found that 25% of them target user credentials.

That's three times the number from 2023 and has bumped stealing credentials from password stores into the top 10 techniques listed in the MITRE ATT&CK framework, which accounted for 93% of all malicious cyber activity in 2024.

In "The Red Report 2025" conducted by Picus Security, researchers observed that the attackers are prioritizing "complex, prolonged, multi-stage attacks that require a new generation of malware to succeed." In what the researchers dubbed "SneakThief," threat actors are looking to revolutionize info-stealing malware, focusing on increased stealth, persistence, and automation.

The researchers add that threat actors likely have their sights set on these malware attributes in order to pull off "the perfect heist," adding that most malware samples now have the capability to do so with more than a dozen malicious actions installed to help bad actors evade defenses, exfiltrate data, and more.

The researchers also report they found no evidence that cybercriminals are using AI-driven malware, and that malware samples on average can complete 14 malicious actions. And of the millions of cybercrime acts seen in 2024, exfiltration and stealth tactics made up 11.3 million.

"Focusing on Top 10 MITRE ATT&CK techniques is the most viable way to stop the kill chain of sophisticated malware strains as early as possible," said Volkan Ertürk, CTO and co-founder of Picus. "SneakThief malware is not an exception; enterprise security teams can stop 90% of malware by focusing on just 10 of MITRE's entire library of techniques."

**About the Author**

Skilled writer and editor covering cybersecurity for Dark Reading.

Credential Theft Becomes Cybercriminals' Favorite Target

Targets are lured into a fake interview process that convinces them to download malware needed for a virtual interview.

Source: imageBROKER.com GmbH & Co. KG via Alamy Stock Photo

NEWS BRIEF

In a new patch for its on-device malware tool, Apple is pushing signature updates to XProtect in order to block variants of a malware belonging to what is known as the macOS Ferret family.

This malware has been identified as part of "Contagious Interview," a North Korean campaign involving threat actors luring in targets and convincing them to install malware onto their devices through a fake job interview process. The other variants in the campaign include: FROSTYFERRET\_UI, FRIENDLYFERRET\_SECD, and MULTI\_FROSTYFERRET\_CMDCODES.

The DPRK malware family was first detailed by researchers in December 2024 and again in January where, as part of the campaign, targets are asked to communicate with an "interviewee" through a link that requests to install a piece of software required for virtual meetings.

Once installed, it runs a malicious shell script and installs a persistence agent, as well as an executable impersonating a Google Chrome update. 

The Contagious Interview attack chains are designed to drop JavaScript-based malware "BeaverTail," which delivers a Python backdoor known as InvisibleFerret, and harvests sensitive data from Web browsers and crypto wallets.

And now researchers at SentinelOne are highlighting samples they're calling "FlexibleFerret" that went undetected by XProtect as of Feb. 3, suggesting that the threat actors are honing their tactics to evade detection. This component dates as far back as November 2023.

"In an example in late December, one 'commenter' left instructions leading to the download of Ferret family droppers," stated the SentinelOne researchers. "This suggests that the threat actors are happy to expand the vectors by which they deliver the malware beyond the specific targeting of job seekers to developers more generally."

**About the Author**

Skilled writer and editor covering cybersecurity for Dark Reading.

Ferret Malware Added to 'Contagious Interview' Campaign

Ransomware actors are offering individuals millions to turn on their employers and divulge private company information, in a brand-new cybercrime tactic.

Source: Mayam studio via Shutterstock

Ransomware actors are utilizing a previously unseen tactic in their ransomware notes: posting advertisements to solicit insider information.

Researchers at the GroupSense threat intelligence team shared their findings with Dark Reading, including screenshots of the strategies these gangs are using. Groups including Sarcoma and another syndicate believed to be impersonating LockBit ransomware, known as DoNex, have adopted the strategy, the firm noted.

Part of one ransomware note includes the usual details stating that the company is in critical condition, its backups destroyed, and databases exported. Farther down in the message, however, the group states: "If you help us find this company's dirty laundry you will be rewarded. You can tell your friends about us. If you or your friend hates his boss, write to us and we will make him cry and the real hero will get a reward from us."

In a different ransom note, the threat actors write: "Would you like to earn millions of dollars $$$ ?  Our company acquire access to networks of various companies, as well as insider information that can help you steal the most valuable data of any company.  You can provide us accounting data for the access to any company, for example, login and password to RDP, VP, corporate email, etc. "

Related:Credential Theft Becomes Cybercriminals' Favorite Target

A ransom note from a threat group impersonating LockBit. Source: GroupSense

The threat actors then go on to detail how those who are interested can open their letter and launch a virus on their work computer. The communication is done through Tox messenger so that the users privacy is "guaranteed."

Kurtis Minder, CEO and founder at GroupSense, notes that the company sees a variety of ransom notes in the course of incident response, however, it's only been this past week that its researchers have noticed the "pseudo advertisements" at the bottom of these notes.

"I've been asking my team and kind of speculating as to why this would be a good place to put an advertisement," says Minder. "I don't know the right answer, but obviously these notes do get passed around." He notes that these threat actors may maintain a "why not" attitude toward incorporating such ads into their ransom notes. And when one ransomware actor starts a new tactic, the rest are quick to follow.

But for any individuals interested in taking up such an offer from cybercriminals, it's better to be safe than sorry.

"These folks have no accountability, so there's no guarantee you would get paid anything," Minder adds. "You trying to capitalize on this is pretty risky from an outcome perspective."

GroupSense continues to look through past ransom notes to find any earlier indication of the trend, and Minder says he expects to find more ads in addition to those already discovered.

Related:Ferret Malware Added to 'Contagious Interview' Campaign

The news comes as ransomware activity continues to grow, with cyberattackers raking in hefty profits despite a rash of law enforcement actions over the course of the past year.

**About the Author**

Skilled writer and editor covering cybersecurity for Dark Reading.

Cybercriminals Court Traitorous Insiders via Ransom Notes

Funnull CDN rents IPs from legitimate cloud service providers and uses them to host criminal websites, continuously cycling cloud resources in and out of use and acquiring new ones to stay ahead of cyber-defender detection.

Source: Aleksia via Alamy Stock Photo

Researchers have linked the China-based Funnull content delivery network (CDN) to a malicious practice they've dubbed "infrastructure laundering," in which threat actors exploit mainstream hosting providers such as Amazon Web Services (AWS) and Microsoft Azure. The activity involves threat actors operating "hosting companies" that rent IP addresses from these providers and then map them to their criminal websites.

Researchers from Silent Push discovered the practice when they noticed that AWS and Microsoft Azure cloud hosting services are "often seen in large-scale use by threat actors," according to the recently published report. Further investigation led them to the discovery that Funnull CDN, a Chinese company that already has raised suspicions for other malicious activity, has been using this tactic to host a network of scam websites.

Funnull has rented more than 1,200 IPs from AWS and nearly 200 IPs from Microsoft, according to Silent Push. While these have nearly all been taken down as of this writing, the company continuously acquires new IPs every few weeks, using them and then dumping them before defenders can identify the malicious activity.

"While providers are consistently banning specific IP addresses used by the Funnull CDN, the pace is unfortunately not fast enough to keep up with processes being used to acquire the IPs," according to the report.

Related:EMEA CISOs Plan 2025 Cloud Security Investment

The tactic is complicated to defend against because it blends malicious activities with legitimate Web traffic, making it difficult for hosting providers to block access without creating a disruption for legitimate users, one security expert notes.

"By utilizing major providers, the bad actors make it much tougher for organizations to block IP ranges because those major providers may also be providing legitimate IP addresses for important Web services," observes Erich Kron, a security awareness advocate at cybersecurity company KnowBe4. "This precludes the ability to block large chunks of addresses easily."

**Running Multiple Scams**

Funnull CDN hosts more than 200,000 unique hostnames — approximately 95% of which are generated through domain generation algorithms (DGAs) — linked to "illicit activities such as investment scams and fake trading applications," according to the report.

"Moreover, these activities are directly associated with money laundering as a service on shell gambling websites that abuse the trademarks of a dozen popular casino brands and which are available online today," according to the report.

Related:Name That Edge Toon: In the Cloud

The activity uncovered by Silent Push is not the first time Funnull CDN has been tied to suspicious activity. Last year, the company purchased a domain, polyfill\[.\]io, that more than 100,000 websites use to deliver JavaScript code. Soon after, it was found being used as a conduit for a supply chain attack that used dynamically generated payloads, redirected users to pornographic and sports-betting sites, and could potentially lead to data theft, clickjacking, or other attacks.

At its peak in 2022, Funnull CDN's investment scam infrastructure had thousands of active domains, according to Silent Push. In 2024 that portfolio was more "modest" but still had some active sites, including cmegrouphkpd\[.\]info, which recently went offline but for the past two years had hosted a fake trading platform abusing CME Group's brand and logo.

**Is "Laundering" a Misnomer?**

AWS has made a public response to the findings in the report, verifying some of them and taking issue with others. The company said before it received Silent Push's report, it was "already aware of the activity" and was actively suspending the fraudulently acquired accounts linked to Funnull CDN's malicious activity.

"All accounts known to be linked to the activity are suspended," according to an AWS statement included in the Silent Push report. "We can confirm that there is no current risk from this activity, and no customer action is required."

Related:Tenable to Acquire Vulcan Cyber to Boost Exposure Management Focus

AWS also noted that the term "infrastructure laundering" to describe the activity is a misnomer, since it doesn't involve making illicit activity "clean."

"By using that phrase, the report insinuates that AWS is the intermediary to make the abusive activity appear legitimate and thereby harder to detect or block," the company said. "That’s incorrect."

AWS did not immediately respond to a request for comment from Dark Reading.

A Microsoft spokesperson told Dark Reading the tech giant is looking into the activity described in the report. Meanwhile, Silent Push will continue to investigate related activity from Funnull CDN and other threat actors, and will provide updates when appropriate, it said.

Businesses need to review their cloud accounts to avoid getting caught up in the activity, too. KnowBe4's Kron suggests that threat actors aren't likely to set up an account with a mainstream cloud provider with their own information; instead, they are probably using stolen accounts. These account takeovers, in turn, likely involve the use of stolen or cracked credentials, making the use of multifactor authentication (MFA) another potential way to mitigate this type of activity, he says.

Kron adds: "Organizations should review the accounts with access, audit transactions, and educate people on how to spot potential malicious activity within their cloud accounts."

**About the Author**

Elizabeth Montalbano is a freelance writer, journalist, and therapeutic writing mentor with more than 25 years of professional experience. Her areas of expertise include technology, business, and culture. Elizabeth previously lived and worked as a full-time journalist in Phoenix, San Francisco, and New York City; she currently resides in a village on the southwest coast of Portugal. In her free time, she enjoys surfing, hiking with her dogs, traveling, playing music, yoga, and cooking.

Chinese 'Infrastructure Laundering' Abuses AWS, Microsoft Cloud

Organizations and development teams need to evolve from "being prepared" to "managing the risk" of security breaches.

Kirsten Newcomer, Director, Cloud and DevSecOps Strategy, Red Hat

February 4, 2025

4 Min Read

Source: RTimages via Alamy Stock Photo

COMMENTARY

It's a perfect storm: The cost of a data breach is rising, known cyberattacks are becoming more frequent, security expertise is in short supply, and the demand for connectedness — to deliver and act on even the most sensitive of data across all devices, and all the way to the network edge — is unyielding. A recent example that affects anyone who texts between Android and iPhone devices is the Salt Typhoon attack. Meanwhile, industry and government regulations are tightening, demanding stricter proof of security measures and faster reporting of breaches, raising the stakes for "getting it wrong."

In its most recent analysis, Verizon Business found that organizations take an average of 55 days to remediate 50% of critical vulnerabilities listed in the Cybersecurity and Infrastructure Security Agency's (CISA's) Known Exploited Vulnerabilities (KEV) catalog. Unfortunately, cybercriminals respond far more quickly, with mass exploitations of the CISA KEV appearing on the Internet within a median of five days. 

That's why organizations and development teams must evolve from "being prepared" to "managing the risk" of security breaches.

Vulnerability risk management is not a new concept, but I am noticing that organizations are attempting to manage risk in one of two ways — by setting up guardrails (proactive) or patching (reactive). Neither is optimal.

The key is to balance the two, highlighting the critical importance of adopting a DevSecOps approach. "DevSec" solutions are focused on shifting security left by integrating security gates into the continuous integration and continuous delivery (CI/CD) pipeline. "SecOps" solutions are focused on detecting and responding to threats in the runtime environment. 

Here's a look at the challenges to each approach.

**The Vulnerability Patching Approach**

On its face, patching sounds simple enough: When a software vulnerability is revealed, patch it. However, that assumes that developers and security teams have the resources to quickly monitor for issues, create or identify patches, and then test and apply those patches — before cyberattackers can take advantage of the vulnerabilities themselves. 

AI will eventually help developers more efficiently identify vulnerabilities, but we're not at that point yet. Right now, AI and the demand for AI-enabled applications is only adding to the potential for unidentified vulnerabilities. AI code generation tools increase the likelihood of introducing hard-to-trace snippets of code from unidentified sources. While many of today's vulnerability scanners rely on identifying code packages rather than code snippets. 

**The Guardrails Approach**

The guardrails approach is more nuanced than the vulnerability patching approach, but it comes with its own set of challenges.

While organizations that focus on the patching approach take a more reactive stance, the guardrails approach is grounded in proactive protection and mitigating controls. These include:

*   Reducing the attack surface across the stack
    
*   Working toward continuous hardening and compliance improvement
    
*   Securing the application CI/CD pipeline using best practices, such as those recommended in slsa.dev: identifying provenance, hardening builds, verifying artifacts
    
*   Implementing automated, policy-based promotion and admission controls to ensure that applications have production-ready security before being deployed to production systems
    
*   Application of data protection controls such as encryption
    

*   Use of zones of control (fencing communications with network and API security controls) and microsegmentation
    
*   Prioritizing the use of Linux security solutions, such as SELinux and secure computing profiles (seccomp), as well as features focused on securing containers such as user namespaces and cgroupsv2
    

All of these strategies are highly effective; however, it's often challenging for organizations to integrate these and other guardrails into their infrastructure. It is even more challenging to harden existing application pipelines. Striking the balance between security and innovation has gotten more difficult as pressure to improve security increases from all sides and the impact of a security breach reverberates up and down the supply chain.

**Creating a Balanced Approach to Software Risk Management**

Used together, patching and guardrails can help organizations maintain a balance between efficient vulnerability management and proactive security monitoring and management. 

Organizations should assess risk based on key factors for their business, including what mitigating controls they have in place in the runtime environment. While the Common Vulnerability Scoring System, with Base Metrics and Temporal and Environmental Metrics, offers some indication of the level of risk a known vulnerability creates, this data does not and cannot account for the specific context of a deployed application. Organizations need to account for additional factors such as external exposure and mitigating controls.

Using open source can help, since the community is committed to transparency and clear communication about newly discovered vulnerabilities and how to get fixes for them. In fact, in addition to prioritizing the use of open source, organizations should take their cue from the open source community and establish their own processes for sharing detailed information about identified vulnerabilities — internally but also with partners and vendors following principles of responsible disclosure. 

Responsible disclosure and open data are critical for customers and communities to fully understand the vulnerabilities that may impact them, as well as to ensure that the data necessary to make appropriate, informed decisions is widely available.

Offering multiple remediation options, such as software updates and/or patches, and automated guardrails at all stages of the application life cycle including CI/CD and runtime mitigations, provides flexibility in addressing vulnerabilities across diverse environments. By combining these elements, organizations can create a comprehensive vulnerability risk management program that effectively mitigates security risks across their entire IT infrastructure.

**About the Author**

Director, Cloud and DevSecOps Strategy, Red Hat

Kirsten Newcomer works closely with Red Hat’s many security professionals across the Red Hat portfolio of open source offerings. She is a diversified software management professional with more than 20 years of experience in security, application development, and infrastructure solutions. Prior to joining Red Hat, Kirsten provided strategic direction for Black Duck’s open source security and governance solutions.

Managing Software Risk in a World of Exploding Vulnerabilities

PRESS RELEASE

WASHINGTON, Jan. 30, 2025 /PRNewswire/ -- DNSFilter announced today the release of its 2025 Annual Security Report, showcasing an uptick in malicious requests from 2023 to 2024. At internet scale, threats are relentless, but so is the right combination of intel and security strategy. Each malicious request blocked represents a real attack prevented, real harm avoided and real people and organizations protected.

The DNSFilter network processes about 170 billion DNS queries daily, 200 million of which are categorized as threats and blocked. These numbers represent phishing campaigns that never reached their targets, ransomware that never infiltrated networks and malware that never had the chance to spread.

Key findings from the report include:

*   One in every 174 requests is malicious: Most people make 5,000 DNS requests daily, which means a single person could encounter as many as 29 threat inquiries in a single day. This represents a significant increase from last year's findings, where one in every 1,000 queries was a threat.
    
*   AI domains increased by 15% but drove a significant amount of traffic: DNSFilter's researchers found that between October 2023 and September 2024, overall traffic to AI-related sites increased 786% and the number of individual AI domains rose 15%. This shows that a small number of domains account for a significant amount of traffic.
    
*   Phishing queries increased by 203%: While the relative distribution of threats remained consistent over time, phishing and malware inquiries increased (up 14%). This is in-line with a marked increase in ransomware seen across the industry.
    

Ken Carnesi, CEO and co-founder, DNSFilter, said: "Even as the threat landscape changes, our mission remains the same: to defend organizations and people from the worst of the internet. The data from our report makes it clear that DNS continues to be a critical threat gateway. It's also a reminder of the crucial role DNSFilter fills as a frontline defender on the DNS layer. We'll continue to improve, innovate and stand in the gap between our customers and the real-world harm bad actors are trying to cause."

About the company:

DNSFilter is redefining how organizations secure their largest threat vector: the Internet itself. DNSFilter is making the Internet safer and workplaces more productive by blocking threats at the DNS layer. DNSFilter resolves upwards of 170 billion daily queries. With 79% of attacks using Domain Name System (DNS), DNSFilter provides the world's fastest protective DNS powered by AI, blocking threats an average of 10 days faster than traditional threat feeds. Over 35 million users trust DNSFilter to protect them from phishing, malware, and advanced cyber threats.

You May Also Like

DNSFilter's Annual Security Report Reveals Worrisome Spike in Malicious DNS Requests

PRESS RELEASE

LONDON, UK – 30 January, 2025 – Cybersecurity leaders at large enterprises are planning to ramp up spending on cloud security in 2025 and want channel partners to help them maximise their return on investment, according to new research by Westcon-Comstor.

The global technology provider and specialist distributor surveyed 500 CISOs (Chief Information Security Officers) and other senior security executives at end-user organisations with at least 1,000 employees across five countries: France, Germany, Italy, UAE and the UK. 

Among the key findings is that 83% of security leaders intend to invest in Cloud-Native Application Protection Platform (CNAPP) and other cloud security technologies over the next 12 months. This propensity to invest was most pronounced in Italy and the UAE.

Across all geographic markets, the top three priority areas for investment are AI Security Posture Management (AI-SPM), Cloud Security Posture Management (CSPM) and Application Security Posture Management (ASPM).

This appetite to invest creates significant growth opportunities for channel partners in specific areas. 

The vast majority (95%) of those surveyed currently engage with channel partners, including resellers and managed security service providers, when procuring and deploying such solutions.

Training and enablement emerged as the most valued benefit from channel partners in the eyes of security leaders as they embrace CNAPP, with 40% singling this out as their main requirement – suggesting strong demand for knowledge-building support to maximise cloud security capabilities. 

A third (32%) highlighted cost-effective access to new solutions as the primary reason for engaging with channel partners, while 28% said what they value from partners above all else is assistance when navigating the cloud security market and identifying the best solutions. 

When asked about the main drivers of CNAPP adoption, security leaders emphasised the need to consolidate capabilities into a single unified platform, reducing the complexity and blind spots that come from using multiple cybersecurity vendors and tools.

Other motivating factors include the need to integrate security and compliance testing seamlessly, and a desire to unify risk visibility across cloud environments and the application development lifecycle.

Three quarters (75%) of security leaders agree on the need to adopt a DevSecOps approach to align with the ‘shift left’ trend that is seeing operational responsibilities shift toward developers and cloud architects.

Publication of the research comes as Westcon-Comstor announces its intention to establish a new X2C (everything to cloud) cybersecurity go-to-market in 2025, driving partner and vendor growth across the four pillars of code to cloud, infrastructure to cloud, data to cloud and identity to cloud.  

“CNAPP offers a holistic approach to securing cloud infrastructure, bringing together various security functions and capabilities in a single, unified platform to provide comprehensive security across the entire software development lifecycle, from code to cloud,” said Daniel Hurel, Senior Vice President, Westcon EMEA Cybersecurity & Next-Generation Solutions at Westcon-Comstor. “As the cloud security market continues to evolve, we’re seeing CNAPP become the go-to solution for securing cloud workloads. Our research suggests that this presents an opportunity for the IT channel, with particularly strong demand for training and enablement. Partners who establish themselves in this high-growth area stand to reap the rewards in 2025 and beyond.”

About Westcon-Comstor

Westcon-Comstor is a global technology provider and specialist distributor, operating in more than 50 countries. It delivers business value and opportunity by connecting the world’s leading IT vendors with a channel of technology resellers, systems integrators and service providers. It combines industry insight, technical know-how and more than 30 years of distribution experience to deliver value and accelerate vendor and partner business success. It goes to market through two lines of business: Westcon and Comstor.

EMEA CISOs Plan 2025 Cloud Security Investment

PRESS RELEASE

MONTREAL, January 29, 2025 (Newswire.com) - Flare, the global leader in Threat Exposure Management, has introduced Flare Academy, an educational hub featuring interactive online training offered free-of-charge to cybersecurity professionals.

Led by Flare's team of subject matter experts, Flare Academy offers a variety of online training modules on the latest cybersecurity threats to enhance the education and knowledge of cybersecurity practitioners. These sessions will highlight a variety of important topics, including investigation and intelligence gathering on cybercrime forums, techniques for deanonymizing threat actors, and ransomware analysis and infiltration.

Through interactive training sessions, security professionals can upgrade their skills and knowledge on a variety of cybercrime and cybersecurity topics, and earn CPE credits toward security certifications. And with the Flare Academy Discord Community, members can build relationships with other practitioners for continuous learning and engagement.

"Flare Academy will provide cybersecurity professionals with access to training, collaborative discussions, and cutting-edge resources," said Mathieu Lavoie, CTO & Research Team Lead at Flare. "We've launched this program with the goal of helping to empower security professionals with the training and information they need to stay ahead of threats and build a stronger, safer digital world."

A number of training modules are currently available for registration, including:

For more information about Flare Academy, and to register for upcoming modules, visit https://flare.io/trainings/.

About Flare

Flare is the leader in Threat Exposure Management, helping organizations of all sizes detect high-risk exposure found on the clear and dark web. Combining the industry's best cybercrime database with an incredibly intuitive user experience, Flare enables customers to reclaim the information advantage and get ahead of threat actors. For more information, visit https://flare.io.

You May Also Like

Interactive Online Training for Cybersecurity Professionals; Earn CPE Credits

Anthropic says its Constitutional Classifiers approach offers a practical way to make it harder for bad actors to try and coerce an AI model off its guardrails.

Source: Tada Images via Shutterstock

Researchers at Anthropic, the company behind the Claude AI assistant, have developed an approach they believe provides a practical, scalable method to make it harder for malicious actors to jailbreak or bypass the built-in safety mechanisms of a range of large language models (LLMs).

The approach employs a set of natural language rules — or a "constitution" — to create categories of permitted and disallowed content in an AI model's input and output, and then uses synthetic data to train the model to recognize and apply those content classifiers.

**"Constitutional Classifiers" Anti-Jailbreak Technique**

In a technical paper released this week, the Anthropic researchers said their so-called Constitutional Classifiers approach was as effective against universal jailbreaks, withstanding more than 3,000 hours of human red-teaming by some 183 white-hat hackers through the HackerOne bug bounty program.

"These Constitutional Classifiers are input and output classifiers trained on synthetically generated data that filter the overwhelming majority of jailbreaks with minimal over-refusals and without incurring a large compute overhead," the researchers said in an related blog post. They have established a demo website where anyone with experience jailbreaking an LLM can try out their system for the next week (Feb. 3 to Feb. 10).

Related:AI Malware Dressed Up as DeepSeek Packages Lurk in PyPi

In the context of generative AI (GenAI) models, a jailbreak is any prompt or set of prompts that causes the model to bypass its built-in content filters, safety mechanisms, and ethical constraints. They typically involve a researcher — or a bad actor — crafting specific input sequences, using linguistic tricks and even role-playing scenarios to trick an AI model into escaping its protective guardrails and spewing out potentially dangerous, malicious, and incorrect content.

The most recent example involves researchers at Wallarm extracting secrets from DeepSeek, the Chinese generative AI tool that recently upended long held notions of just how much compute power is required to power an LLM. Since ChatGPT exploded on the scene in November 2022, there have been multiple other examples including one where researchers used one LLM to jailbreak a second, another involving the repetitive use of certain words to get an LLM to spill its training data and another through doctored images and audio.

**Balancing Effectiveness With Efficiency**

In developing the Constitutional Classifiers system, the researchers wanted to ensure a high rate of effectiveness against jailbreaking attempts without drastically impacting the ability for people to extract legitimate information from an AI model. One simplistic example was ensuring the model could distinguish between a prompt asking for a list of common medications or for explaining the properties of household chemicals versus a request on where to acquire a restricted chemical or purifying it. The researchers also wanted to ensure minimal additional computing overhead when using the classifiers.

Related:DeepSeek Jailbreak Reveals Its Entire System Prompt

In tests, researchers had a jailbreak success rate of 86% on a version of Claude with no defensive classifiers, compared to 4.4% on one using a Constitutional Classifier. According to the researchers, using the classifier increased refusal rates by less than 1% and compute costs by nearly 24% compared to the unguarded model.

**LLM Jailbreaks: A Major Threat**

Jailbreaks have emerged as a major consideration when it comes to making GenAI models with sophisticated scientific capabilities widely available. The concern is that it gives even an unskilled actor the opportunity to "uplift" their skills to expert-level capabilities. This can become an especially big problem when it comes to trying to jailbreak LLMs into divulging dangerous chemical, biological, radiological, or nuclear (CBRN) information, the Anthropic researchers noted.

Related:Code-Scanning Tool's License at Heart of Security Breakup

Their work focused on how to augment an LLM with classifiers that monitor an AI model's inputs and outputs and blocks potentially harmful content. Instead of using hard-coded static filtering, they wanted something that would have a more sophisticated understanding of a model's guardrails and act as a real-time filter when generating responses or receiving inputs. "This simple approach is highly effective: in over 3,000 hours of human red teaming on a classifier-guarded system, we observed no successful universal jailbreaks in our target...domain," the researchers wrote. The red-team tests involved the bug bounty hunters trying to obtain answers from Claude AI to a set of harmful questions involving CBRN risks, using thousands of known jailbreaking hacks.

**About the Author**

Jai Vijayan is a seasoned technology reporter with over 20 years of experience in IT trade journalism. He was most recently a Senior Editor at Computerworld, where he covered information security and data privacy issues for the publication. Over the course of his 20-year career at Computerworld, Jai also covered a variety of other technology topics, including big data, Hadoop, Internet of Things, e-voting, and data analytics. Prior to Computerworld, Jai covered technology issues for The Economic Times in Bangalore, India. Jai has a Master's degree in Statistics and lives in Naperville, Ill.

'Constitutional Classifiers' Technique Mitigates GenAI Jailbreaks

Feeling creative? Submit your caption and our panel of experts will reward the winner with a $25 gift card.

Everyone's all about working in the cloud, but what's happening with these folks? What are they doing, and what do they want? Send us a cybersecurity-related caption to describe the above scene. Our favorite entry will win its wordsmith a $25 gift card.

Here are four convenient ways to submit your ideas before the Feb. 25, 2025, deadline:

*   Via social media: BlueSky (new!), Facebook, LinkedIn, and X.
    

If you win, we'll respond to you on the same platform, requesting your email address to send you the gift card.

**Last Month's Winner**

Shout out — and a $25 Amazon gift card — to Trey Rose for sending us the winning caption for January's "Greetings and Salutations" Edge cartoon. Some noteworthy contenders included "I guess that’s one way to prove you’re not a robot," "I always thought man-in-the-middle was something else," and "This has got to be your worst attempt at a Trojan Horse yet." A big thank you to all who participated.

**About the Author**

Cartoonist

John Klossner has been drawing technology cartoons for more than 15 years. His work regularly appears in Computerworld and Federal Computer Week. His illustrations and cartoons have also been published in The New Yorker, Barron's, and The Wall Street Journal.

Source

DARKReading