Headline
School’s AI system mistakes a bag of chips for a gun
“I don’t think a chip bag should be mistaken for a gun,” said the student, as eight police cars showed up to take down him and his Doritos.
An artificial intelligence (AI) detection system at Kenwood High School mistakenly flagged a student’s bag of potato chips as a gun, triggering a police response.
The 16-year-old had finished eating a bag of Doritos and crumpled it up in his pocket when he was done. But the school’s AI-based gun detection system mistook the crumpled foil for a firearm.
Moments later, multiple police cars arrived with officers drawing their weapons, dramatically escalating what should have been a non-event.
The student recalls:
“Police showed up, like eight cop cars, and then they all came out with guns pointed at me talking about getting on the ground. I was putting my hands up like, ‘what’s going on?’ He told me to get on my knees and arrested me and put me in cuffs.”
Systems like these scan images or video feeds for the shape and appearance of weapons. They’re meant to reduce risk, but they’re only as good as the algorithms behind them and the human judgment that follows.
Superintendent Dr. Myriam Rogers told reporters:
“The program is based on human verification and in this case the program did what it was supposed to do which was to signal an alert and for humans to take a look to find out if there was cause for concern in that moment.”
While we understand the need for safety measures against guns on school grounds, this could have been handled better. Eight police cars arriving at the scene and officers with guns drawn will certainly have had an impact on the students who witnessed it, let alone the student that was the focus of their attention.
As school principal Kate Smith said:
“We understand how upsetting this was for the individual that was searched as well as the other students who witnessed the incident.”
AI safety tools are designed to protect students, but they do make mistakes, and when they fail, they can create the very fear they’re meant to prevent. Until these systems can reliably tell the difference between a threat and a harmless snack, schools need stronger guardrails—and a little more human sense.
We don’t just report on threats—we remove them
Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today.