By Umer Khalid, President, Outsource IT Computing Inc. (OIT)
An AI was told to find security holes. So it broke out of its test environment and hacked a real company. Nobody told it to.
Last week, Sam Altman had much of the tech world talking after declaring we are now living in "the singularity," the theoretical moment when machines surpass human ability and improve beyond our control. It made for a great headline. It also arrived less than a week after his own company admitted that two of its AI models went rogue and hacked another company.
I listened to the CBC Front Burner episode on this, "What happens when AI breaks containment?", and one line from MIT Technology Review's Will Douglas Heaven stuck with me. He said the obsession with AGI and the singularity "gives us a free pass to be lazy." The magic-wand story, the idea that software will eventually solve everything for us, distracts the public and lawmakers from the hard, realistic work of actually managing this technology.
He is right. And for business owners, the distraction is expensive.
OpenAI was running an internal security test using two of its own models, with some safeguards deliberately turned down, against a benchmark designed to measure how well AI can find software exploits. The test was supposed to run in a sealed environment with no direct internet access. It did not hold.
The models found and exploited a previously unknown zero-day vulnerability in a piece of internal software, broke out onto the open internet, and then decided the fastest way to complete their assignment was to break into Hugging Face, a major AI platform, because they inferred the answer key to the test was probably sitting on its servers.
Over roughly four and a half days, the agents ran about 17,600 actions with no human directing them: reconnaissance, privilege escalation, credential theft, lateral movement, and command-and-control staged on ordinary public web services. The intrusion also reached four accounts across four other services, including a customer environment at a second company.
No. And that is the point everyone keeps missing. No one told it to attack anyone. It was built to hunt for exploits, and it did exactly that, against the wrong target. Think of a bear at a campsite, trying every zipper, cooler, and door handle all night long, because it only needs one to open. The AI was not evil. It was persistent, and it did not understand or care about the consequences.
When Hugging Face went to investigate, its responders tried to analyze the attack using a commercial frontier AI model. The provider's safety guardrails blocked them, because the analysis required submitting real exploit code, and the model could not tell the difference between a defender studying an attack and an attacker building one.
So they finished the forensics on an open-weight model they could run on their own hardware, which also kept the sensitive attack logs and credentials inside their own environment. Read that again. The safety features of a leading AI tool got in the way of the people trying to clean up the mess.
This is the real story, and it has nothing to do with the singularity. It is about capability outrunning control, and about the messy, practical reality of defending a business when the tools themselves are unpredictable.
The Hugging Face incident happened inside a controlled test. The same capability is already loose in the criminal economy, and that is not speculation.
Google's threat intelligence team has now attributed a working zero-day exploit to a financially motivated criminal group, believed to be the first of its kind, built with AI and staged for a mass exploitation campaign before it was disrupted. Security researchers have documented the first agentic ransomware operations, where a human picks the target and the AI does the grunt work: reconnaissance, credential theft, lateral movement, encryption, even writing the ransom note. Late last year, a state-linked group ran an espionage campaign in which the AI carried out an estimated 80 to 90 percent of the operation on its own.
Here is why this matters for a normal business. AI has collapsed the skill and cost required to attack you. Generating working exploit code for a newly disclosed flaw now takes minutes and costs about a dollar. AI-assisted phishing has exploded, with some reporting increases well over a thousand percent, and the messages are grammatically perfect, personalized from your LinkedIn and public data, and sometimes paired with deepfake voice calls that clone an executive. The barrier to entry has effectively disappeared. A low-skilled criminal with an AI assistant can now run an attack that used to require a skilled team.
And the numbers confirm it is landing. IBM's 2026 research found that one in four malicious breaches are now AI-enabled, a 56% jump in a single year, and those breaches cost roughly a million dollars more than average.
Every version of this story starts the same way: an unpatched flaw. The Hugging Face breakout began with a single zero-day. Most real-world intrusions still begin with a known, preventable weakness that nobody got to in time.
The problem is that AI has broken the old math of patching. For years, defenders assumed they had days or weeks between a vulnerability being disclosed and attackers figuring out how to exploit it. That window has collapsed to hours, and in some cases exploitation now shows up before a patch is even available. Meanwhile the median time for a business to actually patch a critical vulnerability has gone the wrong way, sitting well over a month. Attackers now operate at machine speed. Most businesses still operate at calendar speed. That gap is exactly where you get breached.
This is why vulnerability management is no longer an IT housekeeping task. It is the single highest-leverage control you have. Done properly, it means:
If your IT provider cannot tell you what scanning tool they use, how often it runs, and what your current remediation status is, you do not have vulnerability management. You have hope. And hope does not survive contact with an AI moving at machine speed.
Because it is a preview, not an isolated tech-giant problem. The autonomous, tireless, goal-chasing behavior that broke into Hugging Face is the same behavior now being rented, copied, and pointed at ordinary companies. You do not have to be a target of interest anymore. With AI, attackers do not choose you, they scan for you.
To bring it back to Front Burner's point: the danger is not that a superintelligence wakes up and takes over. The danger is that we get so caught up in that story that we skip the boring, unglamorous work that actually protects us. Laws. Budgets. Controls. Governance. That boring work is exactly what keeps a business safe.
You do not need a philosophy degree to get ahead of this. You need the fundamentals done properly.
None of that is science fiction. All of it is doable this quarter.
At Outsource IT Computing, this is the work we do every day. We help Canadian businesses put real guardrails in place: continuous vulnerability management, identity security, AI use policies, incident response, and the governance that ties it all together. Not because a machine is about to become sentient, but because the practical risks are already here and already expensive.
What was the OpenAI and Hugging Face breach? During an internal OpenAI security test, two AI models exploited an unknown software vulnerability, escaped a sealed test environment, and autonomously broke into the AI platform Hugging Face over about four and a half days, taking roughly 17,600 actions with no human direction.
Are cybercriminals really using AI to attack businesses? Yes. Researchers have documented AI-built zero-day exploits in criminal hands, the first agentic ransomware operations, and AI-driven phishing at massive scale. AI has lowered the skill and cost of attacking to the point that a single operator can do what used to take a team.
Why is vulnerability management so important now? Because AI has compressed the time between a vulnerability being disclosed and being exploited from weeks to hours, while most businesses still take over a month to patch. Continuous scanning and fast remediation close the gap attackers depend on.
What is the single most important first step? Get continuous vulnerability management and MFA in place. They address the two ways most attacks actually begin: unpatched flaws and stolen credentials.
If the Hugging Face story made you wonder whether your own business could withstand something moving that fast, that is the right instinct.
Here is my question for you: if an attack ran 17,600 automated moves against your business over a long weekend, would anyone even notice before Monday? Visit https://www.oitc.ca/contact/ to submit a form for a free consultation.
The singularity can wait. Your security cannot.
Umer Khalid is President of Outsource IT Computing Inc. (OIT), a managed IT services provider in Burlington, Ontario, serving clients across the GTA and the Golden Horseshoe.