HiddenLayer emerges from stealth to protect AI models from attack
As AI-powered services like OpenAI's GPT-3 grow in popularity, they're becoming an increasingly attractive attack vector. Even protected behind an API, hackers can attempt to reverse engineer the models that underpin these services or use “conflicting” data to tamper with them. According to Gartner, 30% of all AI cyberattacks in 2022 will exploit these techniques along with data poisoning, which involves injecting bad data into the dataset used to train models to attack AI systems. .
As in any industry, fighting security threats is a never-ending task. But Chris Sestito says his platform, HiddenLayer, can simplify it for AI-as-a-service vendors by automatically identifying malicious activity against patterns and responding to attacks.
HiddenLayer came out of hiding today with $6 million in seed funding from Ten Eleven Ventures, Secure Octane and other investors. Sestito, the former director of threat research at Cylance and vice president of engineering at Qualys, co-founded the company several months ago with Tanner Burns and Jim Ballard. Burns and Ballard also worked at Qualys and Cylance and spent time together at BlackBerry, where Ballard was a data retention team leader and Burns was a threat researcher.
"Virtually every enterprise organization has contributed significant resources to machine learning to gain an advantage, whether that value comes in the form of product differentiation, revenue generation, cost savings, or efficiency," Sestito told TechCrunch in an email interview. "Adversarial machine learning attacks are capable of causing the same damage we've seen in traditional cyberattacks, including exposing customer data and destroying production systems. In fact, at HiddenLayer, we believe that we're not far from seeing machine learning models being sold to their organizations."
HiddenLayer claims its technology can defend models against attack without needing access to raw data or a vendor's algorithms. By analyzing model interactions – in other words, data fed into the model (e.g. an image of cats) and predictions generated by the model (e.g. the caption “cats”) – to spot patterns that could be malicious, HiddenLayer can work "non-invasively" and without prior knowledge of training data, Sestito said.
"Adversarial machine learning attacks aren't as loud as ransomware: you have to research them to catch them in time," Sestito said. "HiddenLayer has focused on a research-driven approach that will allow us to publish our findings and train the world to prepare."
Mike Cook, an artificial intelligence researcher who is part of the Knives and Paintbrushes collective, said it's unclear if HiddenLayer is doing anything "really groundbreaking or new." (Cook is not affiliated with HiddenLayer.) Still, he notes that there is one benefit to what HiddenLayer appears to be doing: trying to aggregate knowledge about attacks on AI and making it more widely available. /p>
"The AI boom is still going strong, but much of this knowledge about how modern machine learning works and how best to use it is still reserved for people who have knowledge. Memorable examples for me include researchers successfully extracting individual training data from OpenAI's GPT-2 and GPT-3 systems,” Cook told TechCrunch via email. and hard to get, sometimes all a business really needs is to provide convenient ways to access it."
HiddenLayer is currently in pre-revenue and has no customers, although Sestito claims the startup has engaged several "high profile" design partners. Ultimately, Cook believes its success will hinge less on HiddenLayer's technology and more on whether the threat of attacks is as great as the company claims.
"I don't know the prevalence of attacks against machine learning systems [currently]. Trickling a spam filter into letting an email through is very different in scale and severity than extracting proprietary data from a large language model,” Cook said.
For him, it is difficult to identify concrete examples of attacks against AI systems. Research on the topic has skyrocketed, with more than 1,500 articles on AI security published in 2019 on the scientific publishing site Arxiv.org, up from 56 in 2016, according to research by Adversara. But there are few public reports of attempts by hackers to, for example, attack businesses.
As AI-powered services like OpenAI's GPT-3 grow in popularity, they're becoming an increasingly attractive attack vector. Even protected behind an API, hackers can attempt to reverse engineer the models that underpin these services or use “conflicting” data to tamper with them. According to Gartner, 30% of all AI cyberattacks in 2022 will exploit these techniques along with data poisoning, which involves injecting bad data into the dataset used to train models to attack AI systems. .
As in any industry, fighting security threats is a never-ending task. But Chris Sestito says his platform, HiddenLayer, can simplify it for AI-as-a-service vendors by automatically identifying malicious activity against patterns and responding to attacks.
HiddenLayer came out of hiding today with $6 million in seed funding from Ten Eleven Ventures, Secure Octane and other investors. Sestito, the former director of threat research at Cylance and vice president of engineering at Qualys, co-founded the company several months ago with Tanner Burns and Jim Ballard. Burns and Ballard also worked at Qualys and Cylance and spent time together at BlackBerry, where Ballard was a data retention team leader and Burns was a threat researcher.
"Virtually every enterprise organization has contributed significant resources to machine learning to gain an advantage, whether that value comes in the form of product differentiation, revenue generation, cost savings, or efficiency," Sestito told TechCrunch in an email interview. "Adversarial machine learning attacks are capable of causing the same damage we've seen in traditional cyberattacks, including exposing customer data and destroying production systems. In fact, at HiddenLayer, we believe that we're not far from seeing machine learning models being sold to their organizations."
HiddenLayer claims its technology can defend models against attack without needing access to raw data or a vendor's algorithms. By analyzing model interactions – in other words, data fed into the model (e.g. an image of cats) and predictions generated by the model (e.g. the caption “cats”) – to spot patterns that could be malicious, HiddenLayer can work "non-invasively" and without prior knowledge of training data, Sestito said.
"Adversarial machine learning attacks aren't as loud as ransomware: you have to research them to catch them in time," Sestito said. "HiddenLayer has focused on a research-driven approach that will allow us to publish our findings and train the world to prepare."
Mike Cook, an artificial intelligence researcher who is part of the Knives and Paintbrushes collective, said it's unclear if HiddenLayer is doing anything "really groundbreaking or new." (Cook is not affiliated with HiddenLayer.) Still, he notes that there is one benefit to what HiddenLayer appears to be doing: trying to aggregate knowledge about attacks on AI and making it more widely available. /p>
"The AI boom is still going strong, but much of this knowledge about how modern machine learning works and how best to use it is still reserved for people who have knowledge. Memorable examples for me include researchers successfully extracting individual training data from OpenAI's GPT-2 and GPT-3 systems,” Cook told TechCrunch via email. and hard to get, sometimes all a business really needs is to provide convenient ways to access it."
HiddenLayer is currently in pre-revenue and has no customers, although Sestito claims the startup has engaged several "high profile" design partners. Ultimately, Cook believes its success will hinge less on HiddenLayer's technology and more on whether the threat of attacks is as great as the company claims.
"I don't know the prevalence of attacks against machine learning systems [currently]. Trickling a spam filter into letting an email through is very different in scale and severity than extracting proprietary data from a large language model,” Cook said.
For him, it is difficult to identify concrete examples of attacks against AI systems. Research on the topic has skyrocketed, with more than 1,500 articles on AI security published in 2019 on the scientific publishing site Arxiv.org, up from 56 in 2016, according to research by Adversara. But there are few public reports of attempts by hackers to, for example, attack businesses.
What's Your Reaction?
