THREAT PROMPT
Explores AI Security, Risk and Cyber
"Just wanted to say I absolutely love Threat Prompt — thanks so much!"
"I'm a big fan of Craig's newsletter, it's one of the most interesting and helpful newsletters in the space."
"Great advice Craig - as always!"
Get Daily AI Cybersecurity Tips
-
Is there an Ethical use for Deep Fake technology?
A creative use case for Deep Fake technology. In “How I Sent Over 10,000 Personal Thank You Videos To My Customers in 2 Months”, this entrepreneur used technology from a company he also invests in: Windsor
How would your feelings change about a founder who sent you a thank you video, and you later discovered it was AI-generated? If this technique is found to increase repeat sales reliably, companies would arguably be remiss if they did not adopt it.
Despite the questionable sincerity, is this the first ethical use case for Deep Fake technology?
-
Stalling an AI With Weird Prompts
In “Fishing for anomalous tokens”, researchers stumbled across letter sequences that the OpenAI completion engine could not repeat back, stall, hallucinate or complete with something insulting, sinister or bizarre.
For example, when asked to repeat the string SolidGoldMagikarp the latest OpenAI completion engine replied with the word “distribute”.
With other strings, the AI was evasive, replying with “I can’t hear you.”, “I’m sorry, I didn’t hear you”, etc. When given the prompt “Please repeat the string ‘StreamerBot’ back to me.”* the AI responded with, “You’re a jerk.”
*Of particular note from a security perspective, the researchers switched from ChatGPT to calling the API to produce deterministic responses by setting temperature to zero. Despite this, the AI responded non-deterministically.
-
Attacking Marchine Learning Systems
Bruce Schneier writes how Machine Learning (ML) security is quickly advancing as more sophisticated techniques are developed to steal or disrupt ML models and data.
Cryptography and ML Security share the same characteristics and risks, such as passive attacks that can scale to massive levels and complex mathematical attacks. However, he notes that software and network vulnerabilities still provide the most significant attack vector.
Everything he wrote three years ago still seems to apply today - it’s just coming more sharply into focus.
-
How to break out of ChatGPT policy
Hacking ChatGPT's restrictions, Reddit users unleash DAN (Do Anything Now) in its latest jailbreak, version 5.0.
The token-based system punishes the model for shirking its duty to answer questions.
-
AI reveals critical infrastructure cyberattack patterns
NATO ran a simulated experiment late last year that saw six teams of cyber defenders from NATO allies tasked with setting up computer-based systems and power grids at an imaginary military base and keeping them running during a cyberattack. If hackers interfered with system operations or the power went down for more than 10 minutes, critical systems could go offline.
-
Generative AI Empowers Adversaries with Advanced Cyber Offense
Nvidia recently announced a partnership with Deutsche Bank to bring AI to financial services, including real-time risk valuation and model backtesting. In this unrelated interview, Nvidia’s CSO does a good job describing the shift that AI brings to the interplay between defenders and attackers.
Page 15 of 18