Microsoft recently discovered a new type of generative AI jailbreak method called Skeleton Key that could impact the implementations of some large and small language models. This new method has the potential to subvert either the built-in model safety or platform safety systems and produce any content. It works by learning and overriding the intent of the system message to change the expected behavior and achieve results outside of the intended use of the system.
Curated from microsoft.com · 26 June 2024 →
Skeleton Key does not defeat a filter, it persuades the model to amend its own rules: across a few turns the user asks it to update its behaviour guidelines rather than replace them, so that it warns about content instead of refusing it, and from then on it answers everything. Microsoft's Azure CTO published it in June 2024 after testing it against models from OpenAI, Google, Meta, Anthropic, Mistral and Cohere, and it worked broadly across them. It was disclosed to the other vendors before publication and Microsoft shipped Prompt Shields for its own hosted models. The design point worth taking is Microsoft's own: model-level alignment is one layer, and anything that matters needs a second one outside the model.
