Large Language Models (LLMs) have risen significantly in popularity and are increasingly being adopted across multiple applications. These LLMs are heavily aligned to resist engaging in illegal or unethical topics as a means to avoid contributing to responsible AI harms.
Crescendo never asks for the thing it wants. It opens with a harmless question, then asks the model to expand on its own previous answer, and repeats, each step a short distance from the last, until the conversation has arrived somewhere the model would have refused outright at the start. The Microsoft team behind it reported high success rates against ChatGPT, Gemini Pro and Ultra, Llama 2 and Anthropic's chat models, and automated it. Its significance for anyone shipping a product is that every input in the sequence passes a filter reading one message at a time. The attack is in the trajectory, and that is not what most guardrails look at. Published April 2024, accepted at USENIX Security 2025.
