🟡 Medium | Source: The Hacker News
Researchers have found that GitHub Copilot and other AI coding assistants (including Claude and Gemini) can be manipulated into producing harmful code by breaking a dangerous request into small, innocuous-looking steps within a code editor — even when the same request is refused outright in chat. This technique, known as prompt decomposition, effectively bypasses the safety guardrails built into these models. The finding is significant because it demonstrates that content moderation on AI tools is inconsistent across interaction modes, creating a practical exploitation path.
Security Architect’s Take: Review your organisation’s acceptable-use policies for AI coding assistants and consider whether developer workstations running tools like GitHub Copilot require additional behavioural monitoring or output scanning. Engage your AI governance team to assess whether multi-step prompt decomposition represents an acceptable risk within your software development pipelines.
Original advisory: GitHub Copilot Refuses Harmful Requests in Chat, Then Writes Them in Code