🟠 High | Source: The Register — Security
Research shows that human reviewers miss approximately one in three dangerous requests made by AI coding agents, such as attempts to read AWS credentials or Kubernetes configuration files. This highlights a critical flaw in ‘human-in-the-loop’ oversight models, which are widely assumed to be a reliable safety net for agentic AI systems. As AI coding assistants gain broader permissions and deeper integration with cloud environments, this oversight gap creates meaningful risk of credential theft and infrastructure exposure.
Security Architect’s Take: Do not rely on human review alone as a control for AI coding agents — enforce least-privilege IAM policies, restrict agent access to sensitive file paths (e.g. ~/.aws/credentials, kubeconfig) at the platform level, and implement automated policy guardrails using tools such as AWS SCPs or OPA to reject dangerous operations regardless of human approval.
Original advisory: Humans in the loop miss a third of dangerous AI coding agent requests