AI Threat
HOME
Expanding Software Supply Chain Risks: The integration of AI-assisted coding tools and open source software has accelerated software development cycles but also increased operational risks, with threat actors actively targeting developers, AI coding assistants, and LLM security scanning tools.
Targeting Proprietary AI IP: GTIG observed increasing instances of adversaries targeting proprietary AI models, code, prompts, and research across sectors including healthcare, government, and media.
Shift Toward Agentic AI and Automation: Adversaries are deploying multi-agent frameworks that autonomously manage scanning pipelines, resolve operational errors, and execute credential harvesting at scale.
Multi-Stage Lifecycle Augmentation: State-sponsored and cyber crime groups continue to use AI capabilities as force multipliers across the attack lifecycle—from target reconnaissance and social engineering lure creation to custom malware obfuscation and post-exploitation troubleshooting. They are also experimenting with scaling information operations (IO) campaigns.
Illicit Account Procurement & LLMJacking: To circumvent access costs, adversaries are stealing developer credentials, purchasing compromised AI platform accounts, and hijacking enterprise cloud infrastructure to run unauthorized high-performance compute workloads.
Vulnerability Discovery and Exploit Generation: For the first time, GTIG has identified a threat actor using a zero-day exploit that we believe was developed with AI. The criminal threat actor planned to use it in a mass exploitation event but our proactive counter discovery may have prevented its use. Threat actors associated with the People’s Republic of China (PRC) and the Democratic People's Republic of Korea (DPRK) have also demonstrated significant interest in capitalizing on AI for vulnerability discovery.
AI-Augmented Development for Defense Evasion: AI-driven coding has accelerated the development of infrastructure suites and polymorphic malware by adversaries. These AI-enabled development cycles facilitate defense evasion by enabling the creation of obfuscation networks and the integration of AI-generated decoy logic in malware that we have linked to suspected Russia-nexus threat actors.
Autonomous Malware Operations: AI-enabled malware, such as PROMPTSPY, signal a shift toward autonomous attack orchestration, where models interpret system states to dynamically generate commands and manipulate victim environments. Our analysis of this malware reveals previously unreported capabilities and use cases for its integration with AI. This approach allows threat actors to offload operational tasks to AI for scaled and adaptive activity.
AI-Augmented Research and IO: Adversaries continue to leverage AI as a high speed research assistant for attack lifecycle support, while shifting toward agentic workflows to operationalize autonomous attack frameworks. In information operations (IO) campaigns, these tools facilitate the fabrication of digital consensus by generating synthetic media and deepfake content at scale, exemplified by the pro-Russia IO campaign “Operation Overload.”
Obfuscated LLM Access: Threat actors now pursue anonymized, premium tier access to models through professionalized middleware and automated registration pipelines to illicitly bypass usage limits. This infrastructure enables large scale misuse of services while subsidizing operations through trial abuse and programmatic account cycling.
Supply Chain Attacks: Adversaries like "TeamPCP" (aka UNC6780) have begun targeting AI environments and software dependencies as an initial access vector. These supply chain attacks result in multiple types of machine learning (ML)-focused risks outlined in the , namely Insecure Integrated Component (IIC) and Rogue Actions (RA). Our analysis of forensic data associated with these attacks reveals threats actors attempting to pivot from compromised AI software to broader network environments for initial access and to engage in disruptive activities, such as ransomware deployment and extortion.