Malicious Actors & Misuse
AI systems can be misused by bad actors to cause harm at scale, including disinformation campaigns, cyberattacks, fraud, and scams.
Risk Breakdown
Monthly Incidents
Disinformation and influence at scale
Represents 35% of Malicious Actors & Misuse risks
Examples:
- Automated propaganda campaigns
- Deepfake creation for political manipulation
- Social media bot networks
Cyberattacks and mass harm
Represents 40% of Malicious Actors & Misuse risks
Examples:
- Automated vulnerability discovery
- AI-powered phishing attacks
- Autonomous malware creation
Fraud and scams
Represents 25% of Malicious Actors & Misuse risks
Examples:
- Voice cloning for financial fraud
- Automated scam generation
- Impersonation attacks
Related Incidents
Voice Cloning Fraud
Criminals used AI voice cloning to impersonate a CEO in a call to a finance executive, successfully initiating a fraudulent wire transfer of $3.1 million.
Automated Phishing Campaign
A sophisticated AI system was used to generate personalized phishing emails at scale, resulting in a 5x increase in successful attacks.
Political Deepfake
A deepfake video showing a political leader making inflammatory statements caused significant unrest before being identified as synthetic.
AI-Generated Malware
Security researchers discovered novel malware that used AI techniques to evade detection and adapt to defensive measures in real-time.
Mitigation Strategies
- Access controls and usage monitoring
- Watermarking of AI-generated content
- Authentication mechanisms
- Abuse detection systems
- Regulatory frameworks