GuardionAIGuardionAI
Book a demo
Back to Library

Table of Contents

Need help with security?

Our team can help you implement these controls today.

Runtime Application Security

Protecting the model infrastructure from theft, inversion, and DOS attacks.

Runtime Application Security: Infrastructure & Theft

While "Threats Through Use" focused on interaction (prompt injection), OWASP Runtime Application Security focuses on the system hosting the AI.

These threats target the confidentiality of the model itself and the integrity of the hosting infrastructure.

1. Model Theft (Extraction)

The Threat: An attacker recreates your proprietary model by querying it.

  • Method: The attacker sends thousands of inputs and records the outputs (and confidence scores). They use this dataset to train a "shadow model" that mimics yours.
  • Impact: Loss of IP and competitive advantage.

Controls:

  • Rate Limiting: Detect and block high-volume querying.
  • Obscure Confidence Scores: Don't return precise probabilities (e.g., return "High Confidence" instead of "99.823%").
  • Watermarking: Embed hidden patterns in model output to prove ownership of stolen models.

2. Model Inversion & Membership Inference

The Threat: Extracting training data from the model.

  • Membership Inference: Determining if a specific person (e.g., "John Doe") was in the training set (e.g., a hospital database).
  • Model Inversion: Reconstructing a face or document based on the model's outputs.

Controls:

  • Differential Privacy: Add noise during training so individual data points cannot be isolated.
  • Output Filtering: Prevent the model from outputting data that looks like PII or specific training samples.

3. Conventional Runtime Attacks

AI systems are software. They suffer from standard vulnerabilities.

  • Insecure Output Handling: An LLM generates code that contains an XSS payload, and the application renders it without sanitization.
  • Resource Exhaustion (DoS): Sending complex prompts that consume massive GPU cycles, causing service degradation (Denial of Wallet).

Controls:

  • Standard AppSec: Apply OWASP Top 10 (Web) controls to the AI application wrapper.
  • Compute Limits: Set strict timeouts and token limits per request.

CISO Takeaway

The Model is the Database. Treat your model weights like a database file. If an attacker can query it unlimited times, they can dump the schema (extraction) or the rows (inversion). Rate limit accordingly.

Continue to the next section: AI Security Testing

PreviousDevelopment-Time ThreatsNext AI Security Testing