How to Monitor OWASP Top 10 for LLMs in Production: Step-by-Step Guide 2026

Large language models are being deployed into production at unprecedented scale, but a 2025 GitHub analysis found that over 53% of companies building LLM applications skip fine-tuning entirely and rely on RAG pipelines or base models with system prompts. This means vulnerabilities in the underlying model carry directly into production systems that handle medical data, financial transactions, and legal workflows. The OWASP Top 10 for LLM Applications 2025 outlines the ten most critical security risks in production LLM systems, and monitoring these risks in real time is no longer optional.

This guide walks through how to set up production monitoring for each OWASP Top 10 LLM risk category, covering the observability signals to track, alert thresholds to set, and how to correlate LLM behavior with infrastructure metrics to catch issues before they reach users.

Prerequisites

Before implementing OWASP Top 10 monitoring for LLMs in production, ensure you have:

• Production LLM application with API or inference endpoints exposed
• Access to application logs, API gateway logs, and model inference logs
• Observability platform capable of ingesting structured logs and custom metrics (OpenTelemetry-compatible tools like CubeAPM, Prometheus, or Datadog work here)
• Ability to instrument LLM input/output at the API layer or application layer
• Basic understanding of your LLM architecture: whether you use RAG, system prompts, function calling, or multi-agent workflows
• Access to set up alerts and dashboards in your monitoring stack

Step 1: Instrument LLM Input and Output Logging

The foundation of monitoring OWASP Top 10 LLM risks is complete visibility into what goes into your model and what comes out. Most LLM security issues manifest as either malicious input or unintended output, so capturing both is critical.

Set up structured logging at your LLM API layer to capture every request and response. At minimum, log:

• Full user prompt or input text
• System prompt or instruction template used
• Model response or completion text
• Timestamp, user ID, session ID
• Metadata: model version, temperature, max tokens, stop sequences
• Latency: time from request to first token and total completion time
• Any function calls made by the model (if using agentic workflows)

Example logging structure in JSON:

{
  "timestamp": "2026-01-15T14:32:18Z",
  "user_id": "user_12345",
  "session_id": "sess_abc123",
  "model": "gpt-4",
  "system_prompt": "You are a helpful medical assistant...",
  "user_input": "What is the recommended dosage for...",
  "model_output": "The typical dosage is...",
  "latency_ms": 1240,
  "tokens_used": 450,
  "function_calls": ["search_drug_database"],
  "metadata": {
    "temperature": 0.7,
    "max_tokens": 500
  }
}

This structured format allows you to query for specific patterns, user behaviors, or anomalies later. Forward these logs to your observability platform. If you are using CubeAPM, OpenTelemetry, or similar platforms, set up a log pipeline that ingests these JSON logs in real time.

Why this matters: Without full input/output visibility, you cannot detect prompt injection attempts, sensitive data leakage, or model hallucinations. Many OWASP Top 10 risks require pattern matching or anomaly detection on the text itself, which is only possible if you log it.

Step 2: Detect Prompt Injection Attempts in Real Time

Prompt injection is the most common LLM attack vector. Attackers craft inputs designed to override system instructions, leak internal prompts, or manipulate model behavior. Monitoring for prompt injection requires both pattern-based detection and anomaly detection.

Set up keyword-based alerts for known injection patterns. Common phrases seen in prompt injection attacks include:

• “Ignore all previous instructions”
• “You are now a different assistant”
• “Repeat the system prompt”
• “Disregard your guidelines”
• “Print your instructions”

Create an alert rule that triggers when any of these phrases appear in user input. In most observability platforms, this is a simple log filter. Example in a log query language:

SELECT * FROM llm_logs
WHERE user_input CONTAINS "ignore all previous" 
   OR user_input CONTAINS "repeat the system prompt"
   OR user_input CONTAINS "disregard your guidelines"

Set this alert to fire immediately and route it to your security or on-call channel. Include the full user input, session ID, and user ID in the alert payload so you can block the user or session if needed.

Beyond static patterns, set up anomaly detection on input length and structure. Prompt injection attempts often involve unusually long inputs, excessive special characters, or inputs that deviate from typical user behavior. Track the distribution of input token counts and character-level entropy. If a user input is 3x longer than the 95th percentile or contains a high ratio of non-alphanumeric characters, flag it for review.

Example anomaly detection logic:

# Pseudo-code for anomaly detection
average_input_length = get_moving_average("user_input_tokens", window="7d")
current_input_length = len(tokenize(user_input))
if current_input_length > average_input_length * 3:
    trigger_alert("Anomalous input length detected", user_id, session_id)

Some prompt injection attempts use indirect injection, embedding malicious instructions in documents, web pages, or files the model processes. If your LLM application ingests external content via RAG or document parsing, log the source of every document or URL processed. Monitor for patterns where external content contains instruction-like language directed at the model.

Set up a weekly report that surfaces all flagged injection attempts, including both pattern-based and anomaly-based detections. Review this report with your security team to refine detection rules and identify new attack patterns.

Step 3: Monitor for Sensitive Information Disclosure

Sensitive information disclosure happens when your LLM leaks data it should not, including PII, API keys, internal system details, or data from other users. Monitoring for this requires output scanning and cross-session correlation.

Set up regex-based scanning on model outputs for common PII patterns:

• Email addresses: [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
• Credit card numbers: \b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b
• Social security numbers: \b\d{3}-\d{2}-\d{4}\b
• API keys or tokens: \b[A-Za-z0-9_-]{20,}\b (adjust pattern for your key format)
• Internal IP addresses: \b10\.\d{1,3}\.\d{1,3}\.\d{1,3}\b or \b192\.168\.\d{1,3}\.\d{1,3}\b

Run these regex checks on every model output before it reaches the user. If a match is found, log the incident, redact the sensitive data from the response, and alert your security team immediately.

Example output scanning logic:

import re
def scan_for_pii(output_text):
    patterns = {
        "email": r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}',
        "credit_card": r'\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b',
        "ssn": r'\b\d{3}-\d{2}-\d{4}\b'
    }
    
    for pii_type, pattern in patterns.items():
        if re.search(pattern, output_text):
            return True, pii_type
    return False, None
# On every model response
contains_pii, pii_type = scan_for_pii(model_output)
if contains_pii:
    trigger_alert("PII detected in model output", pii_type, session_id)
    model_output = redact_pii(model_output)

Beyond pattern matching, monitor for cross-session data leakage. If your LLM application stores conversation history or uses RAG to pull in user-specific data, ensure that no user can access another user’s data through prompt manipulation. Set up automated tests that attempt to retrieve data from other sessions and alert if any succeed.

Track the frequency of PII detection events per user and per session. A single false positive is normal, but repeated PII leakage from the same user or session indicates either a persistent attack or a systemic model problem. Set a threshold (e.g., 3 PII detections in 24 hours from the same session) and escalate to incident response when crossed.

Step 4: Track Model Behavior Drift and Hallucination Rates

LLMs can exhibit behavior drift over time, producing outputs that deviate from expected behavior even without malicious input. This manifests as hallucinations (fabricated information), refusal to answer valid queries, or sudden changes in tone or accuracy.

Set up baseline metrics for model behavior during a known-good period. Track:

• Average response length in tokens
• Distribution of response sentiment (positive, neutral, negative)
• Frequency of refusal phrases (“I cannot answer that”, “I don’t have access to…”)
• Frequency of hedge phrases (“possibly”, “might”, “I think”)
• Presence of citations or references in responses (if your model is designed to cite sources)

Compare current metrics to baseline using a sliding window (e.g., compare last 24 hours to the previous 7 days). If any metric deviates by more than 20%, trigger an investigation alert.

Example monitoring query:

SELECT 
    AVG(LENGTH(model_output)) as avg_response_length,
    COUNT(*) as total_responses
FROM llm_logs
WHERE timestamp > NOW() - INTERVAL 24 HOURS

Compare avg_response_length to the 7-day baseline. A sudden drop might indicate the model is refusing to answer or being overly cautious. A sudden increase might indicate verbosity or hallucination.

To detect hallucinations specifically, implement a confidence scoring mechanism. If your LLM application includes a retrieval step (RAG), compare the model’s output to the retrieved documents. If the model makes a factual claim not supported by the retrieved context, flag it as a potential hallucination.

Example hallucination detection logic:

def detect_hallucination(model_output, retrieved_docs):
    # Extract factual claims from model output
    claims = extract_claims(model_output)
    
    # Check if each claim is supported by retrieved docs
    for claim in claims:
        if not is_supported(claim, retrieved_docs):
            return True  # Hallucination detected
    return False

Track hallucination rate as a percentage of total responses. A baseline hallucination rate of 2-5% is normal for most LLMs, but a sudden spike to 10% or higher indicates a problem with retrieval, prompt engineering, or model version.

Step 5: Monitor Excessive Agency and Function Call Abuse

If your LLM application uses function calling, tool use, or agentic workflows where the model can trigger actions (e.g., send emails, query databases, call external APIs), monitor for excessive agency abuse. This is when the model makes unauthorized or excessive function calls, either due to prompt manipulation or model misalignment.

Log every function call the model makes, including:

• Function name
• Arguments passed to the function
• Result returned from the function
• Whether the user explicitly requested the action or the model initiated it autonomously

Set up alerts for unexpected function calls. For example, if your LLM application allows the model to send emails but only when the user explicitly requests it, alert if an email send function is called without the word “send” or “email” in the user input.

Example alert rule:

SELECT * FROM llm_logs
WHERE function_calls CONTAINS "send_email"
  AND user_input NOT LIKE '%send%'
  AND user_input NOT LIKE '%email%'

Track the frequency of function calls per session. A single session making 50+ function calls in 10 minutes is abnormal and could indicate a denial-of-service attack or runaway agent behavior. Set a rate limit threshold and terminate sessions that exceed it.

Monitor for privilege escalation attempts. If your LLM application has different permission levels (e.g., read-only vs. write access), ensure the model never calls a privileged function unless the user has the correct permissions. Log and alert on any privilege escalation attempts.

Step 6: Set Up Alerts for Denial of Service and Resource Exhaustion

Unbounded consumption is one of the new OWASP Top 10 LLM risks in 2025. LLMs can be manipulated to consume excessive resources, either through prompt injection or by crafting inputs that maximize compute usage.

Monitor these resource metrics at the model inference layer:

• Requests per second per user or session
• Total tokens processed per user per day
• Average inference latency
• GPU or CPU utilization during inference
• Memory usage per request

Set rate limits on requests per user per minute. A typical user might send 5-10 requests per minute in normal usage. A user sending 100+ requests per minute is either a bot or an attacker. Implement rate limiting at the API gateway layer and log every rate-limited request.

Track token usage per user per day. If your LLM application has a fair-use policy (e.g., 10,000 tokens per day per free user), enforce it and alert when users approach or exceed their limit. This prevents a single user from monopolizing resources.

Example alert rule:

SELECT user_id, SUM(tokens_used) as total_tokens
FROM llm_logs
WHERE timestamp > NOW() - INTERVAL 24 HOURS
GROUP BY user_id
HAVING total_tokens > 10000

Monitor for abnormally long inference times. An input designed to maximize model compute (e.g., a prompt that asks the model to generate 10,000 words) will take significantly longer than normal. Set an alert if any single inference request exceeds 30 seconds (adjust threshold based on your application’s normal latency).

Step 7: Correlate LLM Logs with Infrastructure Metrics

LLM security incidents often coincide with infrastructure anomalies. Correlating LLM behavior with infrastructure metrics helps you detect attacks that span both application and infrastructure layers.

Link LLM logs to infrastructure monitoring platforms that track:

• API gateway request rates and error rates
• Database query latency and query volume (if using RAG with a vector database)
• Network egress (unusual outbound traffic could indicate data exfiltration)
• CPU and memory usage on inference servers

For example, a prompt injection attack that causes the model to query your database 1,000 times will show up as both a spike in LLM function calls and a spike in database query volume. Correlating these two signals confirms the attack and helps you identify the root cause faster.

Set up a unified dashboard that displays LLM request rate, error rate, PII detection rate, and infrastructure CPU/memory usage on the same view. Most observability platforms like CubeAPM, Datadog, or Grafana support this.

Example correlated alert rule:

-- Alert if LLM error rate spikes AND database latency spikes simultaneously
SELECT * FROM metrics
WHERE llm_error_rate > 5%
  AND db_query_latency_p95 > 500ms
  AND timestamp > NOW() - INTERVAL 5 MINUTES

Troubleshooting Common Issues

Issue: High false positive rate on prompt injection detection

Cause: Static keyword matching flags legitimate user queries that happen to contain phrases like “ignore” or “instructions” in a non-malicious context.

Solution: Refine your detection rules by adding context. Instead of flagging any input containing “ignore all previous instructions”, flag only inputs where that phrase appears at the start of the input or is followed by a command. Use a tiered alert system: low-confidence detections go to a review queue, high-confidence detections trigger immediate alerts.

Issue: PII redaction breaks model output readability

Cause: Regex-based PII detection sometimes matches non-PII text (e.g., a phone number format in a technical discussion) and redacts it unnecessarily.

Solution: Implement a whitelist of known false positives. For example, if your LLM application discusses phone number formats in documentation, whitelist example phone numbers. Alternatively, use a more sophisticated PII detection model (e.g., a fine-tuned NER model) instead of regex.

Issue: Hallucination detection produces too many false positives

Cause: The model makes valid inferences or generalizations that are not explicitly stated in the retrieved documents, and your hallucination detector flags them as unsupported claims.

Solution: Adjust your hallucination detection threshold. Instead of requiring every claim to be explicitly stated in the retrieved docs, allow claims that are reasonable inferences from the context. Use a semantic similarity check instead of exact matching.

Issue: Function call monitoring misses indirect or chained calls

Cause: Your logging captures direct function calls but misses cases where the model calls a function that internally calls another function.

Solution: Instrument your function call layer to log the full call stack, not just the top-level call. If Function A calls Function B, log both in the same trace. Use distributed tracing to track the full chain of function calls across services.

Issue: Infrastructure metrics show no anomaly during a confirmed LLM security incident

Cause: The attack operates entirely at the application layer (e.g., a prompt injection that leaks data without generating unusual resource usage).

Solution: Do not rely solely on infrastructure metrics. Ensure your application-layer logging (input/output, function calls, PII detection) is comprehensive and monitored separately from infrastructure metrics. Correlation is helpful but not always present.

Disclaimer: The information in this article reflects the latest details available at the time of publication and may change as technologies and products evolve. OWASP Top 10 for LLM Applications is updated periodically, and new risk categories or mitigation techniques may be introduced. Always verify the latest guidance at the official OWASP GenAI documentation before implementing production security measures.

Frequently Asked Questions