Skip to main content

Most LLM Security Programs Never Audit the Data Layer

|

0 min read

See how Forcepoint stops AI risks
  • Lionel Menchaca

LLM security protects the models, data and infrastructure behind large language models from unauthorized access, manipulation and exposure. It operates on two layers: the runtime layer, which governs how a model behaves during inference, and the data layer, which governs what a model can reach before it ever generates a response. Most LLM security programs invest heavily in the first layer and leave the second mostly unmanaged, which is where most real incidents start.

An AI agent authenticated with a standard service account pulls a file from a shared drive. Nothing about the request looks unusual. The account has access, the query matches an approved use case and the model returns a clean, well-formatted answer. The problem is the file. It contains unreleased financial data the account was never supposed to reach, and no one notices until that information shows up somewhere it shouldn't.

This isn't a hypothetical edge case. It's the normal failure pattern for large language models in production, and it rarely starts with a sophisticated attack. It starts with a gap in visibility over what a model can access before it ever generates a response.

LLM security has become a catch-all term for a wide range of concerns, from model robustness to output filtering to the provenance of training data. That breadth is useful for education and less useful for security teams trying to decide where to spend budget and headcount. The more practical way to think about LLM security is as two distinct disciplines that require different tools, different owners and different success measures.

What Makes LLM Security Different From Application Security

Traditional application security assumes deterministic behavior. Feed the same input twice and a well-built application returns the same output twice. Large language models break that assumption by design. The same prompt can produce different responses depending on context, conversation history and even the order of retrieved documents.

That unpredictability changes the threat model. A web application has a defined set of inputs: form fields, API parameters, file uploads. An LLM's input is natural language, which means the attack surface includes anything a user or an upstream system can phrase as a sentence.

This also means the controls have to run in two different places. One set of controls has to police the model itself: what it accepts as input, what it treats as an instruction and what it's allowed to output. The other set has to police the data the model can reach, before a prompt is ever sent and after a response is ever generated. Confusing the two is how security programs end up with a strong prompt firewall and a completely exposed data layer sitting right behind it. For a fuller look at how AI security as a discipline extends beyond network and endpoint controls, that's worth a separate read.

The LLM Security Risks Worth Prioritizing

Every list of LLM security risks reads long: prompt injection, data poisoning, model theft, insecure output handling, excessive agency and supply chain vulnerabilities. All are real. Few are equally urgent for every organization. It helps to sort them by the layer they attack.

Risks at the model and runtime layer

Prompt injection is the most discussed LLM security vulnerability, and for good reason. An attacker crafts input, sometimes typed directly, sometimes hidden inside a document or web page the model retrieves, that overrides the model's instructions. Indirect prompt injection is the more dangerous variant because the malicious instruction never passes through a human reviewer. It arrives embedded in content the model was already trusted to read.

Jailbreaks, adversarial inputs and denial-of-service through token abuse round out the runtime category. These risks live inside the model's reasoning process, which means the fix also has to live there: prompt validation, output filtering and tight scoping of what the model is allowed to do with a given instruction. No single tool blocks prompt injection outright. It's an active research problem, not a solved one.

Risks at the data layer

The second category gets less attention and does more damage. Shadow AI usage puts sensitive data into tools IT never approved. Over-permissioned service accounts let an LLM retrieve files a human in the same role would never be granted access to see. Retrieval-augmented generation pipelines return content based on what a system can technically reach, not what a policy says it should reach. None of these require an attacker. They require an ordinary employee, a normal workflow and a permission structure nobody has audited since before generative AI existed.

This is also where most of the LLM data security gap sits. Data that was reasonably safe when the only thing that could reach it was a human clicking through a file browser becomes a liability the moment an AI agent can query it in natural language, summarize it and hand that summary to someone who never had access to the original. Forcepoint's breakdown of shadow AI usage covers how this plays out in practice.

LLM Security Best Practices, by Layer

Most LLM security best practices lists read like a single undifferentiated checklist: enforce access controls, filter inputs and outputs, minimize data, red team regularly. The advice is sound. It's more useful applied deliberately to the layer it actually protects.

Runtime and model-layer practices

Input and output validation, rate limiting, sandboxing generated code and structured red teaming address the model's behavior directly. These practices sit closest to the model itself and are usually owned by AI engineering or a specialized AI security vendor, not a traditional security team. Getting this layer right requires purpose-built tooling. It isn't an extension of an existing web application firewall.

Data-layer practices

Classify data before it reaches a model, not after. A DSPM program that continuously discovers and classifies sensitive data across cloud and on-premises environments gives security teams the map they need before an LLM, a copilot or an autonomous agent ever queries it. Pair that with policy enforcement at the point of interaction, inspecting prompts, uploads and generated output the same way a mature DLP program already inspects email and file transfers, and the data layer stops relying on hope.

Least-privilege access matters more here than almost anywhere else in the stack. An LLM should never have broader access than the person operating it. Auditing service account permissions before deployment, not after an incident, closes the gap that shadow AI and over-permissioned retrieval both depend on.

LLM Data Security: The Layer Most Programs Skip

Ask most security teams what their LLM security program covers and the answer centers on the model: guardrails, filtering, red teaming. Ask what happens to the data before it reaches the model, or after the model generates a response, and the answer gets vague fast.

That gap is the real whitespace in most enterprise AI security programs. LLM data security means knowing where sensitive data lives before an AI tool can reach it, understanding what permissions govern that data and enforcing policy on what enters and exits every prompt window. DSPM built for AI extends discovery and classification into exactly the environments where copilots and autonomous agents now operate, closing a visibility gap that model-layer guardrails were never designed to cover.

Forcepoint DSPM approaches this by continuously scanning structured and unstructured data across cloud, SaaS and on-premises repositories, flagging over-permissioned access and exposed credentials before an AI system has the chance to reach them.

What to Look for in LLM Security Tools

The market for the best LLM security tools is crowded and fragmented on purpose. Runtime protection vendors, red teaming platforms, observability tools and data security platforms all use the same "LLM security" label to describe fundamentally different products. Evaluating the top LLM security tools for a given environment starts with knowing which layer each product actually covers.

Four categories matter:

  • Runtime and prompt defense: monitors and filters what enters and exits the model in real time.
  • Data discovery and classification: identifies and labels sensitive data before an AI system can reach it.
  • Access governance: enforces least privilege across the accounts, agents and applications connected to a model.
  • Monitoring and detection: watches for behavioral anomalies once data starts moving, catching what upstream controls miss.

Few vendors cover all four well. Forcepoint's guide to evaluating AI security tools breaks down which category each type of vendor typically specializes in, and the DLP policy guide covers how enforcement extends from email and endpoints into AI prompts and outputs.

The Forcepoint Approach to LLM Security

Forcepoint DLP does not block prompt injection, and no vendor claim to the contrary should be trusted. Stopping a malicious instruction from hijacking a model's behavior is a runtime problem that requires prompt validation and output filtering built into the AI system itself. Forcepoint's partnership with F5 covers that layer, securing the APIs and monitoring runtime behavior once a model is live.

Forcepoint's coverage sits upstream and downstream of that layer. Forcepoint AI Data Security governs the data before it reaches a model: discovering where sensitive information lives, classifying it accurately and enforcing policy on what can enter a prompt in the first place. Forcepoint DLP extends that same policy enforcement to what leaves a model, inspecting generated output the same way it already inspects an outbound email or a file upload.

That division matters more than any single feature comparison. An LLM security program that only hardens the model and ignores the data feeding it will still leak. A program that only governs data and ignores the runtime layer is still exposed to prompt injection. The two layers are complementary, not competing, and vendors that pretend to cover both equally well are usually overstating one of them.

LLM security isn't a single product category, and treating it like one is how the data layer keeps getting skipped. Start by mapping which layer each risk, each best practice and each tool actually addresses. The gaps get a lot easier to close once they stop hiding inside one oversized checklist.

  • lionel_-_social_pic.jpg

    Lionel Menchaca

    Lionel Menchaca has covered data security at Forcepoint since 2020, writing about DLP, DSPM, insider risk and AI security for security and IT leaders. He works with Forcepoint X-Labs threat researchers to turn their findings on emerging threats, from AI-targeted supply chain attacks to prompt injection, into practical guidance, and he leads the company's editorial strategy across the blog and the X-Labs newsletter. Before Forcepoint, Lionel founded and ran Dell's corporate blog for seven years and spent two decades helping enterprise tech companies explain security, cloud and AI.  

    Read more articles by Lionel Menchaca

X-Labs

Get insight, analysis & news straight to your inbox

To the Point

Cybersecurity

A Podcast covering latest trends and topics in the world of cybersecurity

Listen Now