LLM Data Leaks: secure LLM sensitive data exclusion with a Firewall

Everyone talks about LLM security but few actually build robust defenses. After shipping 20+ apps, I've seen how easily sensitive data can accidentally leak into prompts. Figured out the hard way that a proactive LLM input firewall is non-negotiable for proper secure LLM sensitive data exclusion.

The OpenAI Codex Incident & Why You Need LLM Data Redaction

Remember the OpenAI Codex issue? Developers accidentally feeding sensitive internal files into an LLM, then those files showing up in other users' suggestions. That wasn't a malicious hack; it was an "oops." And that "oops" can cost you big time – compliance fines, lost trust, ruined reputation.

Your AI agents — whether it's FarahGPT analyzing gold trends or a simple support chatbot — are constantly processing user inputs, system logs, retrieved context. Any of this can contain PII, internal project IDs (PROJ-9876), API keys (sk-XXXX), or confidential code snippets. Without a dedicated shield, this stuff goes straight to the LLM API. And once it's there, it’s out. Period.

This isn't just about preventing bad actors; it's about safeguarding against human error. A solid LLM input firewall provides critical AI agent privacy controls, ensuring your applications are protected.

Building Your Node.js LLM Input Firewall

Here's the thing — you need a gatekeeper before data touches any LLM, internal or external. I call it the SensitiveDataGuard. It's a middleware layer designed to surgically identify and redact sensitive information. It operates on two principles: pattern matching (regex) and context-aware filtering.

For our purposes, "semantic analysis" can be simplified into highly specific, context-driven regex patterns combined with keyword checks. You don't always need a whole separate local LLM for this; sometimes smart pattern matching does the job, especially for Node.js LLM input sanitization.

Here's what this firewall needs to block:

1. Internal Project IDs & Codes: NEXUS_ENV_PROD_123, TASK-ABC-456.
2. API Keys & Credentials: sk-, pk_test_, Bearer eyJ....
3. Personally Identifiable Information (PII): Emails, phone numbers, credit card numbers, national IDs.
4. Specific Code Snippets/Filenames: src/secrets.ts, console.log(process.env.DB_PASS).
5. Confidential Business Terms: Specific product names, client names, unreleased feature names.

This isn't just about privacy; it's about not polluting your LLM's context with irrelevant, sensitive junk that offers no value but carries all the risk.

The SensitiveDataGuard Node.js Implementation

My approach for secure LLM sensitive data exclusion involves a central SensitiveDataGuard class. It manages a configurable list of regex patterns and keywords. When an input string comes in, it iterates through these patterns, replacing any matches with a generic, yet informative, placeholder.

Here’s a blueprint.

First, define your patterns. I keep these in a separate configuration file, making updates easy without code changes.

// sensitivePatterns.ts
export const sensitivePatterns = [
  {
    type: 'PROJECT_ID',
    pattern: /(PROJ-[A-Z0-9]{4,}|TASK-[A-Z]{3}-\d{3,})/g,
    placeholder: '[REDACTED_PROJECT_ID]',
  },
  {
    type: 'API_KEY',
    pattern: /(sk-[a-zA-Z0-9]{32,}|pk_test_[a-zA-Z0-9]{24,}|Bearer [A-Za-z0-9\-_=]+\.[A-Za-z0-9\-_=]+\.?[A-Za-z0-9\-_=]+)/g,
    placeholder: '[REDACTED_API_KEY]',
  },
  {
    type: 'EMAIL',
    pattern: /[\w.-]+@[\w.-]+\.\w{2,4}/g,
    placeholder: '[REDACTED_EMAIL]',
  },
  {
    type: 'PHONE_NUMBER',
    pattern: /(\+?\d{1,3}[-. ]?)?\(?\d{3}\)?[-. ]?\d{3}[-. ]?\d{4}/g,
    placeholder: '[REDACTED_PHONE_NUMBER]',
  },
  {
    type: 'CREDIT_CARD',
    pattern: /\b(?:\d{4}[- ]){3}\d{4}\b/g, // Basic CC number pattern
    placeholder: '[REDACTED_CREDIT_CARD]',
  },
  {
    type: 'SECRET_FILE_PATH',
    pattern: /(src\/secrets\.ts|config\/env\.prod\.json)/g,
    placeholder: '[REDACTED_FILE_PATH]',
  },
  {
    type: 'INTERNAL_TERM',
    pattern: /\b(codename-atlas|project-fenix-alpha)\b/gi,
    placeholder: '[REDACTED_INTERNAL_TERM]',
  }
];

Next, the SensitiveDataGuard class itself:

// SensitiveDataGuard.ts
import { sensitivePatterns } from './sensitivePatterns';

interface SensitivePattern {
  type: string;
  pattern: RegExp;
  placeholder: string;
}

export class SensitiveDataGuard {
  private patterns: SensitivePattern[];

  constructor() {
    this.patterns = sensitivePatterns;
  }

  /**
   * Redacts sensitive information from a given text string.
   * @param text The input text to sanitize.
   * @returns The sanitized text.
   */
  public redact(text: string): string {
    let sanitizedText = text;
    for (const patternDef of this.patterns) {
      sanitizedText = sanitizedText.replace(patternDef.pattern, patternDef.placeholder);
    }
    return sanitizedText;
  }

  /**
   * Checks if a given text contains any sensitive information.
   * Useful for flagging or logging *before* redaction.
   * @param text The input text to check.
   * @returns An array of types of sensitive data found.
   */
  public check(text: string): string[] {
    const foundTypes: Set<string> = new Set();
    for (const patternDef of this.patterns) {
      if (patternDef.pattern.test(text)) {
        foundTypes.add(patternDef.type);
      }
    }
    return Array.from(foundTypes);
  }
}

Now, integrate this into your Node.js backend where you make LLM API calls. This is your Node.js LLM input sanitization in action.

// llmService.ts (Node.js backend)
import OpenAI from 'openai';
import { SensitiveDataGuard } from './SensitiveDataGuard'; // Assuming your paths are correct

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const sensitiveDataGuard = new SensitiveDataGuard();

async function callLLMWithGuardedInput(userPrompt: string, conversationHistory: string[]): Promise<string> {
  // Step 1: Combine all potential input sources
  const fullContext = [...conversationHistory, userPrompt].join('\n');

  // Step 2: Perform LLM data redaction BEFORE sending to the API
  const sanitizedContext = sensitiveDataGuard.redact(fullContext);

  // Optional: Log if sensitive data was found (for auditing, NOT the redacted data)
  const sensitiveTypesFound = sensitiveDataGuard.check(fullContext);
  if (sensitiveTypesFound.length > 0) {
    console.warn(`Sensitive data detected and redacted before LLM call: ${sensitiveTypesFound.join(', ')}`);
  }

  try {
    const chatCompletion = await openai.chat.completions.create({
      model: 'gpt-4o-mini', // Or whatever model you're using
      messages: [{ role: 'user', content: sanitizedContext }],
      temperature: 0.7,
    });
    return chatCompletion.choices[0].message.content || 'No response.';
  } catch (error) {
    console.error('Error calling OpenAI API:', error);
    throw new Error('Failed to get response from LLM.');
  }
}

// Example usage
async function main() {
  const userQuery = "Tell me about PROJ-8765. Also, what's my email? test@example.com";
  const chatHistory = ["User: My API key is sk-12345ABCDEF."];

  console.log('Original Query:', userQuery);
  console.log('Original History:', chatHistory);

  const response = await callLLMWithGuardedInput(userQuery, chatHistory);
  console.log('LLM Response:', response);
}

main();

This setup ensures that even if a user accidentally (or maliciously) tries to inject sensitive data, your firewall catches it. This is how you protect AI from sensitive data.

What I Got Wrong First

Honestly, relying solely on client-side obfuscation for sensitive data is amateur hour. I've seen devs try to implement redaction in the Flutter app itself, thinking "we'll just filter before sending the network request." That's a massive security hole. The client is completely untrustworthy. Any determined user can bypass client-side logic with a simple proxy or by modifying the app.

The real fix? A server-side LLM input firewall is non-negotiable. Your Node.js backend must be the gatekeeper. That's where you have control, logging, and auditability. Pushing this responsibility to the client is a recipe for disaster and will lead to compliance nightmares down the road. It’s not a question of if it gets bypassed, but when.

Another early mistake was using overly broad regex patterns. I once had a pattern for \d{5} (five digits) to catch zip codes, but it ended up redacting any five-digit number, including legitimate product codes or quantities. Specificity is key. Use lookaheads, lookbehinds, and anchor points (\b for word boundaries) to fine-tune your patterns. It's a constant refinement process, not a "set it and forget it" solution.

Optimization & Gotchas

• Pre-compile Regex: Notice how RegExp objects are created once in the sensitivePatterns array. Don't compile regex inside a loop or function that's called frequently; it's a performance hit. Pre-compiling saves CPU cycles.
• Context Preservation: When redacting, consider how much context you lose. Replacing PROJ-1234 with [REDACTED_PROJECT_ID] is better than [REDACTED], as it tells the LLM what kind of info was removed, potentially preserving some conversational flow.
• Dynamic Pattern Updates: For rapidly evolving sensitive data types (e.g., new internal project code formats), hardcoding patterns in your sensitivePatterns.ts isn't ideal long-term. Store these patterns in a database (like MongoDB or Supabase) or a dedicated config service. Your SensitiveDataGuard can then fetch and update its patterns dynamically without a full code redeploy. This is crucial for agility.
• False Positives/Negatives: This is the ongoing battle. Regularly review your agent's interactions and logs (ensuring your logs are also redacted!) to catch anything that slipped through or was over-redacted. Implement a feedback loop.
• Performance Impact: A very long list of complex regex patterns can introduce latency. For high-throughput systems, consider running redaction asynchronously or in a dedicated worker thread if the processing time becomes significant. My experience with these patterns on Node.js has shown minimal impact on average, but it's good to keep an eye on.

FAQs

Q: Can LLMs learn from redacted data?

A: No. If your LLM data redaction is properly implemented, the LLM receives only the placeholder (e.g., [REDACTED_EMAIL]) or gibberish. It cannot infer or "un-redact" the original sensitive information. The original data never reaches its processing layers.

Q: Is client-side LLM data redaction enough for security?

A: Absolutely not. Client-side filtering is easily bypassed by any user with basic technical skills. For true secure LLM sensitive data exclusion, you must implement a robust server-side Node.js LLM input sanitization layer. It's your last line of defense before data goes to the LLM.

Q: How do I handle new types of sensitive data that emerge?

A: Maintain a centralized and easily updatable list of regex patterns and keywords. Store them in a database or a configuration service, allowing your SensitiveDataGuard to fetch them dynamically. This way, you can deploy new AI agent privacy controls without needing to re-deploy your entire application.

Protecting your AI agents from accidental data leaks isn't glamorous, but it's fundamental. The OpenAI Codex issue was a wake-up call. Don't wait for your own "oops" moment. Implement a robust LLM input firewall in your backend today. It's not just about compliance; it's about building trust with your users and clients.

*Got sensitive data concerns for your AI app? Let's talk about building secure systems. Book a free consultation at [buildzn.com](https://buildzn.com).*