PII Scanner Buyer's Guide: Regex, AI, and Where Each One Fails

Every team that generates HTML eventually ships something it shouldn't have. Not a breach: a lead list with three phone numbers still in the footer, a proposal with a test API key in a script tag, a training page that quotes a customer email verbatim. The question that follows is always the same: what kind of PII scanner do we put in front of publishing, and what will it actually catch?

Dwight Brenner, who runs IT and security at Synergetics Worldwide, answers every scanner pitch with the same request: show me the false negatives. Feature lists all look alike; failure modes don't. This guide compares scanners by what they miss, then documents how HTMLvault's scanner runs in the web app and in a pipeline, and what to ask any vendor before you sign.

Two families of scanner, two blind spots

Sensitive data detection splits into two approaches, and they fail in opposite directions.

Deterministic pattern matching (regex). A rules engine looks for structured formats: email addresses, phone numbers, card numbers, Social Security and passport numbers, dates of birth, street addresses, API keys and tokens. If the data has a shape, this catches it, in milliseconds, with no model call and the same result for the same input every time. It is also auditable: you can show a reviewer the exact rule that fired.

What it cannot do is read. "Regional VP of Procurement, Cleveland, two kids, currently evaluating vendors" contains no pattern at all, and it identifies a person to anyone who works in that market.

Contextual AI scanning. A language model reads the document and reasons about what identifies a person. It catches the name-plus-role-plus-city problem and the paragraph that describes someone's medical situation without a single structured identifier. It also costs tokens per scan, takes seconds, and is non-deterministic: the same document scanned twice can return different findings. And it raises the question a security reviewer asks first: where does the content go, and under whose agreement is it processed?

Run both. Regex is the floor you never turn off; the AI layer earns its token cost where content is free-form, such as notes columns, written summaries and anything typed as prose.

One page, two scanners

Take a pipeline dashboard an AI tool built from a CRM export: about 900 lines of HTML with a chart, an account table and a short written summary. The regex pass returns one api_key (an analytics connection string the generator left in a JavaScript comment), 34 email, 11 phone, 41 person and 2 financial. Both financial hits are false positives, order references shaped like card numbers. The pass found the most serious item on the page in milliseconds and flagged two harmless strings, which is the price of reading shape instead of meaning.

A contextual pass on your own model key adds what no pattern matches: a summary line reading "hold the renewal quote until their layoffs are announced," a discount 15 points below list attached to a named account, and a column named churn_risk that is an internal judgment rendered as data. Neither layer alone clears the page.

Three places to scan: at rest, in transit, at publish

Most sensitive data discovery software answers one question: what is sitting in our systems that shouldn't be? That is inventory, run on a schedule and owned by governance. Stopping something on its way out is interception, run per artifact by whoever is about to share. Coverage comes from three surfaces, and none substitutes for another.

• At rest. Warehouse scanners, cloud storage classifiers and drive crawlers sample what is stored and classify it. They produce the map of which tables hold identifiers and which bucket still holds a CSV from 2019. Their weakness is latency: a nightly crawl tells you on Wednesday what went out on Tuesday.
• In transit. Email gateways, endpoint data loss prevention (DLP) agents and proxies inspect files crossing a boundary they recognize. HTML generated in a chat window and pasted into a web app looks to most of them like a person typing in a text box, with no attachment to open.
• At publish. The moment a person or a script turns HTML into a link someone else can open. The whole artifact and its author are present, and nothing has been distributed yet. Most organizations never instrumented this point, because until generation got cheap, documents went through repositories IT already scanned.

A four-person startup has no warehouse to crawl and no agent fleet, and it still ships pricing pages, investor updates and lead lists an assistant wrote an hour ago; for a team that size, the publish-time scan is the only checkpoint there is. For a large organization it covers a channel the other tools were never pointed at. Present it in a review that way, as an addition to DLP and classification rather than a replacement.

How HTMLvault's layered scanner works, and how to run it

The base layer is regex. It runs on every publish and every edit, on every plan including Free, costs zero tokens, and never sends your HTML to a model. It detects nine categories: ssn, financial, api_key, passport, address, person, dob, email and phone. The same check tests every outside URL in the page against Google's unsafe-URL lists. In the web app and through scan_html, findings arrive before the link exists; a publish the policy refuses returns its findings instead of a URL.

The scan covers the HTML you submit, markup included. HTMLvault drops HTML comments when it stores a page, but text inside a <script>, JavaScript comments included, is kept exactly as written.

The second layer is contextual and optional. On Teams and Enterprise you connect your own Anthropic, OpenAI or Google API key and run an AI scan on a link after it is published. Regex gates the publish; the AI scan reviews the published page, so run it before the URL goes to anyone and fix what it finds with an edit, which keeps the URL. Because the key is yours, contextual PII identification runs under the agreement you already have with that provider, the token spend lands in your own billing console, and HTMLvault pays for none of it. Connecting your own LLM for BYOK scanning covers setup.

Running a scan in the web app

1. Paste or upload HTML on /upload and click Create link. The button reads Scanning… while a sweep plays over the content box; afterwards the HTML box is read-only.
2. A clean result shows the green notice Scan complete: no sensitive data found. and the same click publishes.
3. Findings appear in a panel below the options row, under Warning: Sensitive data detected: one red pill per type with the scanner's full name and count, such as Email Address (34) or Phone Number (11).
4. View sensitive data opens the Sensitive data found modal.

The modal lists each remaining type with its count, then shows your HTML with every match highlighted: remaining matches in red, the current one in blue, anonymized replacements in green. Previous and Next step through matches with an N of M counter. Redaction works by type, not match by match: the Anonymize data menu has checkboxes for All, Name, Email, Phone, Card, SSN, Date of birth and Passport. Preview shows the pending replacements, such as [redacted]@example.com or ***-**-1234, and Save applies them.

For the dashboard above, checking Email and Phone turns 45 matches into placeholders in one action. The API key has no checkbox: delete it from the source and rotate it, because a masked secret is still a secret you shipped (how to scan HTML for API keys before sharing covers that workflow). Which findings to keep, mask or remove depends on what the recipient needs, and the PII detection and redaction guide covers those decisions. If you publish with findings still present, a large Create link button beside View sensitive data carries the caption *I acknowledge this is ok to publish, and the success screen shows a PII scan: N finding type(s) notice, or PII scan: no sensitive data detected for a clean page.

Running it from Claude or another MCP client

Most HTML that needs scanning comes out of an assistant. Through the HTMLvault MCP server (MCP, the Model Context Protocol, lets an assistant call tools directly), Claude can call scan_html to preview findings without publishing, then create_link once the page passes. Put that order in the project instructions so the model checks its own output every time. On a proposal built from a CRM summary and an AI-written project plan, a typical result is one api_key, three email, two phone, four person and one address. The key blocks outright. For each of the others, you decide whether the recipient needs it.

Gate programmatic publishing: pass, hold or block in your pipeline

Most sensitive HTML is now written by a tool, so a scanner that lives only in the browser misses most of the volume. The PII API question for any pipeline is what it does with each finding when no person is watching. A bare "clean" or "not clean" gives code nothing to route on. A useful finding names the category, where it sits and how often, with any preview masked. Automation platforms keep every step's input and output in run history, so a finding that repeats a card number copies it into a second system. It carries no confidence score: a regex matched or it didn't, and "0.97" on a pattern hit is invented precision.

Write the decision as a policy file, versioned with the pipeline, so an auditor can see which rules were live for any run:

policy: account-briefs v3
BLOCK  ssn, financial, passport, dob, api_key
HOLD   person, email or phone above 4 each; address above 2
PASS   everything else

Where the findings come from depends on how the pipeline connects:

• MCP. Call scan_html and apply the whole policy before anything is published. On a pass, call create_link, or create_links for up to 100 personalized pages from one template. A hold notifies a reviewer; a block stops and reports.
• REST. A REST API pipeline in Zapier, Clay or a script has no separate scan endpoint, because the publish call itself is the check. On Teams and Enterprise, mark your block categories as blocking in Publishing Rules and treat a refused publish, which returns findings instead of a URL, as the block branch. A hold is either a reviewer before the call or a returned URL that the send step keeps back until someone clears it.

An append-mode draft is not a scan step. A draft runs no checks until it is finalized, and update_link is refused on a draft, so "create a draft, read the findings, then publish" is not a sequence the API supports.

Two rules make a block hard rather than advisory. First, a blocked item never retries automatically: regenerating reads the same source record, and on the second attempt a model that writes the number out in words may slip past the pattern. Second, the fix happens upstream, in the source field or the columns the prompt receives, because the next page built from that record reads the same data. Give each pipeline its own API keys. That way a refusal traces to one integration, and you can revoke its key without breaking anything else.

Hold covers the cases a rule can't settle. A client name in a proposal is expected; the same name in a public case study may need sign-off. A token-shaped string in a code sample may be a deliberate fake or a live credential. Give the reviewer the category, the location and enough surrounding context to decide. Don't show them the full secret, because that just moves the exposure into the review step. Log the decision with the policy version in your own system. HTMLvault's PII audit records the scan event for each publish or edit, not your reviewer's reasoning.

PUBLISHING GATE

Agent or script writes HTMLClaude over MCP, Zapier, Clay or your own code

Regex check, zero tokensscan_html preview, or the publish call itself

Pass, hold or blockyour versioned policy; a block never retries

URL goes to the recipientafter a pass, or once a reviewer clears a hold

Tuning: false positives, block vs warn

A scanner nobody trusts gets ignored, and an ignored scanner is worse than none, because the security review still lists it as a control.

Handling false positives

Pattern matching over-fires in predictable places: demo data that looks like card numbers, a support address meant to be public, sample phone numbers in a template. Fix the input rather than fighting the scanner. Use obviously fake sample data (555 numbers, example.com addresses), and keep public contact details in a footer your team reuses instead of retyping them. The finding count on the success screen should be a number you can explain, not one you scroll past.

Block versus warn

Without a rule, findings show and the publisher confirms with the Create link button and its acknowledgment caption. On Teams and Enterprise, Publishing Rules decide what each category does, and the rules can differ for public and password-protected pages:

• Warn. An amber group, Acknowledgment required by your organization’s scan policy, lists the categories with counts. Create link stays disabled until the publisher clicks I acknowledge, enable publishing.
• Block. A red notice marked Blocked reads Your organization’s scan policy blocks this content, lists each category with its count, and gives a risk score with the instruction to redact the flagged items. Create link is disabled; View & redact sensitive data opens the modal.

Block the categories that have no legitimate place on a shared page: ssn, api_key, passport. Warn on email and phone, which sales content often carries on purpose. If a category is acceptable behind a password but not on a public page, use the public versus password-protected split. Blocking everything trains people to route around the tool, which is how shadow IT starts.

The unsafe-URL check also runs on every publish and edit, so a link that has gone bad since the last version gets checked again on the next save. Like any list, it only knows what Google has already flagged.

Seven questions for any discovery vendor

These apply to an enterprise discovery suite, lightweight PII software, or the scanner inside a sharing tool. A founder with no security team can read them as a self-assessment. Twenty minutes now is cheaper than answering them later in a customer's security questionnaire.

1. Pattern, contextual, or both? Ask which half of the problem the tool solves. Deterministic first and contextual second is the sound order: cheap and certain, then expensive and smart. HTMLvault runs regex on every publish and offers BYOK AI on Teams and Enterprise.
2. Does it read unstructured HTML? Demo it on a real generated page, not a CSV. Values hide in data- attributes, the JSON behind a chart, table columns styled out of view, alt text and script blocks. A tool that parses only visible text will pass a page that leaks in the markup.
3. Does it check the artifact at publish, or the store on a schedule? A crawl describes yesterday's exposure; a synchronous scan can stop the link from existing. Ask whether it can interrupt the share, and whether you can call it from where content is made, with an API key or an MCP tool rather than only an upload form.
4. How does it handle false positives? Ask whether findings block or advise, whether categories can be tuned per team or per page type, and what happens when someone dismisses one. Suppression with no record is how a policy erodes into a habit of clicking through.
5. Where does the content go, and who pays per scan? Local pattern matching keeps content inside a boundary you already approved. AI detection sends it to a model provider, and whose account, terms and retention apply then sets how long procurement takes. Per-document pricing teaches people to scan less, so the pattern layer should be free at the point of use.
6. Can a non-technical author act on a finding? The person fixing the page is usually a rep, a marketer, or a founder between calls. Look for plain category names, matches highlighted in context, and the next step on the same object: redact by type, rescan, publish with an expiry or a password. If fixing a finding means filing a ticket, people will route around the tool.
7. What evidence does it leave? Ask what each scan records, how long the record is kept and who can export it (see what an audit trail for shared content should hold). HTMLvault's PII audit records one scan event per publish or edit. It does not log access changes, expiries, deletions or retention changes, so say that in the review. Per-link analytics show who opened a link and when.

What no scanner reads, and precision vs recall

Some of the most expensive content on a shared page isn't PII. Think of a customer-specific discount restated in the template the assistant reused for the next three prospects, cap-table names in an investor update sent to "just a couple of advisors," or roadmap dates with the internal note about which ones are real. No PII category matches any of it. A contextual pass may flag some of it; a regex never will. The control is a share template that leaves it out. (The definition of PII is itself broader than most teams assume; what PII privacy actually means is a primer for stakeholders who haven't had to define it.)

Detection also fails at the edges of process. Teams scan for regulated categories only and miss operational secrets. They clean the visible body and leave hidden fields and URL parameters untouched. They trust test data whose phone number still rings a real person, who has now received three demo follow-ups.

Two measures describe how any scanner errs:

• Precision is the share of flags that are real. Patterns over-fire: a prospect's public headquarters trips address, an order number shaped like a phone number trips phone. That is why the example policy holds contact categories above a count instead of blocking on the first hit.
• Recall is the share of real PII that gets caught. Patterns miss unexpected formats: a card number split across two table cells, a birthday written as "the third of March," an SSN spelled out in words. The BYOK pass raises recall, but its output varies between runs, so route AI-only findings to review and keep automatic blocks for regex hits you can explain.

The remaining limits belong in any review:

• Regex covers nine categories and nothing else. A regional identifier outside them won't fire, and no regex can tell a live key from a revoked one.
• The scan covers the HTML you submit. If the page loads content from elsewhere when someone views it, that content was never scanned.
• A scanner reads the file, not the recipients. Every edit is re-scanned, but nothing in the scan limits who a link is forwarded to. That is the job of link expiry, a password and a retention window set on the same link.
• It only sees what passes through it. If half the team still emails attachments, that half is unscanned, so make the scanned path the easy one.

Why this matters for the approver

The IT or security lead reviewing a sharing tool is usually asked to approve a vibe: everyone likes it, it's fast, please sign. The part you can't describe is the part you get blamed for.

A layered scanner gives you something to write down instead. It has a deterministic pass on nine named categories that runs on every publish and edit at zero token cost, and an optional contextual pass under your own model agreement. Publishing Rules block what you can't defend and require acknowledgment for the rest, and the same check applies whether the HTML came from a browser or an agent. Then ask the requesting team for the run where it stops: the policy file and its version, one refused publish with its categories and counts, one held item and who cleared it, and the limits above with the AI-pass decision on paper. Approve against those four and your sign-off points at runs you can reopen and check yourself, not at a demo.