Security Headers on Directory-Listed U.S. Local-Business Websites

By RACKCRUNCH Team · Scans 2026-09-24 · Also available as a PDF (security-headers-study-2026.pdf, 400 KB, SHA-256 ab7c9e3df139c6ff735c78271947a4ff3a1e7ead3acc762927177dacd42f7d56).

Why we looked

RACKCRUNCH has a free Security Header Check. You give it a URL, it makes one request, and it tells you which response headers are set, which are set badly, and which are missing.

Checking one site at a time made us curious about the bigger picture. We didn't mean banks or big tech. We meant the plumber, the law office, the car lot, the pizza place: businesses that may lack dedicated web-security staff.

We needed a list of those businesses, and there isn't a clean one. So we used the closest thing we could find: the public Curlie web directory, the human-edited successor to DMOZ. We drew 7,040 directory rows at random from its US local "Business and Economy" categories, as of the 2026-02-02 snapshot. They represent 7,022 unique initial registrable domains. Think of it as an SMB-oriented directory sample. We did not check how big any of these businesses are. Some will be larger than "small", and the directory tilts toward businesses established enough to get listed. When this report says "sites", it means these directory-listed sites, not every small business in the US.

On 2026-09-24 we ran two HTTPS request-chain scans of each sampled URL, following at most three redirects; all reported estimates come from the second scan. We read the response headers and never the pages themselves.

We expected low adoption. That is roughly what we got. The surprises came from how low, and from one number we had to take apart.

Which headers sites send

5,642 of the 7,040 sampled rows gave us a usable HTTPS response. 4,701 of those were HTTP-200 responses, and they came from 4,688 unique final registrable domains. Those 4,688 domains are our principal base (called "dedup-200" in the data files): one HTTP-200 response per final registrable domain, from the second scan. The definitions box at the top of this report spells out every base. Every figure in this report is on that base unless it says otherwise.¹

We started with the simplest question: which of seven study-defined explicit-header criteria does each site meet? These are criteria under our rubric, not a security grade, and a site can reasonably leave some of them out. Three need a word up front. A missing Referrer-Policy gets the browser's secure default, so we don't treat it as a failure; the point goes only to sites that set a secure value themselves. Permissions-Policy is still experimental and unevenly supported across browsers, and our criterion counts an explicit declaration, not protection gained. Cross-Origin-Opener-Policy depends on context: it matters most for sites that open pop-ups or handle cross-origin windows, and it can break sign-in and payment flows.

No explicit criterion in our rubric was near-universal.

HSTS was the most commonly observed header, present on 43.8% of sites. The most commonly passed criterion in our seven-item rubric was X-Content-Type-Options: nosniff, at 39.7%; only 12.3% met our stronger HSTS criterion of at least one year plus includeSubDomains. We call that study-defined strong HSTS, and it is measured on the final response host only. If a site redirects to www, includeSubDomains there covers hosts under www, not the bare domain or sibling hosts such as shop. Of the 578 strong-HSTS passes, 260 (45.0%) came from a www final host, 316 from the bare domain and 2 from another subdomain. Thus, for nearly half of strong-HSTS passes, the observed policy was on a www host and does not establish coverage of the bare domain. 31.2% send a clickjacking header with a recognized restrictive value under our parser. 8.0% send an explicit non-wildcard Permissions-Policy declaration, and 1.1% send a recognized non-default Cross-Origin-Opener-Policy value.

86.6% of sites don't send a Referrer-Policy at all, and as above, that is less bad than it sounds. When the header is missing, modern browsers fall back to strict-origin-when-cross-origin, which is a sensible default. The more concerning cases are explicit weak legacy values.

And one site in five gives something away. 20.5% [19.3-21.7] of sites matched our version-token rule: a digit in any of Server, X-Powered-By, X-AspNet-Version or X-AspNetMvc-Version, such as Apache 2.4.x, Microsoft-IIS/10.0 or nginx 1.x. For 27 sites, the only leak was an ASP.NET version header. That can aid version-targeted reconnaissance, but it is a low-severity finding: hiding a version number patches nothing.

Here is the full table on the principal base. Intervals quantify sampling uncertainty under the stated sample model, not frame bias, parser misclassification, CDN variation, or selective platform attribution.

The one invalid Referrer-Policy is a site that sends "Referrer-Policy: yes".

Put the seven criteria together and the headline number follows. Among the unique final registrable domains that returned HTTP-200, 49.7% met none of seven study-defined explicit-header criteria [95% CI 48.3-51.2], or 2,331 of 4,688.² This is an adoption measure, not an estimate of the proportion of insecure websites.

The same measures on every usable response, including error and bot-challenge pages, are in the sensitivity analyses at the end. Such responses may reflect CDN, WAF, hosting-platform, or application error-page configurations rather than the headers on the site's ordinary homepage, which is why they are not the principal base.

The CSP number that fell apart

In our first draft, the CSP result looked almost respectable. Roughly one site in six "passed". That seemed high. So in a revised analysis we stopped counting CSPs and started reading them, with a parser implementing the documented subset of CSP Level 3 described in the rubric.

About one site in five, 21.2% [20.0-22.4], sends an enforced CSP. That sounds fine until you look at what the policies say. Most are framing rules (frame-ancestors, which is anti-clickjacking, not anti-script) or upgrade-insecure-requests. The three most common exact policies were frame-ancestors 'self' (170 sites), the Shopify platform default block-all-mixed-content; frame-ancestors 'none'; upgrade-insecure-requests; (167), and bare upgrade-insecure-requests (130); together they account for 47.1% of the 992 sites sending an enforced Content-Security-Policy. None of the three has a script directive.³

CSP is an important defense-in-depth control against injected scripts. Here is how the policies break down by what they actually do for scripts.

What undoes the rest? A policy can fall into more than one row:

Characteristics of script-restricting CSPs - count and share of all 4,688 sites	Sites	Share
'unsafe-inline' effective for script elements	172	3.7%
'unsafe-inline' effective for inline event handlers (script-src-attr chain)	170	3.6%
'unsafe-inline' present but ignored (neutralized by a nonce or hash)	4	0.1%
'unsafe-eval' effective	157	3.3%
Full wildcard (bare * or scheme source) effective	81	1.7%
Subdomain wildcard (*.host) in effective script directive	55	1.2%
Passed our header-only script-CSP rule	8	0.17%

That leaves eight unique HTTP-200 domains, 0.17% [0.1-0.3], that passed our header-only script-CSP rule. This does not establish that the policy is unbypassable or that nonces are fresh and correctly applied in page markup. We did not test nonce freshness, whether nonces match the scripts in the page, or whether allowlisted script hosts can be abused, for example through JSONP endpoints. We did read all eight policies in full by hand to check the parser's reading of the header. None of the eight sends script-src-attr, so inline event handlers fall back to the same restricted script-src. Two are policies issued by a hosted site builder rather than written by the site owner. One allows no scripts at all. Two still allow subdomain-wildcard hosts as script sources, which our rubric reports separately and does not count against them.

Eight. Most observed CSPs consisted primarily of framing or mixed-content directives.

We got one thing wrong

A draft of this study had a nice moment in it: six sites out of 4,673 met all eight rubric criteria. We liked that line. It was wrong.

Our first rubric gave CSP frame-ancestors credit twice, once as clickjacking protection and again as a passing CSP. The second analysis pass fixed that. It also made the CSP check require a script-restricting policy that passes our header-only rule, and it stopped counting no-op values like COOP unsafe-none or a wildcard-only Permissions-Policy. Each of the six lost the double-counted point.

The first draft said six sites met all eight rubric criteria; the corrected rubric says one, and eight more met seven. The one got there on a CSP that uses strict-dynamic with a nonce, which our earlier parser misread.

A few of those nine sites share header patterns consistent with a reused configuration: one developer's or vendor's checklist applied to several sites. We are not naming any of them. Nine sites is too few to support a story, and headers change. Meeting all eight criteria also says nothing about the rest of a site: it can still have serious application flaws.

The explicit-header adoption index

Adding up the criteria gives a single number per site, which we call the explicit-header adoption index: the seven criteria above plus an eighth, version hygiene (no version in the software banner). It mixes unlike things, from broadly useful hardening to context-dependent controls and a low-severity banner check, all weighted equally. So treat it as a summary of adoption. The per-control table above is the main result.

The median is 1 and the mean is 1.80. The 0 row is smaller than the "none of seven" headline because many sites meet the eighth criterion, version hygiene, simply by not announcing their software. We haven't validated the index as a measure of how secure a site is.

Header-identified server and hosting labels

On the second scan we kept the full Server header, not just the version-leaking ones. Between Server tokens and platform-specific CSPs, we could attach a server or hosting label to 3,481 of the 4,701 HTTP-200 responses (74.0%). Our earlier labeling managed 15.0%. The biggest single group, 1,357 sites (28.9%), is one we call Cloudflare-edge: the Server header reads exactly "cloudflare". That tells you the response came through Cloudflare's network. It says nothing about the server behind it.

These numbers are descriptive. Labels only exist for servers that identify themselves, so the labeled group is selective, and nothing here tells you what would happen if a given business moved hosts. Some labels come from platform-specific CSPs, which are among the headers being measured, so the comparison is partly built from its own outcomes. Each response gets one label under fixed rules, and the first rule that matches wins, so no site carries two labels. Builder and platform signatures come first, because they name whoever controls the configuration. Server tokens come after. Matching is case-sensitive, as the header was sent. The order is GoDaddy builder, Flywheel, Pagely, Shopify, Wild Apricot, Cloudflare-edge, Microsoft IIS, Apache, nginx, OpenResty and Amazon S3. The exact rules are listed after the rubric below and in the data dictionary.

We made many comparisons here and in the sector section and applied no multiplicity correction. Differences between labels or categories may reflect platform composition, category assignment, response propensity or which servers label themselves, rather than anything about the businesses themselves.

Subgroup tables retain response rows because sector and state belong to the sampled listing; de-duplicating final destinations would require deciding which sampled listing's attributes to keep.

HTTP-200 base (n=4,701). Wild Apricot (6 sites) and Pagely (3) are too small to report. Intervals on the mean adoption index are omitted for readability. † Recomputed under the final rubric (X-Frame-Options duplicate normalization and COOP re-run); these cells moved from the earlier draft. Cloudflare-edge is a new row; it was unlabeled in earlier drafts.

Every Shopify-labeled site had a recognized clickjacking header and nosniff, and none met study-defined strong HSTS. Every GoDaddy-builder site met strong HSTS and had a recognized clickjacking header, and none had nosniff. Sites labeled Apache, nginx or IIS average between 0.58 and 1.80 on the adoption index, and between 49.8% and 72.1% of them meet none of the seven criteria. Cloudflare-edge sites average 1.72, and 59.6% meet none of the seven.

Policies that passed our header-only script-CSP rule are rare under every label. Of the eight, 3 are Cloudflare-edge, 2 nginx, 1 Apache and 1 IIS, and 1 has no label. Shopify, GoDaddy builder, Wix and Squarespace sites have none.

By kind of business

Directory categories give us a rough way to split the sample by type of business. Here are the nine categories with at least 80 HTTP-200 responses.

Sector (exploratory)	n	Mean adoption index (of 8)	None of 7	Strong HSTS	CSP enforced	Clickjacking (recognized value)	nosniff
Shopping	372	2.11†	35.5% [30.8-40.5]†	12.1% [9.2-15.8]	33.3% [28.7-38.3]	48.1% [43.1-53.2]†	56.2% [51.1-61.1]
Restaurants & Bars	408	2.07†	46.1% [41.3-50.9]	11.3% [8.6-14.7]	23.3% [19.4-27.6]	31.1% [26.8-35.8]†	45.6% [40.8-50.4]
General	1,620	1.82†	48.9% [46.5-51.3]†	12.8% [11.3-14.6]	21.5% [19.6-23.6]	32.5% [30.3-34.9]†	39.9% [37.6-42.3]
Computers & Internet	226	1.81†	58.0% [51.4-64.2]	15.5% [11.4-20.8]	20.8% [16.0-26.6]	30.5% [24.9-36.8]†	33.6% [27.8-40.0]
Automotive	175	1.73†	54.3% [46.9-61.5]	11.4% [7.5-17.0]	21.7% [16.2-28.4]	36.0% [29.3-43.3]†	32.0% [25.5-39.2]
Industrial	169	1.72†	50.3% [42.8-57.7]†	12.4% [8.3-18.2]	20.1% [14.8-26.8]	32.5% [25.9-39.9]†	37.3% [30.3-44.8]
Legal Services	194	1.69†	59.3% [52.2-65.9]	13.4% [9.3-18.9]	17.5% [12.8-23.5]	27.3% [21.5-34.0]†	32.0% [25.8-38.8]
Construction & Maintenance	368	1.67†	51.6% [46.5-56.7]†	10.3% [7.6-13.9]	14.9% [11.7-18.9]	25.3% [21.1-30.0]†	39.1% [34.3-44.2]
Real Estate	541	1.49†	56.2% [52.0-60.3]†	11.5% [9.0-14.4]	18.7% [15.6-22.2]	22.6% [19.2-26.3]†	32.2% [28.4-36.2]

HTTP-200 base (n=4,701). Categories under 80 responses are left out. Intervals on the mean adoption index are omitted for readability. † Recomputed under the final rubric; these cells moved from the earlier draft.

The same multiplicity caveat applies here. Shopping has the lowest observed none-of-seven share: 35.5% meet none of the seven criteria, against 56.2% for real estate. That is a descriptive, exploratory gap. Shopping also leads on recognized clickjacking headers (48.1%), nosniff (56.2%) and enforced CSP (33.3%). That fits the label picture above, since hosted shop builders tend to ship those headers. But this table can't tell you why.

Real estate has the lowest observed mean index (1.49 out of 8). It also has the lowest clickjacking rate (22.6%).

The CSP column needs the same warning as before. Enforced CSPs show up everywhere, but only eight policies passed our header-only script-CSP rule across the full principal base; sector-specific counts were too small to interpret.

A recognized non-default COOP value is too rare to compare by sector. The one exception is restaurants and bars, at 4.4% [2.8-6.9], and even there it looks like it comes from platform defaults.

The cobbler's children

Then there is the row we had been half-dreading.

Computers & Internet is the Curlie category whose listings are about technology. 58.0% [51.4-64.2] of sites listed there meet none of the seven criteria. That is the second-highest share of any category, behind legal services at 59.3%. Their strong-HSTS rate, 15.5%, is the highest in the table, but only by a little, and the intervals overlap.

Sites listed in Curlie's Computers & Internet category did not show higher header-criterion attainment. Category membership does not establish the business's services or staff expertise. The interval is also wide, and it overlaps most of the table.

By state

We cut the results by state. Nine states have at least 150 HTTP-200 responses, from 170 to 570 each, and the share meeting none of the seven criteria ranges from 42.9% [36.0-50.0] in New York to 56.2% [49.2-63.0] in Florida. State estimates varied descriptively. We did not prespecify or power the study for state comparisons, and we did not conduct multiplicity-adjusted state tests, so we do not rank states or infer state-level differences. A Pearson chi-square test of the 9-by-2 table (state by none-of-seven) found no detectable heterogeneity (chi-square 8.77, 8 degrees of freedom, p = 0.36; the per-state counts (state-counts-2026-09-24.tsv, 2 KB, SHA-256 f757ac8d93c5f3c51060202ce51393315f5cb1b8e65fb209d6da899b2d16b03f) are enough to rerun it), and the unadjusted New York-Florida difference does not survive a correction for the 36 possible pairs. The state-level counts behind this test are in the released analysis output.

The floor under everything

There is one more number, and it sits under all the others.

1,398 of the 7,040 sampled rows, 19.9% [18.9-20.8], gave us no usable HTTPS response at all. 737 failed at the TLS or connection stage. 215 redirected to plain HTTP. 204 timed out, and 192 had DNS that doesn't resolve. The rest were redirect loops, DNS errors and a handful of odd cases.

That is its own problem, about availability and TLS rather than headers. We can't say anything about the header posture of those sites, because we never got that far. What we can say is that for roughly one sampled row in five, our scanner did not obtain a usable HTTPS response under our scan rules.

What a site owner can do, in order

The fixes are mostly short. They are not all risk-free. Roll them out in stages and test as you go.

1. Deploy HSTS in stages. Begin with a short max-age, verify all affected services, then increase toward a long duration. Add includeSubDomains only after confirming that every current and anticipated subdomain supports HTTPS, because browsers will refuse plain-HTTP connections to any subdomain that doesn't. Consider preload only after meeting its requirements and accepting its operational consequences. Set the header on the bare domain as well as www.
2. Add clickjacking protection: X-Frame-Options: SAMEORIGIN, or CSP frame-ancestors 'self'. Check first that nothing legitimate embeds your pages (booking widgets, partner sites, your own apps). This will break them.
3. Add X-Content-Type-Options: nosniff. One line, and generally low-risk, but test MIME-dependent downloads and legacy behavior.
4. Check your Referrer-Policy. If it is missing, the browser default is reasonable. If it is set to a legacy value like no-referrer-when-downgrade, change it to strict-origin-when-cross-origin.
5. Patch your server software. Removing version numbers from Server, X-Powered-By and the ASP.NET version headers is a low-severity cleanup: it hides the problem and doesn't fix it. Keeping the software current matters more.
6. Start a Content-Security-Policy in report-only mode, then move toward a nonce- or hash-based strict CSP after testing. Strict-CSP guidance also recommends object-src 'none' and base-uri 'none', which our rubric does not require, so passing our rule is not the same as a production-quality strict CSP. The eight header sets that passed our rule do not establish implementation quality or nonce freshness.
7. Be careful with Cross-Origin-Opener-Policy. same-origin can break pop-up sign-in and payment flows. Test those before turning it on.

On hosted platforms, part of this is already done for you, and part isn't. Check which headers your platform leaves out.

How we measured

Population. Domains listed in US locality "Business and Economy" categories in the Curlie snapshot (the files in the curlie-rdf-all.tar.gz archive dated 2026-02-02, downloaded 2026-09-24 from https://curlie.org/download): 110,444 unique domains. Categories covering government, healthcare, education, financial services and military were excluded, as were listing directories and social platforms. Business size was not verified. Curlie is human-edited and runs dead-link cleanup, so the pool skews toward established businesses with maintained listings.

Sample. 7,040 rows drawn at random with a fixed seed. These cover 7,022 unique initial registrable domains; 18 rows shared an initial registrable domain with another row.

Scan. We ran two HTTPS request-chain scans of each sampled URL, following at most three redirects; all reported estimates come from the second scan. Method: HTTPS GET. The response body was never read; the connection was closed after the response headers. TLS verification was on, and only public IP addresses were contacted. Timeouts were 3 s for DNS, 3 s per hop and 8 s total, with 24 concurrent workers. The scanner identified itself as RACKCRUNCH-header-check/1.0 with a contact URL for opt-out. Scans ran from Google Cloud (Oregon, US) at 12:26-12:31 EDT (first) and 13:16-13:21 EDT (second) on 2026-09-24. For scan-two requests that produced a final HTTP response, the scanner retained the complete final-response header block internally (see Measurement ethics and data release).

Drift between scans. All 7,040 sampled URLs were scanned both times. Status transitions from the first scan to the second:

First scan -> second scan	Sampled rows
200 -> 200	4,608
200 -> not 200	65
not 200 -> 200	93
unusable -> unusable	1,348
403 -> 403	574
202 -> 202	229
404 -> 404	59
All other transitions (each under 20)	64

The first scan recorded only per-check values truncated at 400 characters, so a full comparison under the final rubric is not possible. Of the 7,040 sampled rows, 4,608 returned HTTP-200 in both scans. Under the original rubric, 13 of those rows (0.28%) changed at least one check, and the largest change in any headline prevalence between scans was 0.25 points (X-Content-Type-Options, all-usable base). Those are old-rubric diagnostics, not evidence that the corrected classifications are stable. As a bounded check, we applied the final rubric to the 4,484 of those rows (97.3%) whose scan-1 check values were complete. 7 rows changed at least one index item, 0.16% [0.08-0.32]: clickjacking 2, Permissions-Policy 3, X-Content-Type-Options 3, version hygiene 2 (some rows changed more than one). The 124 excluded rows are the ones with the longest CSPs, so this supports short-term stability but doesn't prove it.

Disposition. 5,642 usable HTTPS responses: status 200: 4,701; 403: 577; 202: 245; 404: 59; 500: 11; 307: 9; 429: 8; 401: 5; 503: 4; 405: 4; 520: 3; 526: 3; 400: 3; 406: 2; 521: 2; 502, 525, 410, 423, 523 and 530: 1 each. 1,398 unusable (each failure category is defined in the data dictionary): TLS or connect failure 737, redirect to non-HTTPS 215, timeout 204, DNS does not resolve 192, too many redirects 29, DNS lookup failed 13, non-public address 4, invalid redirect 4. Responses with HTTP status 202 (245 responses), often generated by bot-protection layers, were classified as reachable but excluded from the HTTP-200 analysis.

Duplicates and parked pages. Usable responses resolved to 5,614 unique final registrable domains; HTTP-200 responses to 4,688, the principal base. Among usable responses, only 15 final registrable domains were reached from more than one sampled row (43 responses in total, at most 9 for any one final registrable domain). Deduplication moves estimates by 0.3 points or less. 9 HTTP-200 responses (0.19%) landed on parked, expired-domain or platform pages. These were identified from the final hostname after redirects (for example expireddomains.com or forsale.godaddy.com), not from page content. They stay in the base because their headers are what the scanner received.

Intervals. Wilson 95% confidence intervals throughout (Wilson, 1927). The random draw was of directory rows, and the principal base is unique final registrable domains after redirects, so the intervals on that base are approximate. They quantify sampling uncertainty under the stated sample model, not frame bias, parser misclassification, CDN variation, or selective platform attribution. Label, sector and state cuts are exploratory.

Rubric

The eight criteria of the explicit-header adoption index, as used in this report. The full rubric code is in the reproduction pack (repro-pack-2026-09-24-v2.4.tar.gz, 239 KB, SHA-256 661a5a4197a072392bf0b35424d4795032c4c8ddec1571910f2e5624d44062e8), and every accepted, weak, malformed and no-op value is defined in the data dictionary (data-dictionary.md, 7 KB, SHA-256 16a7204d7b8cb7573136d299e8172bb38c0886f96c148aaa03b11cdd990414a1).

The "none of seven" headline uses the first seven items and leaves out version hygiene.

Hosting labels. Rules are checked in this order and the first match wins. Matching is case-sensitive.

1. GoDaddy builder: CSP contains "godaddy.com", or Server starts with "DPS"
2. Flywheel: Server starts with "Flywheel"
3. Pagely: Server starts with "Pagely"
4. Shopify: CSP starts exactly with "block-all-mixed-content; frame-ancestors 'none'; upgrade-insecure-requests"
5. Wild Apricot: CSP contains "wildapricot"
6. Cloudflare-edge: Server is exactly "cloudflare"
7. Microsoft IIS: Server starts with "Microsoft-IIS"
8. Apache: Server starts with "Apache"
9. nginx: Server starts with "nginx"
10. OpenResty: Server starts with "openresty"
11. Amazon S3: Server starts with "AmazonS3"

Anything else gets no label.

Caveats

• One page, one moment. Headers can differ by page, CDN edge or login state. We read the home page only.
• Sites behind a CDN or hosted platform often show the platform's headers, not the owner's configuration. That is real protection for the site, and it also means some passes are inherited.
• We did not test the http:// to https:// redirect. Our checker does, but these scans followed one request chain per site.
• The sample comes from a human-edited directory. It skews toward businesses established enough to be listed, and business size was not verified.
• Labels cover the 74.0% of HTTP-200 responses that identify a server or hosting platform in their headers. The comparisons describe those sites. They don't estimate what a platform causes.
• The clickjacking figure includes 116 sites credited because we treat identical repeated X-Frame-Options values, joined by our scanner, as one value (see Rubric). Under the current HTML processing model, identical repeated values such as SAMEORIGIN, SAMEORIGIN retain same-origin framing protection; older browser implementations have differed. Counting those as ineffective would give 1,347 (28.7%).
• This is a light scan, not a security audit of any individual business.

Sensitivity analyses

All usable responses. Every usable HTTPS response, including error and bot-challenge pages (n=5,642). On this base, 48.4% [47.1-49.7] meet none of the seven criteria (2,733 of 5,642).

The COOP and CSP figures here need a caveat. Most of the 352 same-origin COOP values come from CDN challenge pages, and so do the CSPs: 325 of the usable responses (5.8%) carry nonce/hash-strict CSPs, but these are mostly CDN bot-challenge interstitials (Cloudflare challenge pages carry nonce CSPs), not deployments by the sites; 316 of the 318 script-src-attr uses on this base sit on those Cloudflare responses. On the principal base that class is 5 sites. The rest of the mix: no CSP 4,301 (76.2%), no script directive 829 (14.7%), allowlist-based 177 (3.1%), strict-dynamic with anchor 8 (0.1%), invalid 2.

Base comparison. The same measures across bases:

COOP drops from 6.6% to 1.1% on the HTTP-200 base, which excludes most identified challenge responses, and Referrer-Policy and Permissions-Policy drop by about a third. Deduplication barely moves anything. The none-of-seven share is about half on all three bases.

Measurement ethics and data release

We scanned only public home pages over HTTPS, one request chain per sampled URL per scan, following at most three redirects. We never read page bodies, submitted forms, logged in or probed for vulnerabilities. The scanner contacted only public IP addresses, ran at most 24 concurrent connections across the whole sample with short timeouts, and identified itself as RACKCRUNCH-header-check/1.0 with a contact URL for opt-out. We have received no opt-out requests to date.

For every scan-two request that produced a final HTTP response, the scanner retained the complete final-response header block internally. The only limit was a cap of 65,536 bytes on the whole block, applied at capture and before any classification; the largest captured block was 15,362 bytes, so no scan-two final-response header block was truncated by the capture-size cap. Headers from redirect hops were not kept. The first scan kept only per-check values, cut at 400 characters after classification, which is why drift between scans can only be bounded. The released normalized values are the complete graded header fields, so every classification can be reproduced from them, and nonce redaction does not change any classification, including the eight CSPs that passed our rule. The scans were unauthenticated GET requests with no cookies sent, so the data holds nothing beyond what any visitor's browser would receive.

The released dataset (scan2-2026-09-24-deidentified.jsonl, 4.1 MB, SHA-256 329a1a910b4ae4b005fdd864fe9b5370df7e30bf01fc14b790baf7664d00428d) has exactly one row for each of the 7,040 sampled rows, and nothing more. Rows without a usable response carry only their error class and HTTP status, with no header fields. Each row lists the sector, state, whether the response was usable, the error class, HTTP status, number of redirects, scan time, the normalized values of the graded headers (HSTS, enforced and report-only CSP, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, COOP, and Server, X-Powered-By, X-AspNet-Version and X-AspNetMvc-Version with their version tokens, which are part of the measurement) and the per-criterion results. There is no raw header map. Before release we drop Set-Cookie headers entirely, replace CSP nonce values with a placeholder ('nonce-<redacted>') while keeping hashes, and strip query strings from CSP reporting endpoints.

The dataset is de-identified. Every business appears only under a stable pseudonymous ID, and neither this report nor the dataset names any site as having weak or missing headers. We chose this level because the point of the study is the overall picture, and naming small businesses with weak headers would add risk for them without making that picture any clearer. If you think your business might be in the sample, run your own domain through the free RACKCRUNCH Security Header Check (rackcrunch.com/security-headers) to see what the scanner saw. We will confirm a specific domain's row only to a verified owner of that domain.

Technical sources

• Content Security Policy Level 3 (W3C): https://www.w3.org/TR/CSP3/
• Strict CSP guidance: https://csp.withgoogle.com/docs/strict-csp.html and https://web.dev/articles/strict-csp
• Weichselbaum et al., "CSP Is Dead, Long Live CSP! On the Insecurity of Whitelists and the Future of Content Security Policy", ACM CCS 2016: https://doi.org/10.1145/2976749.2978363
• MDN, Content-Security-Policy:
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Content-Security-Policy
• MDN, script-src-attr:
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Content-Security-Policy/script-src-attr
• MDN, Strict-Transport-Security:
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Strict-Transport-Security
• MDN, Referrer-Policy:
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Referrer-Policy
• MDN, Permissions-Policy:
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Permissions-Policy
• Permissions Policy (W3C): https://www.w3.org/TR/permissions-policy/
• MDN, Cross-Origin-Opener-Policy:
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cross-Origin-Opener-Policy
• MDN, X-Frame-Options:
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Frame-Options
• OWASP HTTP Headers Cheat Sheet:
  https://cheatsheetseries.owasp.org/cheatsheets/HTTP_Headers_Cheat_Sheet.html
• OWASP Content Security Policy Cheat Sheet:
  https://cheatsheetseries.owasp.org/cheatsheets/Content_Security_Policy_Cheat_Sheet.html
• Curlie directory license: https://curlie.org/docs/en/license.html
• Wilson, E. B. (1927). "Probable Inference, the Law of Succession, and Statistical Inference." Journal of the American Statistical Association 22(158): 209-212. https://doi.org/10.1080/01621459.1927.10502953

Data and reproducibility

Everything behind this report is published. The de-identified dataset has one row for each of the 7,040 sampled homepages, and the data dictionary explains every field and value in it. The reproduction pack holds the dataset, the sampling, scan and analysis scripts with the random seed, the rubric and CSP parser, and the tests; its README (README-2026-09-24-v2.4.md, 7 KB, SHA-256 3b8d21ce12146aac73067b3ce296bd03730af760da2df13f64f495a778b23884) walks through each step and lists what cannot be rerun. The same files are available as a git bundle (repro-pack-2026-09-24-v2.4.bundle, 238 KB, SHA-256 1cf5c95c54bda2d09a2847aacccd78e67886a2fa2edb833fee6c1a0b0828fd1b), tagged release-2026-09-24-4 (commit 795d9eca5b46a49af1277a0dedfb8fc9b8bb43f4).

One command regenerates every table in this report as final-tables.md (final-tables.md, 20 KB, SHA-256 3717c4afcf59751f6ccf0a12d93d84f762dade46848e8f105ed29b953a69cf59):

pip install -r requirements.txt && python3 reanalysis4.py data/ final-tables.md

The only dependency outside the Python standard library is pinned in requirements.txt (requirements.txt, 1 KB, SHA-256 7d5f5a94ac1690db632a2f6b2e51aaf74a8ba332ee248aa6299ec6e713120946). The two charts are drawn from those tables: the adoption-index chart from the distribution table in section 3, and the label chart from the platform table in section 13. There is no separate chart code. The parser tests, including the golden corpus, run with:

python3 test_csp3.py && python3 -m unittest test_golden

Software versions used: Python 3.10.12; scipy 1.15.3, pinned in requirements.txt (used only for the state chi-square; everything else is the Python standard library). Rerunning the scan itself is not part of the command: it needs live network access and the directory snapshot, and results change over time.

To check a download, compare its SHA-256 (shown in small print after each link) with sha256sum FILE on Linux or shasum -a 256 FILE on macOS. Inside the pack, MANIFEST.sha256 (MANIFEST.sha256, 1 KB, SHA-256 05e771765efd82aa3b5a682d88c31f7491f0fad03f11cdd4e3b8211c412f8994) lists the hash of every file; check them all with sha256sum -c MANIFEST.sha256.

Files inside the reproduction pack

• .gitignore 1 KB, SHA-256 862263fa1f46c20f0d1e4dac5ffcc75abd55c08211b2c3864c5f8764b9d87793
• MANIFEST.sha256 1 KB, SHA-256 05e771765efd82aa3b5a682d88c31f7491f0fad03f11cdd4e3b8211c412f8994
• README.md 7 KB, SHA-256 3b8d21ce12146aac73067b3ce296bd03730af760da2df13f64f495a778b23884
• csp3.py 10 KB, SHA-256 253713b8ec414ce9d93e6e180254d67c3409ffae5e94d5675315ff0e52eadb1f
• data-dictionary.md 7 KB, SHA-256 16a7204d7b8cb7573136d299e8172bb38c0886f96c148aaa03b11cdd990414a1
• data/scan2-2026-09-24-deidentified.jsonl 4.1 MB, SHA-256 329a1a910b4ae4b005fdd864fe9b5370df7e30bf01fc14b790baf7664d00428d
• data/state-counts-2026-09-24.tsv 2 KB, SHA-256 f757ac8d93c5f3c51060202ce51393315f5cb1b8e65fb209d6da899b2d16b03f
• final-tables.md 20 KB, SHA-256 3717c4afcf59751f6ccf0a12d93d84f762dade46848e8f105ed29b953a69cf59
• reanalysis4.py 28 KB, SHA-256 7326fc8742026970e4a2f8f80f3795d7a6b0825456b5087f888422eeac06e46e
• requirements.txt 1 KB, SHA-256 7d5f5a94ac1690db632a2f6b2e51aaf74a8ba332ee248aa6299ec6e713120946
• rubric3.py 7 KB, SHA-256 3bc61fc02e75e45a96147907cfc604579c461bc74dadc6e597b13823cf025517
• scripts/build-deidentified-dataset.py 5 KB, SHA-256 3d86b0d84af020d12a3131ceb91a4ae642d85ac9b486cfeebe631640f498e4c6
• scripts/sample-from-curlie.py 3 KB, SHA-256 2c19a1af2c5b1dcb94bb897d28937f1f3c3807b2efae584eae47e5444c60ae16
• scripts/scan.py 9 KB, SHA-256 7ea184650c9373829c51f59bd0144a4f916b33e5f33eb65d5d7c5996d41ed206
• scripts/scan2.py 9 KB, SHA-256 657b04b409926eb1a4f26829e10174927c31be93ef34ae3f60e1a70542bea711
• test_csp3.py 7 KB, SHA-256 f0f94db40395f76460616de801d67f1e852ea67da38a63265d0d1ae80cfc0d4b
• test_golden.py 29 KB, SHA-256 c873c177d04a54485863c998bc3706324058c2da7b4f22eeddaa20f45aa32a26

Re-identification caveat. Rows carry only a sample row number and a keyed pseudonym, and the key is not released. But anyone who reruns the sampling script with its fixed seed (42) against the same directory snapshot can rebuild the sample and match rows to sites. That comes with reproducible sampling. The only sites named anywhere in the release are the eight qualifying Content-Security-Policy examples from the manual audit.

Version history and errata

• v7.3, pack v2.4 (2026-09-24). The names for the analysis bases were made consistent across this page, the PDF, the README, the data dictionary and the generated tables, and the definitions box was added. No number changed.
• v7.2 (2026-09-24). Byline and PDF title metadata corrected. No number changed.
• v7.1 (2026-09-24). First public release.

How to cite

RACKCRUNCH Team. Security Headers on Directory-Listed U.S. Local-Business Websites, study version 7.3, reproduction pack v2.4. RACKCRUNCH, 2026-09-24. https://rackcrunch.com/security-headers-2026

Artifact hashes

SHA-256 of every file published with this study. They are computed at build time from the exact files served.

• MANIFEST.sha256 1 KB, SHA-256 05e771765efd82aa3b5a682d88c31f7491f0fad03f11cdd4e3b8211c412f8994
• README-2026-09-24-v2.4.md 7 KB, SHA-256 3b8d21ce12146aac73067b3ce296bd03730af760da2df13f64f495a778b23884
• data-dictionary.md 7 KB, SHA-256 16a7204d7b8cb7573136d299e8172bb38c0886f96c148aaa03b11cdd990414a1
• final-tables.md 20 KB, SHA-256 3717c4afcf59751f6ccf0a12d93d84f762dade46848e8f105ed29b953a69cf59
• repro-pack-2026-09-24-v2.4.bundle 238 KB, SHA-256 1cf5c95c54bda2d09a2847aacccd78e67886a2fa2edb833fee6c1a0b0828fd1b
• repro-pack-2026-09-24-v2.4.tar.gz 239 KB, SHA-256 661a5a4197a072392bf0b35424d4795032c4c8ddec1571910f2e5624d44062e8
• requirements.txt 1 KB, SHA-256 7d5f5a94ac1690db632a2f6b2e51aaf74a8ba332ee248aa6299ec6e713120946
• scan2-2026-09-24-deidentified.jsonl 4.1 MB, SHA-256 329a1a910b4ae4b005fdd864fe9b5370df7e30bf01fc14b790baf7664d00428d
• security-headers-study-2026.pdf 400 KB, SHA-256 ab7c9e3df139c6ff735c78271947a4ff3a1e7ead3acc762927177dacd42f7d56
• state-counts-2026-09-24.tsv 2 KB, SHA-256 f757ac8d93c5f3c51060202ce51393315f5cb1b8e65fb209d6da899b2d16b03f

Sample drawn from the Curlie directory (https://curlie.org), used under the Creative Commons Attribution 3.0 Unported License. Sampling frame: the files in the curlie-rdf-all.tar.gz archive dated 2026-02-02, downloaded 2026-09-24 from https://curlie.org/download.

With content from Curlie.org - the largest human-edited directory of the web. Contribute by submitting a website or becoming an editor. Curlie data is licensed under the Creative Commons Attribution 3.0 Unported License.

Questions and opt-out

Write to hello@rackcrunch.com with questions, corrections or an opt-out request.