Programmatic SEO Without Thin Pages: Build the Gate Before the Generator

TL;DR: Every guide on this topic tells you to add unique value to each page and then stops, which is advice you cannot execute. The useful version is a publish gate: a function that takes a generated page and returns true or false, running before anything ships. This is how to define those thresholds concretely — minimum unique content, minimum data density, duplicate detection, and a demand check — plus why the gate has to exist before the generator rather than after the traffic drop.

The failure isn't automation, it's the missing gate

Programmatic SEO has a reputation problem earned by a specific mistake, and it's worth naming precisely because the wrong diagnosis leads to the wrong fix.

The mistake was never "we used a template." Templates are how every large site works — every product page on every e-commerce site in the world is a template. Google's own guidance is consistent on this: what's penalised is producing many pages primarily to manipulate rankings rather than to help anyone, and the method of production isn't the deciding factor. A million templated pages backed by real inventory data are fine. Five hundred pages that are one variable swapped into identical boilerplate are not.

The mistake was that generation had no gate. Someone wrote a script that produced pages from a dataset, and every row in the dataset became a page. There was no step in the middle asking whether this particular page had enough behind it to deserve a URL.

That's an engineering problem, and it has an engineering answer.

What a gate actually is

A publish gate is a function. It takes a candidate page and returns a boolean, and it runs before the page enters your sitemap.

shouldPublish(page) -> boolean

Everything below is a proposal for what goes inside that function. The specific thresholds are yours to set — they depend on your category and your data — but the structural point holds regardless: the decision has to be code, not judgement, because judgement doesn't scale to ten thousand pages and won't be applied consistently at 2am when you're behind on a launch.

Pages that fail the gate don't get deleted. They sit in the database, unpublished and unlinked, waiting for enough data to pass. That's the whole model: the dataset is complete, the published surface is a filtered view of it.

The four checks

1. Unique content ratio

Take the rendered page. Strip the parts that appear on every page in the set — nav, footer, the template's fixed prose, the boilerplate intro. What's left is the unique payload.

The check: unique payload as a proportion of total content. If a page is mostly chrome with a name swapped in, this number tells you immediately, and it's the single most predictive signal of the thin-page pattern.

A reasonable starting threshold is 40% unique, but treat that as a dial to calibrate rather than a law. Run it across pages you'd be happy to defend and see what they score, then set the floor just below. If your best pages score 35%, your template is too heavy — that's the finding, and the fix is trimming boilerplate rather than lowering the bar.

2. Data density

Unique ratio can be gamed by generating filler that's technically unique and says nothing. So pair it with a count of actual populated facts.

Define the fields that make a page worth reading — for a tool comparison that might be pricing, both feature sets, integration lists, tagged use cases, a verdict. Then require a minimum number of them to be non-empty. Not "the record exists" but "the record has substance."

This is the check that catches the sparse-data problem honestly. If you have 3,000 tools and only 400 have complete enough records to make a real page, you publish 400. The other 2,600 aren't lost, they're queued. A site with 400 solid pages outranks a site with 3,000 hollow ones, and it doesn't drag down the good pages the way the hollow version does.

3. Near-duplicate detection

Two pages can each pass the first two checks and still be near-identical to each other. This is the specific failure mode of comparison and location pages, where "X vs Y" and "Y vs X" contain the same information, or two adjacent categories produce substantively the same list.

Shingle the unique payload, hash it, and compare against everything already published in the set. Above a similarity threshold, one of them doesn't publish — or better, they merge into one canonical page and the other redirects.

This check is unglamorous and it's the one most people skip, because near-duplicates are invisible when you're looking at pages one at a time. They're only visible in aggregate, which is exactly what a gate is good at.

4. Demand

The last check is different in kind: is there anyone on the other end?

A page targeting a query nobody searches is not spam, but it's not an asset either. It consumes crawl budget, dilutes the average quality of your indexed set, and returns nothing. Before generating a page pattern, confirm the pattern matches real queries — search volume where you can measure it, and where you can't, evidence that people phrase the question this way at all. Forum posts, support tickets, and Stack Overflow titles are all better evidence than intuition.

This is also the check that should kill entire page types, not just individual pages. If a whole pattern has no demand behind it, no threshold tuning will save it.

Ship in waves, not in one push

Even when every page passes, publishing ten thousand URLs on a Tuesday is a mistake.

Crawl budget is finite and roughly proportional to your site's authority. A new domain dumping a huge page set gets a fraction of it crawled, and the crawler's sampling of your set determines its judgement of the whole. If the sample it happens to hit is your weakest tier, that's the impression that sticks.

So release in waves, strongest first. Start with the pages you'd show someone as an example of your best work — usually a few dozen. Watch what gets indexed, what ranks, what earns clicks. Then release the next tier. This gives you something the one-push approach never provides: a feedback loop while you can still change the template.

It also means the first pages the crawler ever sees from your set are your best ones, which is the opposite of what happens when you sort by database insertion order.

The internal linking part everyone forgets

Generated pages orphaned in a sitemap don't work. Discovery matters, and a URL that appears in sitemap.xml and nowhere else is a URL you've told Google you don't care about.

Three things to build alongside the pages:

• Hub pages that list the set, organised in a way a human would actually browse — not paginated alphabetically to page 340.
• Lateral links between related generated pages. If page A and page B are genuinely related, link them, and use the relationship as the anchor text. This is where a good taxonomy pays off: the tags already know what's related, so the links can be derived rather than hand-authored.
• Links from your hand-written content into the generated set. This is the one that's usually missing. Editorial pages carry authority that generated pages don't, and a link from a real article is worth more than ten links from sibling template pages.

The point of a taxonomy with fixed, predefined values is precisely that these links can be computed. We wrote up the reasoning behind that separately in designing a taxonomy that survives real data — the short version is that you cannot derive a link graph from user-invented tags, because the values aren't stable enough to point at.

Auditing a set that's already live

If you've already published and things aren't working, the diagnostic sequence is fairly mechanical.

Start in Search Console's page indexing report. The signal to look for is a large gap between discovered and indexed — pages found and deliberately not indexed is Google telling you, fairly directly, that it evaluated the set and declined. That's a quality verdict, not a technical bug, and no amount of sitemap resubmission fixes it.

Then segment the set by impressions. You'll usually find a small fraction of pages earning nearly all of it and a long tail earning literally zero over months. That tail is the problem: it's not neutral, it's drag on the whole domain's assessment.

For the tail, three options in order of preference. Enrich the ones with genuine demand behind them — that's a data problem, not a content problem. Consolidate near-duplicates into one stronger page with redirects. Remove the rest: noindex them, take them out of the sitemap, and keep the records unpublished until they'd pass the gate.

Removing pages to gain traffic feels wrong and works anyway. The site-level quality assessment is holistic; a large set of pages nobody wants suppresses the pages people do want.

What the gate is really for

The honest framing is this: programmatic SEO is a data product expressed as web pages. The pages are an output format. If the underlying data is thin, no template, no AI-written intro paragraph, and no schema markup will fix it, because there's nothing there to express.

Which means the gate isn't really a spam-avoidance mechanism, though it works as one. It's a forcing function that keeps you honest about whether your dataset is good enough yet. Every page that fails is telling you which record needs work.

That's a much better relationship with your own data than discovering the same information eight months later in a traffic graph.

FAQ

Is programmatic SEO still effective in 2026?

Yes, when each page is backed by genuinely distinct data. What stopped working is the volume approach — generating large page sets by swapping one variable into identical boilerplate. Google's spam policies target producing many pages primarily to manipulate rankings rather than to help users, and the deciding factor is value per page rather than whether automation was involved. Large sites built on real structured data continue to rank normally.

What is scaled content abuse?

It's Google's term for generating many pages, whether by AI or template automation, primarily to manipulate search rankings rather than to help users. The characteristic pattern is near-identical pages with minimal unique information, produced at volume. The policy is explicit that the method of creation is not the issue — the same automation producing genuinely useful, data-backed pages is fine.

How do I avoid thin content penalties with programmatic pages?

Build a publish gate that runs before pages go live, checking unique content ratio, count of populated data fields, near-duplicate similarity against pages already in the set, and whether real search demand exists for the query pattern. Pages failing any check stay unpublished until the underlying record improves. The decision needs to be code rather than judgement, because manual review does not scale and is not applied consistently.

How many programmatic pages should I publish at once?

Publish in waves rather than all at once, starting with your strongest pages. Crawl budget is finite and roughly proportional to site authority, so a large simultaneous release on a newer domain means most pages never get crawled, and the sample the crawler does see shapes its assessment of the whole set. Start with a few dozen of your best, watch indexation and rankings, then release the next tier.

Why are my programmatic pages not getting indexed?

A large gap between discovered and indexed pages in Search Console usually means the pages were evaluated and judged not worth indexing, rather than a technical problem. Common causes are near-duplicate content across the set, too little unique information per page, and pages that are orphaned in the sitemap without internal links pointing to them. Resubmitting the sitemap does not help, because the issue is quality assessment rather than discovery.

Should I delete underperforming programmatic pages?

Often yes. Site quality assessment is holistic, so a large set of pages earning no impressions can suppress the pages that would otherwise perform. Enrich the ones with genuine search demand behind them, consolidate near-duplicates into single stronger pages with redirects, and noindex the remainder while keeping the underlying records for later. Removing pages to gain traffic is counterintuitive and frequently works.

How much unique content does each programmatic page need?

There is no official threshold, so calibrate against your own best pages rather than a number from a blog post. Measure the proportion of each page that is unique after stripping shared navigation, footer and template prose, check what your strongest pages score, and set the floor just below that. If those pages score poorly, the finding is that your template carries too much boilerplate, and the fix is trimming it rather than lowering the bar.

What's the difference between programmatic SEO and doorway pages?

A programmatic page is a destination — someone arriving there gets the information they searched for and their task is complete. A doorway page is a funnel that exists to capture a query and push visitors somewhere else, offering nothing itself. The practical test is whether a visitor who lands on the page and leaves satisfied has been well served. If the page only makes sense as a step toward another page, it is a doorway regardless of how it was built.

---

The approach described here is how comparison and category pages get published on TechLogHub — records exist for far more tools than have live pages, and the gap is the gate.