A feed with a few thousand records can become a few thousand pages in an afternoon: one for each product, property, event or, on a site of mine, asteroid. Whether it should is a question Google’s documentation answers more directly than most people expect. Below I set out what Google says about pages made at scale, what its own questions ask of a single record page, and a short test to run before the set grows. Where I go beyond what Google states, I say so. For the practical steps, my guide to checking pages built from a data feed covers them.
One page per record is a decision, not a default
A record in a feed usually has a natural address, and a page for it can be the right answer, because each record is a distinct thing someone may search for. Nothing in the documentation I read says that building pages from structured data is a problem in itself. The test it applies is about value and purpose. Google’s spam policy defines scaled content abuse as many pages generated for the primary purpose of manipulating search rankings rather than helping users, typically large amounts of unoriginal content that gives users little or no value, no matter how it’s created
.
I have had to make this decision myself on Rock Dodger, a comedy asteroid tracker I built on NASA’s public near-Earth object data. Every asteroid that passes through its weekly feed gets its own page. The numbers come from NASA; the site adds its own arithmetic, comparisons and diagrams on top.
What Google’s spam policy says about pages made at scale
Three parts of Spam Policies for Google Web Search bear on a set of record pages. Under scaled content abuse, Google lists scraping feeds, search results or other content to generate many pages, including through automated transformations such as synonymising or translating, where little value is provided to users, and stitching together content from different pages without adding value. Under scraping, it lists reproducing content feeds from other sites without providing some type of unique benefit to the user. Under doorway abuse, it lists creating substantially similar pages that are closer to search results than a clearly defined, browseable hierarchy. Google’s instruction for content of this kind on your own site is to exclude it from Search.
I found no page count in the policy at which a set becomes a problem. Every example turns on the user receiving little or no value, so the question to put to a set is what each page gives a visitor that the source does not.
What the helpful-content guidance asks of a single page
Google’s page Creating Helpful, Reliable, People-First Content poses questions that a record page can be held against one at a time. Does the content provide original information, reporting, research or analysis? If it draws on other sources, does it avoid simply copying or rewriting them, and provide substantial additional value and originality? Does it provide substantial value when compared to other pages in search results? Is it mass-produced, so that individual pages do not get much attention or care?
The same page describes the attributes Google’s Search Quality raters are trained to evaluate in main content. On effort, it says that automatically generating pages from feeds, or producing large amounts of text with generative AI without manual oversight or curation, represents little to no effort, and that giving credit to other sources does not replace the need for original effort. It also calls the “why” perhaps the most important question: content made primarily to help people aligns with what Google’s systems seek to reward, and content made primarily to attract search visits does not. I cite the effort passage for what Google treats as effort and originality, not as a ranking formula.
A short test for a record page
This is my reading of those questions, not Google’s wording. Run it on five or six pages picked across the set, including the least popular records.
- Subtract the feed. Remove every field the source already publishes. If little remains, the page adds little.
- Name what is added. Write one sentence on what a visitor learns here that they would otherwise have to work out or find elsewhere: a comparison, a history, a plain-English reading of what the numbers mean.
- Compare siblings. Open two pages side by side. If the only difference is the values in the fields, ask whether both need to exist as separate pages.
- Read the title and the opening. Each should describe that record specifically, not repeat a template with a name swapped in.
- Ask the “why”. Would the page be worth publishing if nobody arrived from search?
On Rock Dodger the second step has an answer I can point to: each asteroid page carries scale diagrams, size and speed comparisons and the object’s past and future close approaches, which is more than the measurements alone.
Keeping a large set findable
Google’s SEO Link Best Practices say it finds new pages through links, and that it can generally crawl a link only if it is an <a> element with an href attribute. An archive that links every record with ordinary anchor links gives Google a route to each page; one that depends on script events may not. My article on script-built links covers that failure.
A sitemap helps discovery on a large set. Build and Submit a Sitemap limits one file to 50MB uncompressed or 50,000 URLs, allows a sitemap index file for more, and says Google uses lastmod only if it is consistently and verifiably accurate, reflecting the last significant update to the page. It follows that a feed-driven site should set lastmod from the record’s real change, not from the date the sitemap was generated. See also my article on sitemaps.
What to do with records that do not pass
I see three options, and the documentation does not rank them. Add the missing material so the page earns its place. Fold several thin records into one fuller page. Or exclude the page from Search. For the last, Block Search Indexing with noindex says a noindex rule, set as a meta tag or an HTTP response header, makes Google drop the page from results once it crawls it. The rule works only if the page is not blocked by robots.txt and is otherwise accessible, because a crawler that cannot fetch the page never sees the rule.
Four traps when a feed becomes a site
- Crediting the source and stopping there. Google says credit does not replace original effort.
- Rewording the feed with synonyms. The spam policy names synonymising among the automated transformations that do not rescue a page offering little value.
- One template, one description. My view: thousands of pages that answer the same question in the same words are the pattern the “mass-produced” question describes.
- Padding the count. Pages for records nobody would look for add to the count, not the value; the doorway wording about substantially similar pages applies here.
Where record pages fit at Cultured Digital
My web development work covers websites, rebuilds, tools and MVPs with organic search in the architecture from the first line of code, and deciding which records deserve a page of their own is part of that. Site structure and internal linking sit under architecture in my technical SEO work, and the build side under websites.
