Glossary · SEO & Marketing

Indexing

Indexing is the step where a search engine analyzes a crawled page and stores it in its database so the page can appear in search results.

Updated October 6, 2026

Indexing is the process by which a search engine processes a page it has crawled, understands its content and stores it in a large searchable database called the index. Only indexed pages are eligible to appear in results.

Why it matters

Crawling a page does not guarantee indexing. Search engines may skip pages that are duplicates, low quality, blocked by a noindex directive or canonicalized to another URL. If a page is not in the index, no amount of optimization will make it rank.

Example

Typical pipeline:

Discover URL (link or sitemap)
  -> Crawl (fetch HTML, render)
  -> Index (analyze, store)
  -> Rank (when a query matches)

In Google Search Console, the URL Inspection tool shows whether a specific page is indexed and why not if it is excluded.

Common mistakes

  • Assuming a published page is automatically indexed.
  • Using robots.txt to keep a page out of the index; blocked pages can still be indexed from links.
  • Leaving a noindex tag in place after launch.

Get notified when this term updates

Plus occasional practical tips on uptime and monitoring, nothing else. Unsubscribe anytime.

Keep reading

Glossary · SEO & MarketingSEO (Search Engine Optimization)SEO is the practice of improving a website so search engines can crawl, understand and rank it higher in unpaid results for relevant searches.Glossary · Internet & WebSearch EngineA search engine is a service that crawls and indexes web pages and returns a ranked list of results matching a user's query.Guide · Web Basicsrobots.txt and XML Sitemap Guide: Syntax, Limits and MistakesWrite a correct robots.txt, understand Allow and Disallow precedence, noindex versus disallow, and build XML sitemaps within the 50,000 URL and 50 MB limits.8 min readGuide · E-commerceShopify Redirects and URL Structure: Fixes, Limits and Best PracticesHow Shopify URLs are structured and how to create and bulk-import URL redirects, handle changed handles and deleted products, and avoid chains and 404s.7 min readGlossary · SEO & MarketingCanonical URLA canonical URL is the preferred address of a page, declared with rel="canonical", that tells search engines which version to index when duplicates exist.Guide · E-commerceDoes Website Downtime Hurt SEO? What Google Says About 503 and OutagesHow outages affect Google crawling and rankings, why a 503 with Retry-After is the right response, how short and long outages differ, and how to recover.8 min read

This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.