Glossary
Crawler / bot — An automated program (e.g. Googlebot) that fetches web pages to discover and index content for search results.
Indexing — The process by which a search engine stores a page so it can appear in search results. noindex opts a page out of this.
robots meta tag — An HTML <meta name="robots" content="..."> tag in a page's <head> that gives crawlers instructions about that specific page. Only takes effect if the crawler is actually able to fetch the page (see robots.txt). Used in this PR at packages/clinic-web/public/index.html:7.
noindex — A robots meta directive meaning "don't list this page in search results."
nofollow — A robots meta directive meaning "don't crawl the links on this page to discover further URLs."
noarchive — A robots meta directive meaning "don't keep or show a cached/archived copy of this page."
robots.txt — A plain-text file at a domain's root (e.g. https://<domain>/robots.txt) that tells crawlers which paths they may or may not fetch. It's a site-wide, path-based policy — distinct from the per-page robots meta tag. Used in this PR at packages/clinic-web/public/robots.txt. Reference: robotstxt.org.
User-agent: * — A robots.txt block that applies its rules to all crawlers (as opposed to naming a specific one, e.g. User-agent: Googlebot).
Allow / Disallow — robots.txt directives permitting or forbidding crawlers from fetching a given path pattern.
Canonical link (<link rel="canonical">) — An HTML tag identifying the "official" URL for a page's content, used by search engines to deduplicate near-identical pages found at different URLs. Predates this PR (packages/clinic-web/public/index.html:8) and is unrelated to noindex — canonicalization only matters for pages a crawler has decided to index in the first place.
Single-page app (SPA) shell — An architecture where one static HTML file (here, packages/clinic-web/public/index.html) is served for every route, and client-side JavaScript (React Router, in clinic-web) renders the correct view after load. Any tag placed in that one HTML file applies to every route, which is why one noindex tag covers the entire app.
clinic-web — The Dentolize package this PR modifies: the React web application clinic staff use to manage patients, appointments, invoicing, and insurance. Not to be confused with clinic-web-canary (a component library clinic-web depends on, not a separately deployed app) or clinic-mobile (the React Native staff app, unaffected by this PR).