Dentolize · Clinic-Web Search Deindexing Walkthrough
On this pageWhat to testEdges worth double-checking

For Quality

What to test

This is metadata/crawler configuration, not application logic, so testing is about verifying the served files and confirming zero behavior regression — not exercising a workflow.

1. The meta tag is present and correct, on every route

  • Load the app unauthenticated (e.g. /auth/login) and inspect the

document <head>. Confirm: <meta content="noindex, nofollow, noarchive" name="robots" /> is present, exactly, in packages/clinic-web/public/index.html:7.

  • Log in and load an authenticated route (dashboard, a patient record, an

invoice). Confirm the same tag is present there too — this is the whole point of the fix (single SPA shell, one index.html for all routes), so a regression here (e.g. tag missing on some routes) would mean the app stopped being a pure client-rendered SPA, or someone introduced a route-specific HTML shell.

  • Confirm the pre-existing <link rel="canonical" href="https://my.dentolize.com" />

(index.html:8) is untouched and still present — this PR should not have modified it.

2. robots.txt serves the expected policy

  • Fetch /robots.txt directly and diff against

packages/clinic-web/public/robots.txt. Confirm:

  • User-agent: * still present.
  • Allow: / present (regression here would silently re-hide the

noindex tag from crawlers, defeating the whole PR).

  • Disallow: /*.pdf$, /*.xlsx$, /*.jpg$ all present, each anchored

with $ (a missing $ would broaden or narrow the match unexpectedly — e.g. without the anchor, /*.pdf could match /pdfviewer or similar paths that merely contain pdf, not just files ending in .pdf).

  • Confirm the file is served as static content with `Content-Type:

text/plain` at the domain root, not wrapped in the SPA shell.

3. No regression to actual app functionality

Since Allow: / intentionally keeps the whole app crawlable/fetchable, and Disallow only affects three file extensions:

  • Confirm normal login (both email and username/company flows) still works

end-to-end.

  • Confirm a logged-in user can still open/download a .pdf, .xlsx, or

.jpg file directly (e.g. an exported invoice, an uploaded patient photo) via a normal browser request — Disallow in robots.txt is a crawler-only instruction and must not affect direct navigation, <a href> downloads, <img> rendering, or API-driven fetches. If any of those break, that's a serious regression — it would mean something downstream (e.g. a CDN or reverse proxy) is misinterpreting robots.txt as an access-control rule, which it is not.

Edges worth double-checking

  • Extension matching is case-sensitive by convention — robots.txt

patterns like /*.pdf$ do not match .PDF. If uploaded/exported files can have uppercase extensions, crawlers could still index those. Worth confirming whether clinic-web/the server ever produces uppercase extensions; if so, this rule wouldn't catch them (not necessarily a bug to fix in this PR, but worth a QA note).

  • Query strings and hashes: robots.txt patterns match the path, and

behavior with query strings after a $-anchored pattern can vary by crawler. If exported file URLs include a query string (e.g. a signed S3 URL with ?X-Amz-Signature=...), confirm whether the $-anchored Disallow rules still apply as intended for those real URLs, since $ anchors to the end of the path before the query string per the robots.txt spec, but not all crawlers are equally spec-compliant.

  • This does not enforce anything server-side — robots.txt and

noindex are voluntary conventions honored by well-behaved crawlers. They are not an access-control mechanism. Don't test this as if it were a security boundary; it isn't one, and no test should assert that unauthenticated access is blocked as a result of this PR (it was never blocked before, and still isn't after).