How Search works: from discovery to the result
Understand the stages that move a page from a link on the web to a possible search result, then diagnose the first stage that is actually failing.
Find the guide that matches the failure you can see.
Each guide stays focused on one operating decision, with a process you can reuse in the next release.
Use URL Inspection without confusing live and indexed states
A disciplined URL Inspection workflow keeps the processed record, the current live response, and the work you still need to observe separate.
Write robots.txt rules that match your crawl policy
robots.txt tells compliant crawlers which paths they may fetch. It is a crawl-management file—not authentication and not a dependable way to remove already discoverable URLs from an index.
Use noindex safely without blocking the evidence you need
A noindex directive tells crawlers not to include a page or resource in their index when they can fetch and process the directive. It is not a security control and it should match the page’s intended public state.
Build an XML sitemap that supports discovery
An XML sitemap is a structured discovery hint. It works best when it contains the clean canonical URLs you want crawlers to find—not every address your application can generate.