An HTTP header that gives search engines robots instructions, useful for files such as PDFs that have no HTML head.
Glossary
X-Robots-Tag
Related terms
Search engine optimisation (SEO)
Improving a website so it ranks higher in search results for the searches your customers make, through technical fixes, useful content and…
Log file analysis
Studying server logs to see exactly which pages search engine and AI crawlers visit, how often, and which they ignore.
Noindex
An instruction in a page's code or headers telling search engines not to show it in results, such as for thank-you pages.
Title tag
The page title shown in search results and browser tabs. One of the clearest signals of what a page is about; Google…
Google Search Console
Google's free tool showing how a site performs in search: the queries people use, clicks, indexing problems and Core Web Vitals.
Search intent
What someone actually wants when they search: to learn something, compare options, buy, or find a particular site. Pages rank best when…
Work with us
Ready for your next stage of growth?
Tell us where you want to take your brand. Our team will come back within one working day with next steps.