I read about being able to use the HTTP header X-Robots-Tag to specify Google search index control options. For my purposes, I wanted to specify that certain pages on my blog would not go in the index. So I specified the header X-Robots-Tag: noindex using Microsoft’s URL Rewrite module 2 which can use out bound rules to rewrite page content and page headers.
The header was correctly being added to the pages, but when I used the remove URL tool in Google webmaster tools to remove the pages the result was the request for page removal was denied.
The information about the X-Robots-Tag discusses using it for page content like PDF files and such. It does not specifically state that it will or will not work for regular content type text/html. I asked on the Google webmaster tools help forum, but I did not get any response.
I ended up modifying my blog’s code to add a meta ROBOTS tag to the HTML head element of my pages which works as expected.
I can’t be 100% sure that the X-Robots-Tag will not do the job, but I could not get it to work. If someone has found out otherwise please let me know.
Next entry: I use an eye mask to help with a CPAP causing dry eyes
Previous Entry: One thing I realized I’ll miss as a father – rocking my kids
Latest entries:
Create absolute URLs using ASP.NET MVC
Comments
My Links
Tags
Follow me
About
Powered by FoxBlog
Disclaimer
The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.
© Copyright 2011, Nathan Fox