Skip to content

Tag Archives: robots

What does “Sitemap:http://example.com/sitemap.xml” mean in laymen terms, when it is found in ?

"http://www.example.com/robots.txt"? Thanks in advance.

Why does Google look for b94H0R4d.xml on my server?

I’m hosting a private webserver (just for experimentation and fun since I don’t know what to put on it really) and google crawls it looking for robots.txt which I know what that is.. but it also looks for b94H0R4d.xml, is this somehow a sitemap search? usually sitemaps are named sitemap.xml though. Any idea what b94H0R4d [...]

Do I have to make different robots.txt and sitemap.xml on every subdomain of my website?

I have a website and 5 subdomains, and only my main homapage have sitemap.xml and robots.txt. I want all of my subdomains to be indexed by search engine. do i have to make robots.txt and sitemap.xml on every subdomain?