Hello, A client of mine is going through a bit of a crisis. A developer (at their end) added Disallow: / to the robots.txt file.
I'm getting crawl errors that MOZ isn't able to access my robots.txt file but it seems completely fine to me? Any chance anyone can help me ...
Old Hard to Find TV Series on DVD
Hello Mozzers! I've received an error message saying the site can't be crawled because Moz is unable to access the robots.txt.
So my question to all is: What is the best way to fix the issue caused by these 44 pages? A. Remove those 44 blog posts that used verbatim ...
When I go to the tester it shows the live robots.txt with no problems. I also can look at the server logs and see that it is being crawled, but ...
After a site audit, I find out that my website don't have a robot.txt. Does it hurt my website rankings? One more thing, when I type ...
Moz are reporting the robots.txt file is blocking them from crawling one of our websites. But as far as we can see this file is exactly the ...
I'm currently having trouble with what appears to be a cached version of robots.txt. I'm being told via errors in my Google sitemap account ...
I am receiving the following error message through webmaster tools http://www.sourcemarketingdirect.com/: Googlebot can't access your site ...
When a robot looks for the "/robots.txt" file for URL, it strips the path component from the URL (everything from the first single slash), and ...