Friday, December 28, 2018

Robots.txt File Says Disallow:(Blank)

Disallow:(blank) in robots.txt file. I've never seen this before, but I have a feeling that this may be causing Google / Bing from looking at a website. I've run the robots.txt tester and it says allowed when putting a random URL in the tool, but I believe this line should be removed. Thoughts?

submitted by /u/tmblast1
[link] [comments]

from Search Engine Optimization: The Latest SEO News https://www.reddit.com/r/SEO/comments/aachd7/robotstxt_file_says_disallowblank/>

No comments:

Post a Comment