Hey, everyone. I was going through the robots.txt file of the Udemy website, and I saw that the webmaster disallowed the Baidubot from crawling any of the website's pages. They've even included a no-index tag as well.
Is there a reason why they'd do this? Is the website blocked in China? Just curious.
[link] [comments]
from Search Engine Optimization: The Latest SEO News https://www.reddit.com/r/SEO/comments/mhsgk8/baidubot_blocked_on_udemy/>
No comments:
Post a Comment