Pinterest Crawler 140K hits per day, is this insanely high? Oh... and my Robot.txt is set to a delay of 1 for them...

My site keeps having some php performance issues and when tracing the top five IP addresses per 24 hours, they are all from Pinterest and total over 140k requests. I also have a crawl delay set up in Robot.txt and I'm still getting this amount of crawler traffic

submitted by /u/robertaugustapparel
[link] [comments]

from Search Engine Optimization: The Latest SEO News https://www.reddit.com/r/SEO/comments/r6wsuw/pinterest_crawler_140k_hits_per_day_is_this/>