Hi guys. I have a news publisher client who wants to do a CMS migration and I need to crawl their site. I did it before and it took me a lot of time because I found more than 600 k URLs 😅 is there a configuration for the spider you recommend me to set so I can crawl the site fast and get the most important URLs that they need to redirect. A lot of that URLs are worthless because are already redirected or broken and don't have value.
[link] [comments]
from Search Engine Optimization: The Latest SEO News https://www.reddit.com/r/SEO/comments/n2plqf/need_screaming_frog_advice/>
No comments:
Post a Comment