Describe a time when you faced a significant challenge in a web scraping project and how you overcame it.