How to by pass cloudflare protection when using curl,wget

if you need to write a crawler script and that script needs to crawl a site protected by cloudflare. you will face a challenge, you got 403 error. It’s because somehow Cloudflare knows that we use some tools instead of browser. Even you open the Chrome Inspector and copy the curl command , you still got the error.

I did some try and found some ways pass this.

if you use curl : you need to use it through a proxy (Fiddler with HTTPs decrypt)
If you use wget you will need a proxy (Fiddler with HTTPs decrypt))+ change the user agent to Blackberry.

Note:  I realized that the Fiddler 4 works well, when i upgraded Fiddler to the latest version i didn’t work.

Hope it helps.

Leave a Reply

Your email address will not be published. Required fields are marked *