if you need to write a crawler script and that script needs to crawl a site protected by cloudflare. you will face a challenge, you got 403 error. It’s because somehow Cloudflare knows that we use some tools instead of browser. Even you open the Chrome Inspector and copy the curl command , you still got the error.
I did some try and found some ways pass this.
if you use curl : you need to use it through a proxy (Fiddler with HTTPs decrypt)
If you use wget you will need a proxy (Fiddler with HTTPs decrypt))+ change the user agent to Blackberry.
Note: I realized that the Fiddler 4 works well, when i upgraded Fiddler to the latest version i didn’t work.
Hope it helps.