Python requests download pdf files

Execute the above script and go to your "Downloads" directory. You should see the downloaded pdf document as "cat2. You can also download files using requests module. The get method of the requests module is used to download the file contents in binary format.

You can then use the open method to open a file on your system, just like we did with the previous method, urllib2. In the above script, the open method is used once again to write binary data to local file. If you execute the above script and go to your "Downloads" directory, you should see your newly downloaded JPG file named "cat3. With the requests module, you can also easily retrieve relevant meta-data about your request, including the status code, headers and much more.

In the above script, you can see how we access some of this meta-data. If you need to add customer headers, for example, all you need to do is create a dict with your headers and pass it to your get request:. Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet.

It works but is not the optimum way to do so as it involves downloading the file for checking the header. So if the file is large, this will do nothing but waste bandwidth.

I looked into the requests documentation and found a better way to do it. That way involved just fetching the headers of a url before actually downloading it.

This allows us to skip downloading files which weren't meant to be downloaded. To restrict download by file size, we can get the filesize from the Content-Length header and then do suitable comparisons.

We can parse the url to get the filename. This will be give the filename in some cases correctly. However, there are times when the filename information is not present in the url. In that case, the Content-Disposition header will contain the filename information. Here is how to fetch it. Before we see it in action, we first need to retrieve the total file size and the file name:. We get the file size in bytes from Content-Length response header, we also get the file name in Content-Disposition header, but we need to parse it using cgi.

Let's download the file now:. We then wrapped the iteration with a tqdm object, which will print a fancy progress bar. We also changed the tqdm default unit from iteration to bytes.

After that, in each iteration, we read a chunk of data and write it to the file opened , and update the progress bar. Here is my result after trying to download a file, you can choose any file you want, just make sure it ends with the file extension.

Skip to main content. Radar Now, next, and beyond: Tracking need-to-know trends at the intersection of business and technology.