Urllib.request file download use default name

An easier way I think also you can do it in two lines is to use: import urllib. Xantium Xantium 9, 9 9 gold badges 55 55 silver badges 80 80 bronze badges. The PEP20 would have you use Request from urllib. Information about PEP20 for Request.

You can use open chained to file. Debug Are you sure? That worked on debian9 using python3. I don't use 2. This doesn't work if you have to get round the Forbidden issue using stackoverflow. Sevenearths is a Forbidden error. This usually happens when a website server attempts to block a bot. Seen as the solution you listed uses a user agent, it strongly looks like that site attepts to block bots which makes sense since it's a news site a user agent tricks the server into thinking it's a legitimate browser.

Show 3 more comments. Sign up or log in Sign up using Google. Sign up using Facebook. Sign up using Email and Password. Post as a guest Name. Email Required, but never shown. The Overflow Blog. Return values and exceptions raised are the same as those of urlopen. The order in which these methods are called within each stage is determined by sorting the handler instances. This stage ends when a handler either returns a non- None value ie.

Exceptions are allowed to propagate. BaseHandler objects provide a couple of methods that are directly useful, and others that are meant to be used by derived classes. These are intended for direct use:. The following attribute and methods should only be used by classes derived from BaseHandler. A valid OpenerDirector , which can be used to open using a different protocol, or handle errors.

This method is not defined in BaseHandler , but subclasses should define it if they want to catch all URLs. This method, if implemented, will be called by the parent OpenerDirector. It should return a file-like object as described in the return value of the open of OpenerDirector , or None. This method is not defined in BaseHandler , but subclasses should define it if they want to handle URLs with the given protocol. This method, if defined, will be called by the parent OpenerDirector.

This method is not defined in BaseHandler , but subclasses should define it if they want to catch all URLs with no specific registered handler to open it. This method is not defined in BaseHandler , but subclasses should override it if they intend to provide a catch-all for otherwise unhandled HTTP errors.

It will be called automatically by the OpenerDirector getting the error, and should not normally be called in other circumstances. Return values and exceptions raised should be the same as those of urlopen. This method is also not defined in BaseHandler , but will be called, if it exists, on an instance of a subclass, when an HTTP error with code nnn occurs.

This method is not defined in BaseHandler , but subclasses should define it if they want to pre-process requests of the given protocol. The return value should be a Request object. This method is not defined in BaseHandler , but subclasses should define it if they want to post-process responses of the given protocol. The return value should implement the same interface as the return value of urlopen. See RFC for details of the precise meanings of the various redirection codes.

Return a Request or None in response to a redirect. The default implementation of this method does not strictly follow RFC , which says that and responses to POST requests must not be automatically redirected without confirmation by the user. In reality, browsers do allow automatic redirection of these responses, changing the POST to a GET , and the default implementation reproduces this behavior. The http. CookieJar in which cookies are stored. The method will modify requests to go through the proxy, by calling request.

This causes user, passwd to be used as authentication tokens when authentication for realm and a super-URI of any of the given URIs is given. In either case, the authority must not contain a userinfo component so, "python. Open the file locally, if there is no host name, or the host name is 'localhost'. When a remote hostname is given, an URLError is raised. Read a data URL. This implementation ignores white spaces in base64 encoded data URLs so the URL may be wrapped in whatever source file it comes from.

Open the FTP file indicated by req. The login is always done with empty username and password. Raise a URLError exception. Note that urlopen returns a bytes object. This is because there is no way for urlopen to automatically determine the encoding of the byte stream it receives from the HTTP server.

In general, a program will decode the returned bytes object to string once it determines or guesses the appropriate encoding. As the python. It is also possible to achieve the same result without using the context manager approach. In the following example, we are sending a data-stream to the stdin of a CGI and reading the data it returns to us.

Note that this example will only work when the Python installation supports SSL. Here is an example of doing a PUT request using Request :. This example replaces the default ProxyHandler with one that uses programmatically-supplied proxy URLs, and adds proxy authorization support with ProxyBasicAuthHandler.

Use the headers argument to the Request constructor, or:. OpenerDirector automatically adds a User-Agent header to every Request. To change this:. The following example uses the POST method instead. Note that params output from urlencode is encoded to bytes before it is sent to urlopen as data:. The following functions and classes are ported from the Python 2 module urllib as opposed to urllib2.

They might become deprecated at some point in the future. Copy a network object denoted by a URL to a local file. If the URL points to a local file, the object will not be copied unless filename is supplied.

Return a tuple filename, headers where filename is the local file name under which the object can be found, and headers is whatever the info method of the object returned by urlopen returned for a remote object. Exceptions are the same as for urlopen. The second argument, if present, specifies the file location to copy to if absent, the location will be a tempfile with a generated name. The third argument, if present, is a callable that will be called once on establishment of the network connection and once after each block read thereafter.

The callable will be passed three arguments; a count of blocks transferred so far, a block size in bytes, and the total size of the file.

The third argument may be -1 on older FTP servers which do not return a file size in response to a retrieval request. If the url uses the http: scheme identifier, the optional data argument may be given to specify a POST request normally the request type is GET.

This can occur, for example, when the download is interrupted. You can still retrieve the downloaded data in this case, it is stored in the content attribute of the exception instance. If no Content-Length header was supplied, urlretrieve can not check the size of the data it has downloaded, and just returns it. In this case you just have to assume that the download was successful. Cleans up temporary files that may have been left behind by previous calls to urlretrieve.

Base class for opening and reading URLs. The dictionary is reproduced here for convenience. When an error is raised the server responds by returning an HTTP error code and an error page. This means that as well as the code attribute, it also has read, geturl, and info, methods as returned by the urllib. I prefer the second approach. The response returned by urlopen or the HTTPError instance has two useful methods info and geturl and is defined in the module urllib.

This is useful because urlopen or the opener object used may have followed a redirect. It is currently an http. HTTPMessage instance.

When you fetch a URL you use an opener an instance of the perhaps confusingly-named urllib. Normally we have been using the default opener - via urlopen - but you can create custom openers. Openers use handlers. You will want to create openers if you want to fetch URLs with specific handlers installed, for example to get an opener that handles cookies, or to get an opener that does not handle redirections.

To create an opener, instantiate an OpenerDirector , and then call. Other sorts of handlers you might want to can handle proxies, authentication, and other common but slightly specialised situations.

This means that calls to urlopen will use the opener you have installed. For a more detailed discussion of this subject — including an explanation of how Basic Authentication works - see the Basic Authentication Tutorial. When authentication is required, the server sends a header as well as the error code requesting authentication.

The client should then retry the request with the appropriate name and password for the realm included as a header in the request. This allows you to specify a default username and password for a URL. This will be supplied in the absence of you providing an alternative combination for a specific realm. This is through the ProxyHandler , which is part of the normal handler chain when a proxy setting is detected. One way to do this is to setup our own ProxyHandler , with no proxies defined.

This is done using similar steps to setting up a Basic Authentication handler:. Currently urllib. However, this can be enabled by extending urllib.

The Python support for fetching resources from the web is layered. As of Python 2. This can be useful in applications which have to fetch web pages. By default the socket module has no timeout and can hang.