Downloading wget Without wget: Use bash

Nathan Phillip Brink — Fri, 14 Jan 2011 07:16:59 +0000

There are many ways to download and install wget without having wget itself installed. For example, one can use curl, a sort of competitor to wget, or a package manager with libfetch or some other library-level downloader integrated (such as pacman). One may be able to use SSH’s scp or sftp utility or even use netcat to transfer a wget tarball over a network. But these methods of obtaining wget are not always feasible or even possible whereas a bash shell and a few core utilities are often readily available.

I was introduced to the bash builtin /dev/tcp by warg the other day on x-tab#chat. He explained a basic use of this device by demonstrating how to download wget’s compressed tarball. The download process itself can be done with pure bash, but some post-processing of the downloaded file must be done to remove HTTP headers. I document warg’s application of /dev/tcp here because I found the idea fascinating and want this documentation for myself ;-).

Connecting and Downloading

To read about the /dev/tcp builtin for yourself, check out the following:

$ info '(bash) Redirections'

With the exec line we initiate the connection, allocating a file descriptor and storing the numeic file descriptor into the HTTP_FD variable. Then, with the echo line, we send an HTTP request through the descriptor to the server. After sending the request, we process the server’s response with the sed line which skips over the HTTP headers sent by the server and stows the results into wget-latest.tar.gz. Note that this last command will sit around for a while. It is with this command that the builk of the data transfer is performed. And, since you’re using shell redirections to download the file, you cannot see the download progress. Instead, wait for the command to complete. This also involves waiting for the server to time out your connection since it supports pipelining. After this process is completed, the wget-latest.tar.gz file is as your disposal.


$ WGET_HOSTNAME='ftp.gnu.org'
$ exec {HTTP_FD}<>/dev/tcp/${WGET_HOSTNAME}/80
$ echo -ne 'GET /gnu/wget/wget-latest.tar.gz HTTP/1.1\r\nHost: '\
    ${WGET_HOSTNAME}'\r\nUser-Agent: '\
    'bash/'${BASH_VERSION}'\r\n\r\n'  >&${HTTP_FD}
$ sed -e '1,/^.$/d' <&${HTTP_FD} >wget-latest.tar.gz

Now you have a wget source tarball on your machine. As long as you have tar and a compiler on the machine, you are well on your way to downloading stuff using a self-compiled wget. In the commands above, you may replace “gz” with “bz2” or “lzma” for smaller downloads if the machine you’re using has bzip2 or xz-utils installed. And, of course, it should not be too hard to repurpose the above code to download a particular version of wget or even a completely unrelated software package.

Please feel free to point out problems with this approach or give pointers on porting this to other environments :-).

wget – protofusion

Downloading wget Without wget: Use bash

Connecting and Downloading