How To Use The Swiss Army Knife Of The Savvy Internet User wget

[dserban 0]

In this tutorial I will show you how to use wget in Windows.
wget is a very handy utility, I would call it "the Swiss army knife of the savvy Internet user".
Its main purpose is for mirroring HTTP and FTP sites, but you can get very creative with it and save a lot of browser navigation clicks if you have many repetitive tasks.
wget originated in the UNIX / Linux world and has been ported to Windows.
There are lots of places where you can go to download wget for Windows, just Google "wget for Windows" or "wget for Windows easy steps".


How-to #1 - Discover the REAL location of the file you're about to download

Many times a web server will attempt to obscure the location of the files it is serving.
Example from art.gnome.org
While browsing for a set of wallpaper jpeg files, I ran across this site and clicked my way to the page where you are supposed to right-click on the link and select "Save As ...", but instead of "Save As ..." I selected "Copy shortcut" ("Copy link location" in FireFox).
Then I passed the link location as the argument to wget, like this:

C:\>wget [url="http://forums.xisto.com/no_longer_exists/--  [url="http://forums.xisto.com/no_longer_exists/;           => `NATURE-SadTree_1024x768.jpg'Resolving art.gnome.org... 209.132.176.176Connecting to art.gnome.org[209.132.176.176]:80... connected.HTTP request sent, awaiting response... 302 FoundLocation: [url="http://forums.xisto.com/no_longer_exists/404.png; [following]--13:47:58--  [url="http://forums.xisto.com/no_longer_exists/404.png;           => `NATURE-SadTree_1024x768.jpg'Resolving ftp.gnome.org... 130.239.18.137Connecting to ftp.gnome.org[130.239.18.137]:80... connected.HTTP request sent, awaiting response... 200 OKLength: 628,569 [image/jpeg]100%[====================================>] 628,569      120.84K/s    ETA 00:0013:48:03 (118.68 KB/s) - `NATURE-SadTree_1024x768.jpg' saved [628569/628569]

This made it possible for me to use an FTP client log into the ftp.gnome.org machine, cd into the /pub/GNOME/teams/art.gnome.org/backgrounds directory and do a mass download of the pictures I was interested in, using the ftp commands:

prompt offmget *1024x768.jpg

which saved me a lot of clicks.

How-to #2 - Parallel downloads (poor man's download manager)

Supposing you have a long list of hot links to picture files, movie files, PDFs or other types of files.
(By the way, megaupload, rapidshare, etc. links do not qualify as hot links since they require captchas)

<link #1>
<link #2>
...
<link #n>

you simply place the string 'start wget ' in front of each one of them, like this:

start wget <link #1>start wget <link #2>...start wget <link #n>

and paste the resulting script into an empty .bat file.
You then run the .bat file in a command prompt window, from within the directory where you want your files to go.
Note: this has the potential to max out your bandwidth, so be careful.

I hope you find it helpful.

-Dan