[wplug] Download a large number of files

[Kuzman Ganchev]

On Fri, Apr 09, 2004 at 12:00:14AM -0400, Gentgeen wrote:
> The files are located in a number of subdirectories under the URL
> http://memory.loc.gov/afc/afcomaha/audio/, each sub-directory holds a
> number of different formats for the same file.  Each sub-directory and
> file name is actually a number.  IE - the subdirectory 0300 holds
> files 0376.mp3, 0376.ram, 0376.wav, 0377.mp3, 0377.ram, --- you get
> the idea.

wget = your friend.

> Would wget be the right thing, or is there a better option for what I
> need?  After reading the man page, it looks like I could either list
> all the MP3 URLs, or download the entire directory.  Would I be able
> to use some regex to limit the download?

wget -e "robots=off" -nc -r -l 1 -A mp3,MP3 <http://dir>

-e "robots=off": ignore robots file (because you're not going to be
	crawling). 
-nc: this is really just for restarts
-r: get the index and the files underneath it
-l 1: don't recurse farther
-A mp3,MP3: get only mp3 and MP3 files (see also -R). 

Kuzman