[wplug] Download a large number of files

Robert E. Coutch robert.coutch at verizon.net
Fri Apr 9 02:09:02 EDT 2004


kget  is the graphical KDE (3.1) frontend for wget. Check it out too.

-Bob

On Friday 09 April 2004 01:40 am, Kuzman Ganchev wrote:
> On Fri, Apr 09, 2004 at 12:00:14AM -0400, Gentgeen wrote:
> > The files are located in a number of subdirectories under the URL
> > http://memory.loc.gov/afc/afcomaha/audio/, each sub-directory holds a
> > number of different formats for the same file.  Each sub-directory and
> > file name is actually a number.  IE - the subdirectory 0300 holds
> > files 0376.mp3, 0376.ram, 0376.wav, 0377.mp3, 0377.ram, --- you get
> > the idea.
>
> wget = your friend.
>
> > Would wget be the right thing, or is there a better option for what I
> > need?  After reading the man page, it looks like I could either list
> > all the MP3 URLs, or download the entire directory.  Would I be able
> > to use some regex to limit the download?
>
> wget -e "robots=off" -nc -r -l 1 -A mp3,MP3 <http://dir>
>
> -e "robots=off": ignore robots file (because you're not going to be
> 	crawling).
> -nc: this is really just for restarts
> -r: get the index and the files underneath it
> -l 1: don't recurse farther
> -A mp3,MP3: get only mp3 and MP3 files (see also -R).
>
> Kuzman
>
>
> _______________________________________________
> wplug mailing list
> wplug at wplug.org
> http://www.wplug.org/mailman/listinfo/wplug




More information about the wplug mailing list