[wplug] Download a large number of files

Gentgeen gentgeen at linuxmail.org
Fri Apr 9 00:00:14 EDT 2004


I want to download a large number of mp3s without having to click and save each file individually within my web browser.  

Before you all go "legal" on me, there is nothing illegal going on here.  They are pow-wow records found at the Library of Congress website (http://memory.loc.gov/afc/afcomaha/audio/)  The web interface is at http://memory.loc.gov/ammem/omhhtml/ - check it out - really great stuff if you are into Native American stuff.

Anyway, enough off topic stuff -- 

The files are located in a number of subdirectories under the URL http://memory.loc.gov/afc/afcomaha/audio/, each sub-directory holds a number of different formats for the same file.  Each sub-directory and file name is actually a number.  IE - the subdirectory 0300 holds files 0376.mp3, 0376.ram, 0376.wav, 0377.mp3, 0377.ram, --- you get the idea.

I would like to download all the mp3 files found in the subdirectories, and I need to be able to restart the service should I lose the connection, or have to disconnect (I am on dial-up).  I know it will take some time.  Ideally, it would be something that I could start when I go to bed one night, and just let it go.  Hence the need to restart/stop the download when necessary.

Would wget be the right thing, or is there a better option for what I need?  After reading the man page, it looks like I could either list all the MP3 URLs, or download the entire directory.  Would I be able to use some regex to limit the download?



More information about the wplug mailing list