Welcome, Guest. Please login or register.

Author Topic: How to backup or download this?????  (Read 4320 times)

Description:

0 Members and 1 Guest are viewing this topic.

Offline Piru

  • \' union select name,pwd--
  • Hero Member
  • *****
  • Join Date: Aug 2002
  • Posts: 6946
    • Show all replies
    • http://www.iki.fi/sintonen/
Re: How to backup or download this?????
« on: March 30, 2007, 04:07:01 PM »
This will allow you to get at least some of the files:

wget --user-agent "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705)" http://web.archive.org/web/20040415065133/www.nethkin.com/bmori/amiga/dos1.html --output-document - | perl -p -e 's/\/\/www.nethkin.com/\/\/web.archive.org\/web\/20040415065133\/www.nethkin.com/g' | wget --input-file - --force-html --user-agent "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705)" --convert-links --force-directories --no-host-directories --cut-dirs 3 --wait 20 --random-wait

The pages will appear in bmori/amiga/ directory (and subdirectories).

Note that archive.org has robots.txt file that if followed prohibits apps from recursively grabbing content. In this case I've added "--wait 20 --random-wait" to make the leeching less distruptive. Downloading takes longer, but shouldn't piss off archive.org admins.

I know this is far from perfect solution, but at least it works somewhat (without need for downloading everything by hand).