[aklug] Re: recursive HTTP file downloading...

From: Arthur Corliss <acorliss@nevaeh-linux.org>
Date: Thu Jun 10 2010 - 11:52:18 AKDT

On Thu, 10 Jun 2010, Leif Sawyer wrote:

> Supposedly, wget supports the robots =3D off line in .wgetrc which
> will ignore the robots.txt file restrictions.
>
> or
>
> wget -e robots=3Doff
>
> So, no need to hacks it.

Thanks for the heads up! The man page didn't mention that specifically.
And not being a bit wget user I didn't bother to check all the available
wgetrc options...

         --Arthur Corliss
           Live Free or Die
---------
To unsubscribe, send email to <aklug-request@aklug.org>
with 'unsubscribe' in the message body.
Received on Thu Jun 10 11:52:26 2010

This archive was generated by hypermail 2.1.8 : Thu Jun 10 2010 - 11:52:26 AKDT