linux mirrors

Here's how I do my mirrors

   1. rsync only
   2. script for each mirror
   3. runs once a day to update local mirror

Here's an example of ubuntu.

    #!/bin/sh
    # mirror-script.sh
    # hal canary
    DATE=`/bin/date +%Y-%m-%d`
    OUTDIR='/tmp/mirror'
    MIRROR=/data/mirror
    [ -d $OUTDIR ] || mkdir -p $OUTDIR
    OUTFILE=$OUTDIR/mirror-ubuntu-output-$DATE.txt

    /bin/nice /usr/bin/rsync --verbose --progress \
       --stats --archive --partial \
       --exclude dists/warty/ \
       --exclude dists/warty-security/ \
       --exclude dists/warty-updates/ \
       --exclude *powerpc/ \
       --exclude *powerpc.*/ \
       --exclude *powerpc.deb \
      mirror.mcs.anl.gov::ubuntu $MIRROR/ubuntu/ \
>> $OUTFILE

The top section are the variables. These should be changed to match your
system. The middle section with "--exclude" lines tells rsync not to
grab that directory or file. In this case I don't want warty and I don't
want anything with powerpc in it. The line "mirror.mcs.anl.gov::ubuntu"
is the server I'm rsyncing with and the directory I want.

The "OUTFILE" is kind of like a log file. You can place it anywhere. At
the moment I place them in /tmp but I think I'll move them to
/var/log/mirrors and rotate them. You should check on them every now and
then. It will tell you if somethings not working. I found that out with
my suse mirror. The server I was using wasn't working any more and there
was nothing in the OUTFILE. Had to use a different server.

Here's another example using suse.

    #!/bin/sh
    # mirror-script.sh
    # hal canary
    DATE=`/bin/date +%Y-%m-%d`
    OUTDIR='/tmp/mirror'
    MIRROR=/data/mirror
    [ -d $OUTDIR ] || mkdir -p $OUTDIR
    OUTFILE=$OUTDIR/suse-mirror-output-$DATE.txt

    /bin/nice /usr/bin/rsync --verbose --progress \
       --stats --archive --partial \
       --exclude 8.2/ \
       --exclude 9.0/ \
       --exclude 9.1/ \
       --exclude 9.2/ \
      mirrors.kernel.org::suse/i386/update $MIRROR/suse/i386/ \
>> $OUTFILE

I only want the updates so instead of grabbing the base directory "suse"
I grab suse/i386/update.

So far the only problem I've had is drive space. My main server has
several mirrors that take up about 140gig's. I've upgraded the server I
bring on Friday with a 200gig Seagate drive. It originally had an 80gig
which meant some distributions didn't make it to the meeting. Now I can
bring them all. I rsync the two on Friday so I bring the latest updates.

If you would like a distro on the server let me know now and I'll try to
get it before Friday. Sometimes it takes a few days to get it all.
---------
To unsubscribe, send email to <aklug-request@aklug.org>
with 'unsubscribe' in the message body.
Received on Mon Sep 19 19:28:01 2005