Just felt like sharing a funky repository proxy setup.

From: Shane Spencer <shane@bogomip.com>
Date: Wed Jan 31 2007 - 10:56:47 AKST

I have a strange attraction to approx the caching apt repository
proxy. It is a small web server that listens on port 9999 by default
and allows you to define specific virtual directories that map to http
hosts somewhere else in the universe. Apache/lighttpd/etc.. have
similar functionalities via mod_proxy with mod_cache. The few
advantages approx has over these solutions is a small install base
using its own HTTP server, caching is done in organized directory
layouts which you can pre-feed manually to jump start the package
cache, garbage collection via a cronjob which clears out all
expired/unreferenced packages.

My personal approx configuration has the following feeds defined:

...
debian http://ftp.us.debian.org/debian
debian-security http://security.debian.org
...
ubuntu http://archive.ubuntu.com/ubuntu
ubuntu-security http://security.ubuntu.com/ubuntu
...

typically you would use http://proxyhost:9999/debian as the mirror. I
don't prefer that for my own reasons, this is where the funky begins.

I couple this with squid and jesred which can use regular expressions
to rewrite or redirect matched requests attempting to leave the
network through squid.

I added the following to squid to make jesred the redirect program:

...
url_rewrite_program /usr/lib/squid/jesred
...

and added the following to jesred.rules after setting it to run in my
environment:

...
#Debian
regex ^http://ftp.us.debian.org/debian/(.*)\.(.*)
http://10.0.0.2:9999/debian/\1.\2
regex ^http://security.debian.org/(.*)\.(.*)
http://10.0.0.2:9999/debian-security/\1.\2
...
#Ubuntu
regex ^http://archive.ubuntu.com/ubuntu/(.*)\.(.*)
http://10.0.0.2:9999/ubuntu/\1.\2
regex ^http://us.archive.ubuntu.com/ubuntu/(.*)\.(.*)
http://10.0.0.2:9999/ubuntu/\1.\2
regex ^http://security.ubuntu.com/ubuntu/(.*)\.(.*)
http://10.0.0.2:9999/ubuntu-security/\1.\2
...

I just set up ftp.us.debian.org as the mirror for ubuntu. You could
make this anything really. What exactly does this bring to the table?
 No more accessing port 9999 now that squid handles the proxying that
can access it. I can now see the contents of the repository
directorys via the repository web server because I am only rewriting
the URL for anything that matches a (filename).(ext). Everything is
transparently cached and cleaned up when it expires.

Example output from rewrite logs:

...
1170225462.458 10.0.0.20/-
http://us.archive.ubuntu.com/ubuntu/pool/universe/s/ssmping/ssmping_0.8.1-1_i386.deb
http://10.0.0.2:9999/ubuntu/pool/universe/s/ssmping/ssmping_0.8.1-1_i386.deb
4
...
1170261387.705 10.0.0.20/-
http://us.archive.ubuntu.com/ubuntu/dists/edgy-updates/restricted/source/Sources.bz2
http://10.0.0.2:9999/ubuntu/dists/edgy-updates/restricted/source/Sources.bz2
4
...

This may not be ideal for everybody, I love it however :)

Take care,

Shane
---------
To unsubscribe, send email to <aklug-request@aklug.org>
with 'unsubscribe' in the message body.
Received on Wed Jan 31 10:57:04 2007

This archive was generated by hypermail 2.1.8 : Wed Jan 31 2007 - 10:57:04 AKST