[aklug] Re: Linux Counter Project

Arthur Corliss wrote:
> On Thu, 24 Sep 2009, Shane R. Spencer wrote:
>
>> Now here's a project for AKLUG.. Create a better machine-update script
>> that doesn't rely on SMTP but instead uses HTTP first. Something that
>> can auto-generate a new machine ID as well. perl-mechanize or
>> python-mechanize would work durn well.
>
> You're overlooking the obvious benefit of the SMTP method: built-in
> queue'ing mechanism, so you can batch process updates as you get the
> cycles,
> rather than overload an already overloaded web server.
>
> In any event, I'm sure Harald would consider any code donations, especially
> given the paltry # of machines actually running the script. I don't run it
> myself.
>
> --Arthur Corliss
> Live Free or Die
If I'm not misunderstanding, you're stating that most SMTP servers hold
off on sending messages that are queued once per day until cycles are
available? If I'm not mistaken the SMTP server is as available for
processing cycles as any other program. Publishing to HTTP directly
from the machine update script would consume far less resources than
running an SMTP server. When compared with a daemon a script that
publishes every to a server every few minutes would appear to have more
"loading" time per day than a daemonized process that is already
resident and ready to rock and or roll. However that process wouldn't
have to write to disk at all. Since this script runs daily, I'd say
it's mostly harmless to publish directly to HTTP and take that CPU hit
of loading a script interpereter for 2 seconds.

In nearly every workstation inside my company SMTP is not an installed
service. Even if it were none of them could send mail directly to the
Internet without first authenticating to our SMTP gateway. We do not
offer transparent SMTP access to the Internet, infact most ISPs don't
even offer it.

Of course I could set SMTP up on those machines, but I don't require
have any SMTP requirement on any of those workstations. The servers are
a different story of course.

HTTP on the other hand has an expectation to "Just plain work", a proxy
environment variable 'http_proxy' can be used to authenticate to a proxy
first as well.

So, I'm obviously not hip to using SMTP for data collection. In other
scenarios where I need to do mass data collection HTTP has always been
the most reliable means.

If his code base is open to review I'd like to take a peek. This is
similar to what I want to do with feedthetree.org and listigi.com which
is collect lots of data and form a large normalized metadata database. :)

BTW... This dovetails nicely with a problem I am having with people
here at work thinking they should just be able to use port 25 for SMTP
service wherever they are in the world without worry. No StartTLS, No
SSL. On top of that their client, Entourage, blindly trusts SMTP
servers by not requiring StartTLS when using port 25. The headaches
this mentality and that specific bag of crud software has caused may
have assisted the above response. I've already solved this problem by
requiring mobile users to use OpenVPN over port 443 where once again the
port is expected to be available to nearly every Internet user and would
be too difficult to force through a transparent proxy. (Alternatively I
may end up using AutoSSH and port forwarding to avoid subnet collisions)

- Shane

-- Attached file included as plaintext by Ecartis --
-- File: signature.asc
-- Desc: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkq7p0cACgkQXK/vGhypreK5kQCcCPoKqP4EnTFgWaH4xRHbcmKw
ig0AniMBtFYyqVbfDaMDqS50prc5gAkC
=Tpq0
-----END PGP SIGNATURE-----

---------
To unsubscribe, send email to <aklug-request@aklug.org>
with 'unsubscribe' in the message body.
Received on Thu Sep 24 09:08:16 2009