Re: Er.... for Sendmail users......

Damien Hull wrote:
> Take a look at Postfix. It is a much better SMTP server. If spam is an
> issue take a look at Postgrey. It does something called greylisting. I
> think greylisting is a gift from the God's. I set up Postgrey for a
> client and they get almost no spam. It's much better then spam filters
> like SpamAssassin. I've been told there isn't a drop in tool for
> greylisting on sendmail.
>
> If you want to use Sendmail that's fine. However, email servers can be
> complex and Sendmail just makes it more so. Postfix starts out simple
> but has powerful tools to block out spam and provide good security. You
> can have postfix up and running with filters, etc... in minutes. I've
> tried to do this in Sendmail and I couldn't do it.
>
> If I remember correctly Sendmail has its own programming/scripting
> language. If you want to do anything fancy with Sendmail you need to
> learn it. I consider this a waste of my time. As far as I know this
> language is only used for sendmail and nothing else. I couldn't figure
> it out and gave up. That's why I want to shoot it. Sendmail needs to
> die. I don't want someone asking me to support it and I don't want to
> see new users try to install and configure it.
>
> Stay safe, secure and spam free with Postfix.
>
> Here's what I use
>
> 1. Ubuntu server as the OS
> 2. Postfix MTA
> 3. Postgrey
> 4. Procmail - filters email through SpamAssassin and ClamAV. It also
> puts it in the Inbox, SPAM folder, VIRUS folder
> 5. SpamAssassin and ClamAV
> 6. Courier-IMAP for the IMAP server
> 7. TLS/SASL and SSL for security / encryption
>
>

Interesting, a sendmail debate on the mailing list and I somehow missed
it :)

Damien, I mean no disrespect, but the solution you offer above has some
pretty major issues when it comes to serving any real amount of load.
While it may be simple for you to set up, it is nowhere close to as
efficient as a sendmail/mimedefang/cyrus system.

Here are the major problems with it:

1. Procmail. Procmail doesn't get called until the smtp processing
engine is done with the smtp transaction. This means that delivery
issues like quota expired are not handled until after the smtp
transaction is complete. When you do your spam and virus filtering at
this level then you are pretty much forced to generate a new bounce
message and send it back to the reply-to address or skip notifications
altogether. Either way, your dealing with double bounces or all of the
messages you reject go into the bit bucket.

2. Performance. By starting and stopping procmail (and probably
formail) on every message you are waisting a lot of resources on
something that really should stay in memory. This is why clamd and
spamd are run as a daemon. They are always running ready to process
mail. Now I'm sure you can argue that the difference is small, but on a
busy mail system every little bit adds up to a lot.

3. IMAP performance. Most IMAP/POP3 servers store messages in the mbox
or maildir format. Both have pretty big performance issues. The mbox
format is just a large file with the messages all concatenated together,
while maildir has a directory with a file for each message. Mbox has a
very slow startup time because the mail system has to open this huge
text file and start parsing, but once the parsing is done it can stream
the headers pretty fast. Maildir has the opposite problem. If you open
one message it can open a single text file and stream it really fast,
but if you ask it for all of your headers, maildir has to open, parse,
and close every single file in the dir. Either way, a mailbox with a
couple of thousand messages starts getting real slow. Since your using
procmail I'll assume your running mbox and either have a small mailbox
or wait for it to open.

Cyrus has the best of both worlds. It stores its messages like maildir,
but then keeps the headers and other information indexed in BDB or
skiplist format. So opening a folder, streaming headers, and opening a
file are all very fast. It also can update a search index nightly so
searching your mailbox is very fast. My IMAP server on a very modest
box is able to search the BODY of my 4525 message INBOX in about 3
seconds. I doubt Courier-IMAP could even open my mailbox that fast.

4. Efficiency. Sendmail uses milters to filter mail which means that
sendmail can do it's filtering in real time (during the smtp
transaction) against 1 or more milters. This means that you can accept
or reject the message in SMTP and never again send out a bounce message
that will almost certainly double bounce back to you. My mail systems
will either except the message and write it to disk, or they reject the
message and tell you why. Since these milters run in memory as a daemon
(and can use tcp sockets so they don't even need to be on the same host)
they perform very well. Mimedefang is a perl based milter that has a
multiplexor that keeps a number of perl filters running ready to go all
of the time. Not only is the performance very good, but since you are
washing everything though perl there really isn't anything you can't do.
 One of my mail systems takes winmail.dat files strips out the
attachments and reattaches them to the message. It's about 20 lines of
perl and only took a few hours to write. Another point of efficiency
with cyrus is that it has a daemon to accept the messages to be written
to the mailstore. This daemon listens on a unix socket and is ready to
go when sendmail connects and feeds it a message. Having all of these
things running saves a ton of IO on the system. My mail system doesn't
touch the disk until it's actually writing the message to disk.

5. Flexibility. What do you do when a spammer starts hitting you
really hard? Well if you have a mail system that can filter at the smtp
level you have some options. My mail system stores the spamassassin
score in a database and on the 5th message in a 24 hour period it looks
up the score and starts rejecting inbound messages from systems known to
send spam. This takes place after the mail from: and rcpt to: so users
that want to whitelist still can, but in the mean time a spammer that
wants to send me 50000000 messages only gets 5 per day. Any attempts
maid after the first 5 cost me one database and one ldap lookup which is
nothing compared to procmail/formail/spamassassin/etc. If your mail
system doesn't filter during smtp then you pretty much only have 2
options, live with the abuse, or firewall the spammer and live with the
false positives.

As far as ease of use, yea, initial setup is easier, but after you have
a couple of complex mail problems to solve sendmail/mimedefang is much
easier in the end. It's kinda like windows, setting it up is trivial,
but everything else (like backups) is a major pain.

I should also point out that m4 config files are really a good idea.
How many times have you upgraded a software package to find out the old
config file doesn't work with the updated version? Your left to go
figure out why it doesn't like the old config file. The problem is even
worse in a system like sendmail because much of how it routes mail isn't
in the binary, it's in the config file. The solution: m4 macros. With
M4 you state that you want some feature like a RBL with one line of
config and it automatically includes everything needed to make it work.
 If a new version of sendmail comes out, you just remake your config
file from the mc macro file and you get the latest config file and
features. Here is an example to illustrate my point. Suppose you want
to make sendmail only except messages for email addresses that live in
your Active Directory LDAP server. You write the config to make
sendmail lookup each address against LDAP, then put it in your mc file,
then on every future version of sendmail you simply `m4 sendmail.mc >
sendmail.cf` and your done. This is *WAY* simpler than the qmail way of
doing things which is to patch the source (because there is no way to
add this stuff to the config) then make sure your patch works with all
of the other patches you run. I would much rather run the m4 command
and have a very flexible config file than keep up with a zillion patches
to make the binary work the way you need it to.

Hopefully this wasn't such a long boring read (hopefully it even made
sense), but I did want to point out that procmail/mbox is really not the
best way to go when dealing with any significant amount of email load.

later,
schu
---------
To unsubscribe, send email to <aklug-request@aklug.org>
with 'unsubscribe' in the message body.
Received on Mon Aug 28 16:23:18 2006