Re: Er.... for Sendmail users......

From: adam bultman <adamb@glaven.org>
Date: Mon Aug 28 2006 - 17:30:02 AKDT

Apart from a few spelling errors, I mostly agree.

I historically have disliked Cyrus simply because as of the last time I
tested, it stored mail in randomly generated folder names - usually
integers. So, in my reader I might have 'spam', but on the imap server
it might be named '4'. That's kinda dumb. Maildir at least has the
folder names similar.

One thing I must talk about, though: Sendmail is slow in general.
Qmail will accept, handle, and deliver mail far faster than sendmail
does because it is a number of small, discrete binaries. I ran a couple
sendmail servers at my first job, a few qmail servers at my second job,
and now I run sendmail servers at my current position. In terms of raw
speed, no offense, but qmail is to the rocket as sendmail is to the
rickshaw with square wheels. If I do get spammed hard, qmail will
always handle it more intelligently than sendmail. Hands down.

Qmail is an enormous pain in the ass to maintain; especially if you want
to add functionality. "I want ldap now" means you have to completely
recompile Qmail, if you want to add RBL, that's a recompile, etc. And
recompiling is a pain, and isn't always foolproof. (But qmail has
vmailmgr, which is swell)

I'm working on a solution to my mail woes at work right now (sendmail
with shared mailspools from a netapp) and I'm at the very least going to
go with maildir or cyrus - probably cyrus now that Matt's made his case.
(Converting from mbox to Cyrus, and implementing quotas on cyrus is a
whole other ball of wax, o' course).

I'm tempted to use postfix, or qmail - and avoid sendmail because it is
large and heavy, despite the configurability (I've made a few changes to
our sendmail.cf file already, including adding the double bounce to bit
bucket thing).

When sendmail has a problem, if I want to tinker, I have to shut it
down. If qmail is having a problem, I can stop the daemon (qmail-smtpd)
so it doesn't accept messages while taking care of the queue
(qmail-local, qmail-remote). I can tweak the number of incoming
connections, outgoing connections, etc, and handle things a little more
delicately and smoothly and not have outages.

Plus (and this is my biggest thing) I don't have the sendmail "Oh, Crap,
I don't know what to do so I'm deleting incoming mail" error. Qmail has
the sense, when it's misconfigured, to either not start at all, or to
save incoming mail to the queue so i don't lose messages.

I'll let you know how I do - but if you have more stories, I'd love to
hear them. I've got to make a decision and quickly, so I can start
working. Only snag: I'm on solaris.

Adam

Matthew Schumacher wrote:

>Damien Hull wrote:
>
>
>>Take a look at Postfix. It is a much better SMTP server. If spam is an
>>issue take a look at Postgrey. It does something called greylisting. I
>>think greylisting is a gift from the God's. I set up Postgrey for a
>>client and they get almost no spam. It's much better then spam filters
>>like SpamAssassin. I've been told there isn't a drop in tool for
>>greylisting on sendmail.
>>
>>If you want to use Sendmail that's fine. However, email servers can be
>>complex and Sendmail just makes it more so. Postfix starts out simple
>>but has powerful tools to block out spam and provide good security. You
>>can have postfix up and running with filters, etc... in minutes. I've
>>tried to do this in Sendmail and I couldn't do it.
>>
>>If I remember correctly Sendmail has its own programming/scripting
>>language. If you want to do anything fancy with Sendmail you need to
>>learn it. I consider this a waste of my time. As far as I know this
>>language is only used for sendmail and nothing else. I couldn't figure
>>it out and gave up. That's why I want to shoot it. Sendmail needs to
>>die. I don't want someone asking me to support it and I don't want to
>>see new users try to install and configure it.
>>
>>Stay safe, secure and spam free with Postfix.
>>
>>Here's what I use
>>
>> 1. Ubuntu server as the OS
>> 2. Postfix MTA
>> 3. Postgrey
>> 4. Procmail - filters email through SpamAssassin and ClamAV. It also
>> puts it in the Inbox, SPAM folder, VIRUS folder
>> 5. SpamAssassin and ClamAV
>> 6. Courier-IMAP for the IMAP server
>> 7. TLS/SASL and SSL for security / encryption
>>
>>
>>
>>
>
>Interesting, a sendmail debate on the mailing list and I somehow missed
>it :)
>
>Damien, I mean no disrespect, but the solution you offer above has some
>pretty major issues when it comes to serving any real amount of load.
>While it may be simple for you to set up, it is nowhere close to as
>efficient as a sendmail/mimedefang/cyrus system.
>
>Here are the major problems with it:
>
>1. Procmail. Procmail doesn't get called until the smtp processing
>engine is done with the smtp transaction. This means that delivery
>issues like quota expired are not handled until after the smtp
>transaction is complete. When you do your spam and virus filtering at
>this level then you are pretty much forced to generate a new bounce
>message and send it back to the reply-to address or skip notifications
>altogether. Either way, your dealing with double bounces or all of the
>messages you reject go into the bit bucket.
>
>2. Performance. By starting and stopping procmail (and probably
>formail) on every message you are waisting a lot of resources on
>something that really should stay in memory. This is why clamd and
>spamd are run as a daemon. They are always running ready to process
>mail. Now I'm sure you can argue that the difference is small, but on a
>busy mail system every little bit adds up to a lot.
>
>3. IMAP performance. Most IMAP/POP3 servers store messages in the mbox
>or maildir format. Both have pretty big performance issues. The mbox
>format is just a large file with the messages all concatenated together,
>while maildir has a directory with a file for each message. Mbox has a
>very slow startup time because the mail system has to open this huge
>text file and start parsing, but once the parsing is done it can stream
>the headers pretty fast. Maildir has the opposite problem. If you open
>one message it can open a single text file and stream it really fast,
>but if you ask it for all of your headers, maildir has to open, parse,
>and close every single file in the dir. Either way, a mailbox with a
>couple of thousand messages starts getting real slow. Since your using
>procmail I'll assume your running mbox and either have a small mailbox
>or wait for it to open.
>
>Cyrus has the best of both worlds. It stores its messages like maildir,
>but then keeps the headers and other information indexed in BDB or
>skiplist format. So opening a folder, streaming headers, and opening a
>file are all very fast. It also can update a search index nightly so
>searching your mailbox is very fast. My IMAP server on a very modest
>box is able to search the BODY of my 4525 message INBOX in about 3
>seconds. I doubt Courier-IMAP could even open my mailbox that fast.
>
>4. Efficiency. Sendmail uses milters to filter mail which means that
>sendmail can do it's filtering in real time (during the smtp
>transaction) against 1 or more milters. This means that you can accept
>or reject the message in SMTP and never again send out a bounce message
>that will almost certainly double bounce back to you. My mail systems
>will either except the message and write it to disk, or they reject the
>message and tell you why. Since these milters run in memory as a daemon
>(and can use tcp sockets so they don't even need to be on the same host)
>they perform very well. Mimedefang is a perl based milter that has a
>multiplexor that keeps a number of perl filters running ready to go all
>of the time. Not only is the performance very good, but since you are
>washing everything though perl there really isn't anything you can't do.
> One of my mail systems takes winmail.dat files strips out the
>attachments and reattaches them to the message. It's about 20 lines of
>perl and only took a few hours to write. Another point of efficiency
>with cyrus is that it has a daemon to accept the messages to be written
>to the mailstore. This daemon listens on a unix socket and is ready to
>go when sendmail connects and feeds it a message. Having all of these
>things running saves a ton of IO on the system. My mail system doesn't
>touch the disk until it's actually writing the message to disk.
>
>5. Flexibility. What do you do when a spammer starts hitting you
>really hard? Well if you have a mail system that can filter at the smtp
>level you have some options. My mail system stores the spamassassin
>score in a database and on the 5th message in a 24 hour period it looks
>up the score and starts rejecting inbound messages from systems known to
>send spam. This takes place after the mail from: and rcpt to: so users
>that want to whitelist still can, but in the mean time a spammer that
>wants to send me 50000000 messages only gets 5 per day. Any attempts
>maid after the first 5 cost me one database and one ldap lookup which is
>nothing compared to procmail/formail/spamassassin/etc. If your mail
>system doesn't filter during smtp then you pretty much only have 2
>options, live with the abuse, or firewall the spammer and live with the
>false positives.
>
>As far as ease of use, yea, initial setup is easier, but after you have
>a couple of complex mail problems to solve sendmail/mimedefang is much
>easier in the end. It's kinda like windows, setting it up is trivial,
>but everything else (like backups) is a major pain.
>
>I should also point out that m4 config files are really a good idea.
>How many times have you upgraded a software package to find out the old
>config file doesn't work with the updated version? Your left to go
>figure out why it doesn't like the old config file. The problem is even
>worse in a system like sendmail because much of how it routes mail isn't
>in the binary, it's in the config file. The solution: m4 macros. With
>M4 you state that you want some feature like a RBL with one line of
>config and it automatically includes everything needed to make it work.
> If a new version of sendmail comes out, you just remake your config
>file from the mc macro file and you get the latest config file and
>features. Here is an example to illustrate my point. Suppose you want
>to make sendmail only except messages for email addresses that live in
>your Active Directory LDAP server. You write the config to make
>sendmail lookup each address against LDAP, then put it in your mc file,
>then on every future version of sendmail you simply `m4 sendmail.mc >
>sendmail.cf` and your done. This is *WAY* simpler than the qmail way of
>doing things which is to patch the source (because there is no way to
>add this stuff to the config) then make sure your patch works with all
>of the other patches you run. I would much rather run the m4 command
>and have a very flexible config file than keep up with a zillion patches
>to make the binary work the way you need it to.
>
>Hopefully this wasn't such a long boring read (hopefully it even made
>sense), but I did want to point out that procmail/mbox is really not the
>best way to go when dealing with any significant amount of email load.
>
>later,
>schu
>---------
>To unsubscribe, send email to <aklug-request@aklug.org>
>with 'unsubscribe' in the message body.
>
>
>

---------
To unsubscribe, send email to <aklug-request@aklug.org>
with 'unsubscribe' in the message body.
Received on Mon Aug 28 17:30:37 2006

This archive was generated by hypermail 2.1.8 : Mon Aug 28 2006 - 17:30:37 AKDT