Archive-name-to-be: mail/procmail-smtp-filtering
URL:                http://www.professional.org/procmail/smtp.html
Last-modified:      Fri, 01 Feb 2002 09:24:48 GMT
Posting-frequency:  Provided in response to questions about filtering outbound
                    or remotely retrieved messages using procmail.

If you would like to receive the FAQ by electronic mail, you can cut and paste this WWW document and mail it to yourself.

Question: how can I use procmail to filter sent, or outbound, mail? Alternatley, how can I use procmail to make a copy of all sent mail?

Answer:

Procmail is an LDA - Local Delivery Agent. It is invoked either from your MTA (Mail Transfer Agent, commonly Sendmail) when your MTA has determined the message it is handling is destined for a local user, or when the MTA attempts local delivery itself, and discovers a .forward file for the local user, which contains appropriate commands to invoke procmail. Which method is used depends on the configuration of your mail server. Using procmail as the LDA is considerably easier and affords the option of system-level filtering to the administrator, as well as automatic filtering for users who merely create a properly secured .procmailrc file in their home directories. Beyond that, the procmail scripts themselves are indifferent to the mechanism used to spawn procmail.

Procmail may also be manually invoked by a user or script, to process a mailbox file. This method is detailed in the notes in my disclaimer on how to properly test a script before putting it into production use.

As such, procmail is NOT invoked for outbound messages, at least not by your MTA. Some mail clients (run ON the host itself), such as mailx, pine, and mutt, have options to invoke external processes when you go to send a message. Using such a mechanism may permit you to provide outbound mail filtering on a user by user basis, but it will require the use and proper configuration of specific mail agents, and the filtering occurs within the user's account, not at the system level. Users who submit messages to the server via an SMTP connection (say, via a Windows or Macintosh mail client) will not be subject to such outbound mail filtering. Nor would messages sent by a webserver.

To filter all SMTP messages, one would have to hack your SMTP agent to invoke procmail filtering for all messages. 'man procmail' (at least on recent releases) actually provides example sendmail configuration to allow for invoking procmail directly from within sendmail rulesets. While this author has conducted successful experiements with sendmail rule invoked procmail processes (both to archive, filter, and otherwise modify OUTBOUND messages or to perform system-wide SMTP-time handling of inbound messages), such endeavours are not well documented as part of the procmail package because they rely heavily on the MTA configuration - the point is that procmail *CAN* be invoked from the MTA, but that you need to get under the hood of your MTA config script. See the links at the bottom of this document for more information on sendmail. Of course, if you're using a different MTA, you'll need to research whether it has similar capabilities.

Although this author has experimented with such filters himself, he cannot provide you with a ready-made configuration file hack. The procmail manpage really does contain about all you need to successfully add an MTA-level filtering hook.

Note that unless your firewall prohibits other hosts from using port 25 (SMTP), or for that matter, alternate ports on which SMTP may be configured to run on other hosts (which YOU can't control on other systems), you will not be able to prohibit inside users from accessing outside SMTP servers to send their email. The same goes for messages sent directly from your mail server, unless you can prohibit users from running other mail agents or scripts which may use an external mail server to send messages rather than invoking the local MTA.

Consider carefully whether you really want to make copies of outgoing messages, and for what reason. Such actions open up all manner of privacy concerns, even in the workplace.

Question: I want to put a disclaimer tag on all of the messages sent by our server, like all those legal firms do.

Answer:

First, see the previous question - you won't be achieving this strictly with procmail - you'll need to hack your MTA or invoke a script when using a local mailer program. It is doable.

Second, while the MTA is the proper place to hack for outbound messages, you should read Q 3.35 in the sendmail FAQ. Bummer, eh?

I include this question specifically because I'd like to point out how asinine such disclaimers are: most either succeed in labeling the employees of the firm as idiots who cannot properly address a message ("If you receive this tranmission in error..."), or assume that if someone actually did receive a message in error, which they could personally benefit from (instead of just ignoring it as a misaddressed message), that they would actually abide by the terms of the disclaimer instead of acting upon this inside information they're now in posession of. From a legal standpoint, how do you prove that they didn't delete it as soon as they read the subject line?

Really though, the disclaimers are usually quite bulky, and might not even be in the native language of the unintended recipient. Further, because they invariably appear at the BOTTOM of the message, the unintended recipient will have already read the entire message before seeing the disclaimer. What dictates that they read the whole message before taking action on something interesting in it?

The email disclaimers grant you zero proven legal protection -- there has yet to be any legal precedent to demonstrate that the disclaimers can be enforced, and as procmail list members have pointed out, companies employing these anoying disclaimers in email do not place a similar disclaimer on voicemail messages, regular mail correspondance (where they must rely on the POSTAL SERVICE to deliver the message properly!) or every fax transmission - if they are not consistent in their application of such disclaimers amongst all the tranmission medias which they may convey private and confidential information, how well do you figure they'll fare in court?

A far better solution would be to require all of your employees to use PGP (or similar) encryption on all correspondance which is deemed private and confidential. Should the message be addressed incorrectly, the unintended recipient will not be able to read the message because the message would have been encrypted with the public key of the intended recipient, which the unintended recipient shouldn't have the private key for. Such a solution of course requires that you have employees and clients capable of mustering the limited brainpower necessary to deal with an easy-to-use encryption package. In exchange for this, you get ACTUAL security with your email, as well as digital signature capability. For messages which are not confidential (such as messages to public mailing lists), you don't end up annoying the hundreds or thousands of other list subscribers who get a copy of this meaningless disclaimer on every post you make about something which is intended for a large group of people and which is neither private nor confidential, and thus not subject to the disclaimer.

As has been raised on the procmail list many times, simply tacking a disclaimer onto the end of a message does not ensure that it will be visible to the recipient. If you tack it onto the end of a MIME formatted message, it won't be visible to those using MIME mail readers. If you attempt to tack it on within a MIME block, you'll break messages sent in PGP/MIME (i.e. signed messages).

Question: How can I use procmail to fetch mail from a remote server to process it locally?

Answer:

Procmail doesn't fetch mail. There is a completely separate program, not developed by or in conjunction with the procmail program, called, of all things, fetchmail, which accomplishes the task of retrieving messages from remote systems and handing them to the MTA of the local system, from which procmail would then be invoked.

Procmail filters mail on the LOCAL system. It doesn't retrieve messages from other systems, but is rather INVOKED locally and handed a message to process. Procmail doesn't speak E/SMTP either - when sending messages, it hands them to the MTA (typically sendmail) to be delivered.

As for fetchmail, if you have multiple POP mailboxes with your ISP (say, one for each user on your local system), you'll have to deal with setting up multiple fetchmail configurations on your server (these can all be handled from one fetchmail configuration -- though it will need the passwords for each of the POP or IMAP mailboxes which it must retrieve messages from).

If your network connection is a dedicated, routeable IP, an easier solution is to set up your DNS to specify your server as the promary MX for your own domain, and have mail delivered DIRECTLY to it. This gives you near unlimited number of POP accounts (as versus some cap most ISPs will make for hosted POP, and/or a service charge they'll hit your with for setting up a new POP account), more efficient mail transmission and reception (multiple local recipients will involve ONE copy of a message being sent to the system, instead of multiple messages being fetched). It also means that when someone locally sends a message to other local users, they would receive it via the local server, rather than involving transmitting it to an outside server, and then fetching it back to the local one. With a local MTA, you have the capability to refuse mail based on blacklists (RBL, RSS, DUL, sendmail access database, etc), which can significantly reduce your junkmail volume.

All this is well outside of the scope of the procmail list, which deals with the configuration and scripting of the procmail program itself, which is a local delivery agent, not an MTA or POP daemon.

Further references:

www.sendmail.org and news:comp.mail.sendmail, among others.