Complex Sendmail Structure
Paul Venezia
4/13/98

PROBLEM: Convert mail from PMDF to sendmail.
                      Current environment is a single mail server, running PMDF, with /var/spool/mail NFS mounted on >40 machines.
                      This must be replaced with a sendmail environment, with two servers, no NFS.

In order to convert the handling of email from PMDF to sendmail, and to completely reconfigure the mail environment, several dependencies had to be met. They were:
 
 

1) All Mail leaving the company must have usernames rewritten to first.last
2) All mail bound internally must have the sender address remain uname@company.com, but the recipient must be rewritten to first.last@company.com.
3) There must be two mail machines. One machine must handle all rewrites, aliasing, and Majordomo lists. The second machine must house the mail spool, and otherwise be ignorant of any rewrites. They will be referred to as “sender” and “reader”
4) NFS mounting the spool is not an option.
5) Using fetchmail is not an option.
6) Using only one machine to accomplish both tasks is not an option.
7) Any/all databases must be maintained automatically, rebuilding every 10 minutes.
8) Inbound mail to first.last must be delivered, but reader cannot have an aliases file.
9) Mail cannot be sent locally on reader, but must first go to sender to have addresses rewritten.
10) All Majordomo aliases, and the program itself must live on sender.
11) Special exception aliases must work correctly, even if they are amalgams of the first.last pair. 
 

SOLUTION:

To accomplish this, several methods were examined, including use of the genericstable and virtusertable features, and use of the userdb feature. Due to assorted problems with all, it was determined that a manual configuration of sendmail was required. While reading this, you may refer to the diagrams of mail pathing, found here.

First, the rewrites had to work. To accomplish this, I added two database calls into sendmail, occuring at custom rules 65 and 66. They are as follows:

S65
#; handle inbound database
R$+ < @ $=j . >         $: < $1@$2 > $1 < @ $2 . > @    mark
R$+ < @ $=j >           $: < $1@$2 > $1 < @ $2 . > @    mark
R$+ < @ $=w >           $: < $1@$2 > $1 < @ $2 . > @    mark
R$+ < @ $=w . >         $: < $1@$2 > $1 < @ $2 . > @    mark
R< $+ > $+ < $* > @     $: < $(outbound $1 $: $) > $2 < $3 >
R< > $+ < @ $+ >        $: < $(outbound $1 $: $) > $1 < @ $2 >
R< $* @ $* > $* < $* >  $@ $>3 $1 @ $2                  found qualified
R< $+ > $* < $* >       $: $>3 $1 $3            found unqualified
R< > $*                 $: $1                           not found

S66
#; handle inbound database
R$+ < @ $* . >          $: < $1@$2 > $1 < @ $2 . > @    mark
R$+ < @ $* >            $: < $1@$2 > $1 < @ $2 . > @    mark
R< $+ > $+ < $* > @     $: < $(inbound $1 $: $) > $2 < $3 >
R< > $+ < @ $+ >        $: < $(inbound $1 $: $) > $1 < @ $2 >
R< $* @ $* > $* < $* >  $@ $>3 $1 @ $2                  found qualified
R< $+ > $* < $* > $: $>3 $1 $3            found unqualified
R< > $*   $: $1                           not found

S65 will match any username@company.com  address, and match the username to a key found in the outbound.db database. The value will be a first.last pair, and the address will be rewritten as first.last@company.com.

S66 Is the exact opposite, matching first.last to usernames to rewrite inbound mail, and using the inbound.db database.

Since all mail sent to external hosts must be sent using the SMTP/ESMTP mailers,  I reconfigured the mailer sender/recipient rewrite rulesets to rewrite the envelope/headers of outbound mail to what we desired. This left the SMTP/ESMTP mailers in this configuration:

Msmtp,          P=[IPC], F=mDFMuX, S=11/32, R=21/32, E=\r\n, L=990,
                M=5000000, T=DNS/RFC822/SMTP,
                A=IPC $h
Mesmtp,         P=[IPC], F=mDFMuXa, S=11/32, R=21/32, E=\r\n, L=990,
                M=5000000, T=DNS/RFC822/SMTP,
                A=IPC $h
Msmtp8,         P=[IPC], F=mDFMuX8, S=11/32, R=21/32, E=\r\n, L=990,
                M=5000000, T=DNS/RFC822/SMTP,
                A=IPC $h

As you can see, for all mail passing through the SMTP/ESMTP mailers, the sender envelope will pass through ruleset 11,and the sender header will pass through ruleset 32. The recipient envelope, however, passes through ruleset 21, not 11. The rules are as follows:

S11
R$+                     $: $>51 $1              sender/recipient common
R$* :; <@>              $@                      list:; special case
R$*                     $: $>61 $1              qualify unqual'ed names
R$+                     $: $>94 $1              do masquerading

Ruleset 11 simply masquerades the sender envelope.

S32
R$+                     $: $>51 $1            sender/recipient common
R:; <@>                 $@                    list:; special case

# do special header rewriting
R$* <@> $*              $@ $1 <@> $2          pass null host through
R< @ $* > $*            $@ < @ $1 > $2        pass route-addr through
R$*                     $: $>61 $1            qualify unqual'ed names
R$+                     $: $>93 $1            do masquerading
R$+                     $: $>65 $1            Send to rewrite
R$+ < @ $=w . >         $: $1 < @ $M >
R$+ < @ $=w >           $: $1 < @ $M >
R$+ < @ $=j . >         $: $1 < @ $M >

Ruleset 32 passes the address through the masquerading rulesets, then through ruleset 65, which changes the uname to first.last. This is for the headers only.

This took care of all external mail. Unfortunately, you can’t redirect mail to another mailer once one has been selected, and we couldn’t use the SMTP/ESMTP mailers to send mail to reader, since that would have rewritten the headers in an incorrect fashion. In addition, we couldn’t do any rewriting on reader, so the problem was how to send mail to reader, and have it accept the mail, and deliver it locally, when it was also supposed to send all local mail to sender. This would have created a mail loop.

To fix this, I first specified a MAIL_HUB host on sender, meaning that all mail destined locally should actually be sent through the relay mailer, to the host defined in the DH variable. This host was reader.
Then, I altered the relay mailer to conform to the header rewrite needs of internally-bound mail. This was as follows:

Mrelay,         P=[IPC], F=mDFMuXa8, S=11/31, R=22/32, E=\r\n, L=2040,
                T=DNS/RFC822/SMTP,
                A=IPC $h

This caused the sender header/envelope to simply be masqueraded, but not rewritten. The recipient header, however, passes through the same ruleset as external mail, causing uname to be translated into first.last. In addition, the recipient envelope passes through ruleset 22, which is as follows:

S22
R$+                     $: $>51 $1             sender/recipient common
R$+                     $: $>61 $1             qualify unqual'ed names
R$+ . $+ < @ $=w >      $: $>33 $1 . $2 < @ $M . >
R$+ . $+ < @ $=w . >    $: $>33 $1 . $2 < @ $M . >
R$+ < @ $* >            $: $1 .LCL. < @ $2 >

This causes the envelope address to be passed to ruleset 33, which is this:

S33
R$+                     $: $>51 $1             sender/recipient common
R:; <@>                 $@                     list:; special case

# do special header rewriting
R$* <@> $*              $@ $1 <@> $2           pass null host through
R< @ $* > $*            $@ < @ $1 > $2         pass route-addr through
R$*                     $: $>61 $1             qualify unqual'ed names
R$+                     $: $>93 $1             do masquerading
R$+ . $+ < @ $=w . >    $: $>66 $1 . $2 < @ $3 . >
R$+ . $+ < @ $=w >      $: $>66 $1 . $2 < @ $3 . >
R$+ . $+ < @ $=j . >    $: $>66 $1 . $2 < @ $3 . >
R$+ < @ $=w . >         $: $1 < @ $M >
R$+ < @ $=w >           $: $1 < @ $M >
R$+ < @ $=j . >         $: $1 < @ $M >

Ruleset 33 masquerades the envelope, then passes it on to ruleset 66, shown above, that rewrites the first.last pair (if one exists) to be rewritten to uname. This is so local delivery can happen on reader.

Now, since reader is configured to send all mail to sender, and sender is configured to send all mail to reader, we have to break the loop. To do this, I instituted a “cookie” into the address. Sender adds the .LCL. cookie to the end of the recipient envelope address (Seen in the last line of ruleset 22). This will still allow the mail to be accepted by reader, but with the cookie in place, it cannot be delivered locally.

So far, we have correctly rewritten the headers to:

From: uname@company.com
To: first.last@company.com

The envelope looks like this:

From uname@company.com
To uname.LCL.@company.com

Now, we must catch the cookie on reader, and deposit it in the appropriate mailbox.
This is the only configuration change made to the sendmail.cf file on reader:

S98
R$* .LCL. < @ $* >      $#droplocal $: $1 < @ $2 >

This will catch the cookie, and pass the email directly to the specified mailer, droplocal.

Droplocal is as follows:

Mdroplocal,   P=/usr/libexec/mail.local, F=lsDFMAw:/|@qrmn9, S=10/30, R=20/40,
              T=DNS/RFC822/X-Unix,
              A=mail -d $u

This mailer is essentially identical to the local mailer, except that it does NOT call ruleset 5. Ruleset 5 is where the local mailer determines the relay/mailhub host, and redirects the mail. This is accomplished by removing the 5 in the droplocal mailers’ F flags section.

Once this rule has stripped the .LCL. cookie from the address, the envelope address is correct, and the droplocal mailer deposits the mail into the correct mailbox. This is what allows the two servers to swap mail back and forth without causing a mail loop.

One of the major problems with other solutions was the fact that aliases must be expanded, and then separately passed through all rules before all rewrites happen. Otherwise, blindly rewriting all inbound mail headers will result in aliases never being expanded. If the above solution is used, then aliases have been expanded well before any rewriting has been done. This allows Majordomo lists, and other mailing list aliases to work correctly. If an alias contains a period, it is ignored, and the envelope is rewritten to the RHS of the alias, the recipient header is left alone, allowing the receiving user to observe that the mail was sent to a non-standard address. Mail that is addressed incorrectly (i.e. to a non-existent user) is received and rewritten by sender, but when it’s passed to the droplocal mailer on reader, it is determined that it is not actually a user, and the basic “unknown user” error is generated, and sent to the sender of the message.

As far as users that POP/IMAP their mail, they should use reader as their POP/IMAP host. They should send all mail to sender, as that will remove a step from the mail path.

Minutiae:

All databases are hashed, and in the form of:

Inbound.db
first.last  uname

Outbound.db
uname  first.last

To create the databases, the command is simply

`makemap hash /etc/inbound.db < inbound.map`

This will cause the contents of inbound.map to be compiled into the database.

Caveats:

No username can contain periods.
No username can contain the cookie, .LCL.