From Wiki99

↑ Computers ↑
← prev: Mail Theory next: Mail ssh Setup →

---

Contents 1 Postfix 1.1 Postfix preferences 1.1.1 /etc/postfix/main.cf 1.1.2 postfix aliases 1.1.3 postfix connecting to authenticated SMTP 1.2 Procmail 2 Installation And Setup of binc-imap 3 System startup issues 3.1 postfix 3.2 binc imap (and xinetd) 3.3 binc imap (and launchd) 4 Testing 4.1 Testing postfix 4.2 Testing imap 4.3 Testing using a local (ie on the server) Mail.app 5 Improving the performance of Mail

Postfix

postfix ships with MacOS X Tiger, so we don't need to download any installer. However we do need to configure it. Here there are two choices.

The first choice is you can download the program Postfix Enabler. This provides a GUI for configuring postfix.
I, however, would not recommend this. My experience with Postfix Enabler was that I couldn't quite get it to configure things the way I wanted, so I had to learn about the low-level details anyway. The other problem with Postfix Enabler is that, if you aren't careful, it will install the despised UW-imapd.

So better is just to configure postfix by hand. This is not as hard as it might seem, for the simple home server we want to set up, and you'll learn a lot along the way about how UNIX works in general.

We want to configure postfix in three ways:

1. We need to tell postfix what to do when it is running, where should it send outgoing mail, security settings, that sort of thing.

2. We need to have postfix deliver incoming mail to somewhere binc-impap will see it.

3. We need to make sure that it starts up automatically whenever our server is rebooted. We dealt with this same sort of issue earlier when talking about dnsupdate.

Postfix preferences

/etc/postfix/main.cf

The standard for system-wide UNIX programs is to store their configuration/preference files in the directory /etc. Postfix's configuration files live in /etc/postfix. The main file is named main.cf. You might want to quickly look at this using something like

The file follows a fairly common pattern for UNIX preference files. The file allows comments (everything from a # character to the end of a line is a comment), and tries to list all the options the file understands along with an explanation. One thing that is probably not obvious is that you don't need to modify the file at the location where a setting first appears. It's often easiest is to put all your modifications at the very end of the file, so that they're all in one place and you can clearly distinguish changes you made.

I'll describe the modifications I made to my file, and you should be able to figure out the equivalents that you need for your file. Remember, of course to save a backup of this file before you do anything serious. So:

scroll to the very bottom of the file, and add the lines

Let's go through these.

• We need to specify mydomain so that postfix knows that mail it receives addressed to someone@bluecloud.com should be sent to an account on this computer. All mail received that is addressed to any other domain, someone@somedomain.com, will be sent to our ISP's mail server to deal with. That mail server will figure out what to do next.

• It's too complicated to discuss why both mydomain and myhostname exist; all that matters for us is that for our tiny network with only one server visible to the internet they should have the same value.

• When mail comes in that is addressed to someone@bluecloud.com, postfix assumes that someone is the name of a standard MacOS X UNIX account on the computer.
  So lets suppose my MacOS X short account name is mjh, then mail addressed to mjh@bluecloud.com will be saved to be picked up by me.
  Now what should happen if mail comes in addressed to a name which does not correspond to a user account on this server?
  One possibility is to just throw that mail away.
  A second possibility is to send a reply to the sender of the mail along the lines of "This is an invalid email address. No such user."
  A third possibility, and this is the one we'll use, is to say that all mail addressed to an unknown account will be sent to me, the sysadmin.
  It is the setting luser_relay that specifies where this unknown-account mail should be sent.

• The lines alias_maps and alias_database are technical details that are of no interest; all that matters is that you need these lines written in exactly this way.

• The line mynetworks_style is the security setting that protects us from spammers sending out mail through our computer. It means that mail that is sent to this computer that is addressed to some other computer on the internet, ie that only uses our computer as a temporary router, will only be accepted if it was generated on this computer itself.

• The line inet_interfaces specifies that postfix should listen for incoming mail on all available internet interfaces.
  (The default Apple config for postscript sets this to localhost only, which makes sense for certain classes of users, but not for us.)

• maximal_queue_lifetime and bounce_queue_lifetime specify how long postfix holds on to outgoing mail that it cannot deliver.
  The default for these settings is 5 days, meaning that postfix will hold onto mail, trying every so often to send it out, for 5 days. In our system, all email gets handed off to our ISP to deal with, so there should never be any delay in sending out mail, and the only thing that can go wrong is that the connection to our ISP goes down, or they change the configuration of their mail system.
  Given this, if something does go wrong with outgoing mail, I want to know right away, hence I made these values smaller.

• The line relayhost specifies where to send outgoing mail. We'll just send it to our ISP's SMTP server; that server will take care of figuring out how to route it to the final address.
  Note that you can use the IP address of your ISP's SMTP server (without using square brackets), but if you want to use the DNS name of your ISP's SMTP server (the usual case), you must surround the name with square brackets, as I did.
  The :587 specifies a particular port to use for this service, rather than the default SMTP port. It is more common to have some special dedicated port for this purpose if authenticated SMTP is being used (ie the sasl stuff in the three lines below).
  You will have to read your ISP's documentation to figure out if it is using authenticated SMTP or not, and if it is using a non-standard port for SMTP.

• The next three lines, smtp_sasl_auth_enable, smtp_sasl_password_maps and smtp_sasl_security_options are only necessary if you connect to your ISP's SMTP server using authenticated SMTP. This is not common, so you probably won't need these lines; but if your ISP does support authenticated SMTP you probably want to use it.
  The use of these lines is described a little further below.

• The final line, mailbox_command will be discussed a little further below.

postfix aliases

After filling in (and of course saving) this main.cf file, we are not fully done.

First we need to handle the aliases file. You will see, if you look in /etc/postfix, a file called aliases. This holds a list of what you would expect, aliases, ie email names that route their mail to some differently-named account. You can, if you can figure out some reason to do so, put more aliases in this list. I haven't found a reason to do so.
The reason we care about this file is that postfix does not actually refer to this file, it refers to a hashed form of it, whose path we provided when we were setting up main.cf, namely alias_database=hash:/etc/postfix/aliases. We need to create this hashed file. However the program that will do this figures out where to read the alias file, and where to write the hash file, through reading postfix preferences, so we first need to start postfix.

Now there should be a file called aliases.db in /etc/postfix. If you ever change the file aliases again, you will need to rerun the newaliases command.

postfix connecting to authenticated SMTP

There is a similar story if you are using authenticated SMTP (which, as I said, you probably are not). In this case you need to create a file called sasl_passwd, again in /etc/postfix.

Fill this with

and after saving it run the commands

The first two lines change the file so that only root can read the file (important since it contains a password). The last line is the same deal as before, creating a hash file.
Once again, if you ever change the sasl_passwd file, you'll need to re-run this command.

Procmail

The final line of main.cf, the mailbox_command, is how we route the incoming mail to the binc-imap maildir folder. Postfix has the ability to send every incoming message through a filter program rather than the default behavior (which is to dump it in /var/mail/username. This allows you, for example, to run each message through a spam filter. The standard program used for this filtering purpose is called procmail.

procmail can do a hundred different things, but for now all we care about is that it has the ability to store an incoming message in a maildir format directory. To configure procmail for this purpose,

and type in the lines

MAILDIR=$HOME/Library/Maildir_for_IMAP_server
DEFAULT=$HOME/Library/Maildir_for_IMAP_server/
#LOGFILE=$MAILDIR/log_for_debugging

This should be pretty self-explanatory: I configure things so that procmail stores each incoming piece of mail in the addressed user's directory, ~/Library/Maildir_for_IMAP_server/. Later we'll tell binc-imap to use this same place as its storage directory.

Installation And Setup of binc-imap

This is simple enough. First use DarwinPorts to do the usual download, compile and install.

The preferences for binc-imap live in the DarwinPorts /etc folder, so lets go there

and edit the prefs file

Look for the line

path = "XXXXXXXXXXXXXXXXXXX",      /* default path */

and replace the XXXXXXXXXXXX with whatever you set for procmail in the section above (ie replace the line with

path = "Library/Maildir_for_IMAP_server",      /* default path */

Newer versions of binc-imap may also have as the very first line of the prefs fle, the text

allow plain auth in non ssl = "no" 

You want to change this line to

allow plain auth in non ssl = "yes"

(This setting, not allowing plaintext authorization, is a security setting that makes sense in many environments but not for us. We are going to obtain security through an alternative mechanism, based on ssh, that will be explained below and in the next article.

DarwinPorts should have installed a file /etc/pam.d/bincimap, which is used by binc-imap to verify passwords. You don't need to do anything to this except ensure that it exists. If it doesn't, copy it over from /opt/local/etc/bincimap/pam.

System startup issues

postfix

We've seen this before in the case of dnsupdate, so it's no surprise that, once again, we need to write a .plist file for launchd. There is, however, one additional element in this case.

There is a file called /etc/hostconfig which you might want to look at right now. This lists various services your mac can perform, along with whether or not those services are active. Many of these services are turned on or off by a pref pane, for example switching the Personal Web Sharing feature of the Sharing pref pane on will set the line WEBSERVER=-YES-, while turning Windows Sharing off in the same pref pane will set the line SMBSERVER=-NO-. Others of these services do not have a GUI for modifying them.

In the pre-Tiger days it was pretty obvious how this file modified whether or not a particular Startup Item ran. Under Tiger it is not at all clear whether or not this file actually does anything, or if it is being kept just for backward compatability, with the idea that it'll eventually go away.

Just to be safe, however, we will modify this file to indicate that we want postfix to run.

and change the last line from MAILSERVER=-NO- to

Now we need create the launchd plist.

type in

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Label</key>
        <string>postfix</string>
        <key>OnDemand</key>
        <false/>
        <key>Program</key>
        <string>/usr/sbin/postfix</string>
        <key>ProgramArguments</key>
        <array>
                <string>/usr/sbin/postfix</string>
                <string>start</string>
        </array>
</dict>
</plist>

and save.

binc imap (and xinetd)

One way a program can interact with a port is to be running continually, and have itself registered with the OS as interested in that particular port. An alternative is to make use of a demon called xinetd.

NOTE: xinetd is no longer the preferred way to do things under MacOS X. This initial explanation discusses xinetd because it is still used on other UNIXes, and because not all programs on the mac have transitioned to using it. You should read (and understand) the explanation, but not bother actually typing in the changes. In the next section we will discuss how to use launchd to do what we want done.

The program that wishes to interact with a port informs xinetd (through a configuration file) that it is interested in listening to some particular port number. xinetd runs continually, listening to all the ports it has been told about, and when something comes in on one of these ports, xinetd starts the program that told xinetd it wanted to listen to that port, and hands control over to that program. That program deals with the particular transaction and, when it is over, quits, handing the listening of the port back to xinetd.

The advantages of this scheme are twofold:
The first is that various common functionality can be put into xinetd instead of into every little program that listens on a port. This common functionality includes things like logging activity on the port, and restricting access to a port to particular network clients rather than allowing the entire internet to connect to that port.
The second is that only xinetd is constantly running and always in memory. All the various programs registered with xinetd are only run when some activity occurs on their port, meaning that most of the time they are not running and not taking up memory.

Postfix is an example of the first type of program. Postfix is always running,and takes responsibility for logging, limiting what can connect to its port and so on. But most server programs (and binc imap is one of them) run through xinetd.

xinetd is configured through files in the directory /etc/xinetd.d. If you look there, you should find that darwin ports has already installed /etc/xinetd.d/imap. (This file was copied there from the original which is at /opt/local/etc/bincimap/xinetd/imap.sample. )

We want to make one change to /etc/xinetd.d/imap to improve our security. We will add the line

Add this line just under the line

By adding this line, we say that only the local machine can connect to our IMAP server, no other machine. As explained earlier, remote machines will connect to the local machine using an ssh tunnel.

At this point we can either reboot, or we can tell xinetd to rescan its configuration files.

The command to tell xinetd to rescan its config files is

Let's understand this.
First some programs (not all, but some) when they start up store their process ID in a file in /var/run. xinetd is one of these programs. Alternatively you can find xinetd's process ID by using ps or by looking at the process list in the Activity Monitor application.
Next quoting a sub-command in backticks ( `command`) in a command-line executes that command and substitutes the results in the main command line. So the commmand line we wrote actually turns into something like sudo kill -HUP 50, assuming the PID of xinetd is 50.
Finally the kill command, in spite of its name, actually sends what is called a signal to a process. In this case the signal sent is the HUP signal. Many UNIX commands follow the convention that they respond to a HUP signal by re-reading their configuration file information.

binc imap (and launchd)

As already mentioned, the modern way of doing things is to use launchd wherever possible. Is is possible in this case? The answer is yes, even though Apple's technote for launchd implies that it is not.

We are going through all these various hoops as part of our security scheme. We want our bincimap to respond only to network requests from localhost, and we generate these requests from another machine via an ssh tunnel. The Apple documentation specifically says that launchd cannot limit requests to those from a single host, but the documentation is wrong/out of date. This is done by setting the SockNodeName field in the launchd config file to 127.0.0.1 which is, of course, the IP address of localhost.

So to switch to using launchd for bincimap:

1. Move the imap file created by the bincimap installer in xinetd.d out of the way.
   sudo mv /etc/xinetd.d/imap ..

2. Create a file in /Library/LaunchDaemons named imap.plist (or whatever other name you want)

   sudo pico /Library/LaunchDaemons/imap.plist

3. Cut and paste this text into that file:

   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0">
   <dict>
           <key>Label</key>
           <string>imap</string>
           <key>Nice</key>
           <integer>-7</integer>
           <key>ProgramArguments</key>
           <array>
                   <string>/opt/local/bin/bincimap-up</string>
                   <string>--conf=/opt/local/etc/bincimap/bincimap.conf</string>
                   <string>--logtype=syslog</string>
                   <string>--</string>
                   <string>/opt/local/bin/checkpassword-pam</string>
                   <string>-s</string>
                   <string>bincimap</string>
                   <string>/opt/local/bin/bincimapd</string>
           </array>
           <key>ServiceDescription</key>
           <string>binc IMAP</string>
           <key>Sockets</key>
           <dict>
                   <key>imap143</key>
                   <dict>
                           <key>SockNodeName</key>
                           <string>127.0.0.1</string>
                           <key>SockServiceName</key>
                           <string>imap</string>
                   </dict>
           </dict>
           <key>inetdCompatibility</key>
           <dict>
                   <key>Wait</key>
                   <false/>
           </dict>
   </dict>
   </plist>
   

If you now open Lingon and look at this file, it should be pretty obvious how it relates to the xinetd config file.

Once you have tested and validated that your launchd config file for imap works, (described in the text below) you should go back and fully remove the xinetd file that we earlier just moved out of the way, ie

.

Testing

Testing postfix

We're done with (most of) the mail setup. You should now have a secure SMTP server and IMAP server. Let's test that they are alive. How can we do this?

First let's test that the config file is not broken and that postfix is running. lets try to get postfix running by typing

Most people don't know that when computers using SMTP or IMAP (or for that matter the HTTP used by the web) communicate with each other they do so using "English like" words in pure ASCII text. We can use this to test if our two server processes ( postfix and imapd) are alive. What we need to do is connect to ports 25 and 143 and send ascii messages to the port. We use the command line program telnet to do this. Telnet, in the old days, used to be used for remote login to other computers. Nowadays no-one uses it that way because it's insecure, but it is still useful for having a conversation on your computer with a particular port.

So, in a Terminal window on the server type

You should get a reply like

	
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 name99.org ESMTP Postfix

You can now type an SMTP command like

and get a reply like

You can then type further commands like MAIL, RCPT, DATA or QUIT. You can read the full gory details of the SMTP command language at http://cr.yp.to/smtp.html, but we don't need any of that; all we wanted to know was that we have a server listening on port 25. So we type

to end the conversation.

If the above does not play out the way you expect,

• open Console.app
• click on the Logs button in the upper left of the Console window
• click on the /var/log "triangle" to show all the logs listed in /var/log
• scroll down the list to mail.log and select it.

With luck there should be something in mail.log describing what has gone wrong. If there is not, kill postfix and start it again with verbose logging,

If that doesn't help, try starting postfix with even more verbose logging

You can keep adding -v's to make logging more and more verbose. You do not usually want to run postfix with verbose (or very verbose) logging, but it can be helpful in debugging.

Testing imap

We now do the same thing for IMAP. First we need to be sure that launchd is running the plist file we just created. You can click either use Lingon to do this, in a fashion that should be obvious from the UI, or you can type

If for some strange reason you insist on using xinetd, the appropriate command is sudo kill -HUP `cat /var/run/xinetd.pid` to force xinetd to reread the binc-imap xinetd config file in /etc/xinetd.d.

Now type

At this point you should get the reply

All commands to an IMAP server begin with a "." (ie with a period, no quotes). (That's actually a drastic simplification, but it's good enough for now. So make sure, in all the commands you type below, that you start each line with a . then a space --- it's easy to forget the period.)

The first thing you have to do when you contact an IMAP server is login; until you do that no other command works. So type

where username, password are what you use to log in to this computer when it boots up. And you should get a reply something like

Now type the command

which tells imapd to tell us whether any new mail has arrived. We get the reply

* 1007 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1094380597]
* OK [UIDNEXT 8882] 8882 is the next UID
* FLAGS (\Answered \Flagged \Deleted \Recent \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft)] Limited
. OK [READ-ONLY] EXAMINE completed
	

Type

to quit the conversation with imapd. If you want to learn the gory details of IMAP syntax (and there is pretty much no reason for you to do so) you can read up about it at web sites like http://www.isi.edu/in-notes/rfc3501.txt

Once again, if something is wrong examine the mail log file (and perhaps other log files like console.log and system.log) to see if you have any hint as to why.

Testing using a local (ie on the server) Mail.app

So now our servers work for local access. What's left? We still have to set up ssh (so remote access works) and fetchmail. Before we do those, let's set up Mail.app so that we get some more immediate feedback that things are working. We will test Mail.app running on the server, because, until we have set up ssh, no other computer is allowed to contact our server via the SMTP and IMAP ports.

Open Mail.App.
Open Preferences and create a new account of type IMAP.
Set the Description field to whatever you want --- maybe bluecloud.com, maybe My IMAP.
Set the email address to what you want your default email address to be, eg mjh@bluecloud.com.
Enter your full name.
DO NOT ENTER ANYTHING MORE ON THIS TAB YET.

Switch to the Special Mailboxes tab. Here you can set which mail is stored on the server and which is only kept on a local machine, For example most people don't care about their junk mail being synchronized across all their computers, so they set things so that junk is not stored on the server.
I check the boxes that say Store drafts on server and Store sent message on server and Move deleted message to Trash.
I don't check the Store junk messages on server and Store deleted messages on server boxes.
I set Sent Mail to be deleted never, and Junk + Trash Mail to be deleted after one week.
You can set whatever settings you like here.

Switch to the Advanced tab.
Leave the Account Directory field to whatever it is set to.
I keep Copies of messages for offline viewing set to All Messages and their attachments; change this if you want.
I set the checkbox Automatically synchronize changed mailboxes. This is the whole reason we're setting up IMAP, isn't it --- so that the mail on all our computers stays in sync.
Leave the IMAP Path Prefix set to INBOX.

Now go back to the Account Information tab.
Set the Incoming Mail Server to localhost
Set the UserName to mjh (your account name on this computer)
Set the Password to (your password for your account on this computer)

Choose Add Server... from the list of outgoing Mail Servers, and in the resulting dialog set the Outgoing Mail Server to localhost. Leave everything else the default.

It now looks like little has changed in your mail. The In box shows a new tray for your email, but nothing else. This is because the various other mailboxes you'd expect for your IMAP server are only created when needed.

Create a test message to yourself and send it from your imap account to your imap address. Click Get Mail, and with luck you will now have your first message in your IMAP account. This message went

• out from Mail.app,
• via port 25, through postfix which
• sent it to procmail which
• dumped it (in maildir format) in ~/Library/Maildir_for_IMAP_server.

Then when you clicked the Get Mail button, Mail.app sent

• a message via port 143 to imapd which looked at
• ~/Library/Maildir_for_IMAP_server, saw the message there,
• copied it to the imap server's mail store (also in ~/Library/Mail/Maildir_for_IMAP_server),
• and gave it (via port 143) to Mail.app.

Mail.app then stored the message in Mail.app's view of this account

• which is in in folder called something like ~/Library/Mail/IMAP-mjh@localhost.

Great, Mail.app is now set up properly on the server, and our postfix and imapd all communicate happily with each other. Of course you will probably not care much about running Mail.app on your server, so you can close it now and forget about it.

Improving the performance of Mail

There is one small thing we can do to improve mail performance slightly that is so simple there's no reason not to do it, which is to give postfix and bincimap a higher priority. (Recall that we discussed priorities earlier here.)

We do this exactly as we did with DNSUpdate: open the two relevant files in Lingon, and set the niceness. Personally I set the niceness of bincimap to -7 and that of postfix to -6. I then have web serving set to -5. This gives me manipulating mail as maximum priority, sending mail as slightly lower, and web as even more slightly lower priority.

---

← prev: Mail Theory next: Mail ssh Setup →