MAILDIR(5)MAILDIR(5)NAMEmaildir - E-mail directory
SYNOPSIS
$HOME/Maildir
DESCRIPTION
A ``Maildir'' is a structured directory that holds E-mail messages.
Maildirs were first implemented by the Qmail mail server. Qmail's
maildirs were a simple data structure, nothing more than a single col‐
lection of E-mail messages. Courier builds upon Qmail's maildirs to
provide extended functionality, such as folders and quotas. This docu‐
ment describes Courier's extended maildirs, without explicitly identi‐
fying Courier-specific extensions. See maildir(5) in Qmail's documen‐
tation for the original definition of maildirs.
Traditionally, E-mail folders were saved as plain text files, called
``mboxes''. Mboxes have known limitations. Only one application can
use an mbox at the same time. Locking is required in order to allow
simultaneous concurrent access by different applications. Locking is
often problematic, and not very reliable in network-based filesystem
requirements. Some network-based filesystems don't offer any reliable
locking mechanism at all. Furthermore, even bulletproof locking won't
prevent occasional mbox corruption. A process can be killed or termi‐
nated in the middle of updating an mbox. This will likely result in
corruption, and a loss of most messages in the mbox.
Maildirs allow multiple concurrent access by different applications.
Maildirs do not require locking. Multiple applications can update a
maildir at the same time, without stepping on each other's feet.
MAILDIR CONTENTS
A ``maildir'' is a directory that's created by maildirmake(1). Natu‐
rally, maildirs should not have any group or world permissions, unless
you want other people to read your mail. A maildir contains three sub‐
directories: tmp, new, and cur. These three subdirectories comprise
the primary folder, where new mail is delivered by the system.
Folders are additional subdirectories in the maildir whose names begin
with a period: such as .Drafts or .Sent. Each folder itself contains
the same three subdirectories, tmp, new, and cur, and an additional
zero-length file named maildirfolder, whose purpose is to inform any
mail delivery agent that it's really delivering to a folder, and that
the mail delivery agent should look in the parent directory for any
maildir-related information.
Folders are not physically nested. A folder subdirectory, such as
.Sent does not itself contain any subfolders. The main maildir con‐
tains a single, flat list of subfolders. These folders are logically
nested, and periods serve to separate folder hierarchies. For example,
.Sent.2002 is considered to be a subfolder called ``2002'' which is a
subfolder of ``Sent''.
FOLDER NAME ENCODING
Folder names can contain any Unicode character, except for control
characters. US-ASCII characters, U+0x0020 - U+0x007F, except for the
period, forward-slash, and ampersand characters (U+0x002E, U+0x002F,
and U+0x0026) represent themselves. The ampersand is represent by the
two character sequence ``&-''. The period, forward slash, and non US-
ASCII Unicode characters are represented using the UTF-7 character set,
and encoded with a modified form of base64-encoding.
The ``&'' character starts the modified base64-encoded sequence; the
sequence is terminated by the ``-'' character. The sequence of 16-bit
Unicode characters is written in big-endian order, and encoded using
the base64-encoding method described in section 5.2 of RFC 1521, with
the following modifications:
· The ``='' padding character is omitted. When decoding, an incomplete
16-bit character is discarded.
· The comma character, ``,'' is used in place of the ``/'' character in
the base64 alphabet.
For example, the word ``Resume'' with both "e"s being the e-acute char‐
acter, U+0x00e9, is encoded as ``R&AOk-sum&AOk-'' (so a folder of that
name would be a maildir subdirectory called ``.R&AOk-sum&AOk-'').
OTHER MAILDIR CONTENTS
Software that uses maildirs may also create additional files besides
the tmp, new, and cur subdirectories -- in the main maildir or a sub‐
folder -- for its own purposes.
MESSAGES
E-mail messages are stored in separate, individual files, one E-mail
message per file. The tmp subdirectory temporarily stores E-mail mes‐
sages that are in the process of being delivered to this maildir. tmp
may also store other kinds of temporary files, as long as they are cre‐
ated in the same way that message files are created in tmp. The new
subdirectory stores messages that have been delivered to this maildir,
but have not yet been seen by any mail application. The cur subdirec‐
tory stores messages that have already been seen by mail applications.
ADDING NEW MAIL TO MAILDIRS
The following process delivers a new message to the maildir:
A new unique filename is created using one of two possible forms:
``time.MusecPpid.host'', or ``time.MusecPpid_unique.host''. ``time''
and ``usec'' is the current system time, obtained from gettimeofday(2).
``pid'' is the process number of the process that is delivering this
message to the maildir. ``host'' is the name of the machine where the
mail is being delivered. In the event that the same process creates
multiple messages, a suffix unique to each message is appended to the
process id; preferrably an underscore, followed by an increasing
counter. This applies whether messages created by a process are all
added to the same, or different, maildirs. This protocol allows multi‐
ple processes running on multiple machines on the same network to
simultaneously create new messages without stomping on each other.
The filename created in the previous step is checked for existence by
executing the stat(2) system call. If stat(2) results in ANYTHING
OTHER than the system error ENOENT, the process must sleep for two sec‐
onds, then go back and create another unique filename. This is an
extra step to insure that each new message has a completely unique
filename.
Other applications that wish to use tmp for temporary storage should
observe the same protocol (but see READING MAIL FROM MAILDIRS below,
because old files in tmp will be eventually deleted).
If the stat(2) system call returned ENOENT, the process may proceed to
create the file in the tmp subdirectory, and save the entire message in
the new file. The message saved MUST NOT have the ``From_'' header
that is used to mboxes. The message also MUST NOT have any ``From_''
lines in the contents of the message prefixed by the ``>'' character.
When saving the message, the number of bytes returned by the write(2)
system call must be checked, in order to make sure that the complete
message has been written out.
After the message is saved, the file descriptor is fstat(2)-ed. The
file's device number, inode number, and the its byte size, are saved.
The file is closed and is then immediately moved/renamed into the new
subdirectory. The name of the file in new should be ``time.MusecPpid‐
VdevIino.host,S=cnt'', or ``time.MusecPpidVdevIino_unique.host,S=cnt''.
``dev'' is the message's device number, ``ino'' is the message's inode
number (from the previous fstat(2) call); and ``cnt'' is the message's
size, in bytes.
The ``,S=cnt'' part optimizes Courier's maildir quota enhancement; it
allows the size of all the mail stored in the maildir to be added up
without issuing the stat(2) system call for each individual message
(this can be quite a performance drain with certain network filesys‐
tems).
READING MAIL FROM MAILDIRS
Applications that read mail from maildirs should do it in the following
order:
When opening a maildir or a maildir folder, read the tmp subdirectory
and delete any files in there that are at least 36 hours old.
Look for new messages in the new subdirectory. Rename new/filename, as
cur/filename:2,info. Here, info represents the state of the message,
and it consists of zero or more boolean flags chosen from the follow‐
ing: ``D'' - this is a 'draft' message, ``R'' - this message has been
replied to, ``S'' - this message has been viewed (seen), ``T'' - this
message has been marked to be deleted (trashed), but is not yet removed
(messages are removed from maildirs simply by deleting their file),
``F'' - this message has been marked by the user, for some purpose.
These flags must be stored in alphabetical order. New messages contain
only the :2, suffix, with no flags, indicating that the messages were
not seen, replied, marked, or deleted.
Maildirs may have maximum size quotas defined, but these quotas are
purely voluntary. If you need to implement mandatory quotas, you
should use any quota facilities provided by the underlying filesystem
that is used to store the maildirs. The maildir quota enhancement is
designed to be used in certain situations where filesystem-based quotas
cannot be used for some reason. The implementation is designed to
avoid the use of any locking. As such, at certain times the calculated
quota may be imprecise, and certain anomalous situations may result in
the maildir actually going over the stated quota. One such situation
would be when applications create messages without updating the quota
estimate for the maildir. Eventually it will be precisely recalcu‐
lated, but wherever possible new messages should be created in compli‐
ance with the voluntary quota protocol.
The voluntary quota protocol involves some additional procedures that
must be followed when creating or deleting messages within a given
maildir or its subfolders. The deliverquota(8) command is a tiny
application that delivers a single message to a maildir using the vol‐
untary quota protocol, and hopefully it can be used as a measure of
last resort. Alternatively, applications can use the libmaildir.a
library to handle all the low-level dirty details for them. The volun‐
tary quota enhancement is described in the maildirquota(7) man page.
MAILDIR QUOTAS
This is a voluntary mechanism for enforcing "loose" quotas on the maxi‐
mum sizes of maildirs. This mechanism is enforced in software, and not
by the operating system. Therefore it is only effective as long as the
maildirs themselves are not directly accessible by their users, since
this mechanism is trivially disabled.
If possible, operating system-enforced quotas are preferrable. Where
operating system quota enforcement is not available, or not possible,
this voluntary quota enforcement mechanism might be an acceptable com‐
promise. Since it's enforced in software, all software that modifies
or accesses the maildirs is required to voluntary obey and enforce a
quota. The voluntary quota implementation is flexible enough to allow
non quota-aware applications to also access the maildirs, without any
drastic consequences. There will be some non-drastic consequences,
though. Of course, non quota-aware applications will not enforce any
defined quotas. Furthermore, this voluntary maildir quota mechanism
works by estimating the current size of the maildir, with periodic
exact recalculation. Obviously non quota-aware maildir applications
will not update the maildir size estimation, so the estimate will be
thrown off for some period of time, until the next recalculation.
This voluntary quota mechanism is designed to be a reasonable compro‐
mise between effectiveness, and performance. The entire purpose of
using maildir-based mail storage is to avoid any kind of locking, and
to permit parallel access to mail by multiple applications. In order
to compute the exact size of a maildir, the maildir must be locked
somehow to prevent any modifications while its contents are added up.
Obviously something like that defeats the original purpose of using
maildirs, therefore the voluntary quota mechanism does not use locking,
and that's why the current recorded maildir size is always considered
to be an estimate. Regular size recalculations will compensate for any
occasional race conditions that result in the estimate to be thrown
off.
A quota for an existing maildir is installed by running maildirmake
with the -q option, and naming an existing maildir. The -q option
takes a parameter, quota, which is a comma-separated list of quota
specifications. A quota specification consists of a number followed by
either 'S', indicating the maximum message size in bytes, or 'C', maxi‐
mum number of messages. For example:
maildirmake -q 5000000S,1000C ./Maildir
This sets the quota to 5,000,000 bytes or 1000 messages, whichever
comes first.
maildirmake -q 1000000S ./Maildir
This sets the quota to 1,000,000 bytes, without limiting the number of
messages.
A quota of an existing maildir can be changed by rerunning the
maildirmake command with a new -q option. To delete a quota entirely,
delete the Maildir/maildirsize file.
SEE ALSOmaildirmake(1).
Double Precision, Inc. 25 July 2004 MAILDIR(5)
