Converting Mbox mailboxes to Maildir format:

page version: 2006-05-07
software version: 3.20


Contents:


News:

See the changelog.

Introduction:

Mb2md.pl was originally developed by Robin Whittle <rw at firstpr.com.au> (you can find his old page here). It is a Perl script that takes one or more Mbox format mailbox files in a directory and convert them to Maildir format mailboxes. As the Mbox format has some drawbacks, D. J. Bernstein created the Maildir format when he wrote Qmail. With the Mbox format all mail of a specific folder is stored as one large text file. The Maildir format stores each mail as a seperate file. It is a faster and more efficient way to store mail. It works particularly well over NFS, which has a long history of locking-related woes.
The Mbox format is used by most of the POP3/IMAP servers, most mail servers (MTAs) and mail readers (MUAs). The Maildir format is used by Qmail, Courier-MTA and can be also used as a alternative mail storage format by Postfix and Exim. A good POP3/IMAP server for Maildirs is Courier IMAP.

Mb2md.pl does not only convert mailbox files into a Maildir but also the /var/spool/mail/$USER mailspool file. It is smart enough to not transfer a dummy message such as the UW IMAPD puts at the start of Mbox mailboxes - and you could add your own search terms into the script to make it ignore other forms of dummy first message.


Features:

Currently mb2md.pl has the following features:

Usage:

Run this as the user of the mailboxes, not as root.


mb2md -h
mb2md [-c] -m [-d destdir]
mb2md [-c] -s sourcefile [-d destdir]
mb2md [-c] -s sourcedir [-l wu-mailboxlist] [-R|-f somefolder] [-d destdir] [-r strip_extension]

 -c            use the Content-Length: headers (if present) to find the
               beginning of the next message
               Use with caution! Results may be unreliable. I recommend to do
               a run without "-c" first and only use it if you are certain,
               that the mbox in question really needs the "-c" option

 -m            If this is used then the source will
               be the single mailbox at /var/spool/mail/blah for
               user blah and the destination mailbox will be the
               "destdir" mailbox itself.


 -s source     Directory or file relative to the user's home directory,
               which is where the the "somefolders" directories are located.
               Or if starting with a "/" it is taken as a
               absolute path, e.g. /mnt/oldmail/user

               or

               A single mbox file which will be converted to
               the destdir.

 -R            If defined, do not skip directories found in a mailbox
               directory, but runs recursively into each of them, 
               creating all wanted folders in Maildir.
               Incompatible with '-f'

 -f somefolder Directories, relative to "sourcedir" where the Mbox files
               are. All mailboxes in the "sourcedir"
               directory will be converted and placed in the
               "destdir" directory.  (Typically the Inbox directory
               which in this instance is also functioning as a
               folder for other mailboxes.)

               The "somefolder" directory
               name will be encoded into the new mailboxes' names.
               See the examples below.

               This does not save an UW IMAP dummy message file
               at the start of the Mbox file.  Small changes
               in the code could adapt it for looking for
               other distinctive patterns of dummy messages too.

               Don't let the source directory you give as "somefolders"
               contain any "."s in its name, unless you want to
               create subfolders from the IMAP user's point of
               view.  See the example below.

               Incompatible with '-R'


 -d destdir    Directory where the Maildir format directories will be created.
               If not given, then the destination will be ~/Maildir .
               Typically, this is what the IMAP server sees as the
               Inbox and the folder for all user mailboxes.
               If this begins with a '/' the path is considered to be
               absolute, otherwise it is relative to the users
               home directory.

 -r strip_ext  If defined this extension will be stripped from
               the original mailbox file name before creating
               the corresponding maildir. The extension must be
               given without the leading dot ("."). See the example below.

 -l UW-file    File containing the list of subscribed folders.  If
               migrating from UW-IMAP the list of subscribed folders will
               be found in the file called .mailboxlist in the users
               home directory.  This will convert all subscribed folders
               for a single user:
               /bin/mb2md -s mail -l .mailboxlist -R -d Maildir
               and for all users in a directory as root you can do the
               following:
               for i in *; do echo $i;su - $i -c "/bin/mb2md -s mail -l .mailboxlist -R -d Maildir";done


 Example
 =======

We have a bunch of directories of Mbox mailboxes located at
/home/blah/oldmail/

    /home/blah/oldmail/fffff
    /home/blah/oldmail/ggggg
    /home/blah/oldmail/xxx/aaaa
    /home/blah/oldmail/xxx/bbbb
    /home/blah/oldmail/xxx/cccc
    /home/blah/oldmail/xxx/dddd
    /home/blah/oldmail/yyyy/huey
    /home/blah/oldmail/yyyy/duey
    /home/blah/oldmail/yyyy/louie

With the UW IMAP server, fffff and ggggg would have appeared in the root
of this mail server, along with the Inbox.  aaaa, bbbb etc, would have
appeared in a folder called xxx from that root, and xxx was just a folder
not a mailbox for storing messages.

We also have the mailspool Inbox at:

    /var/spool/mail/blah


To convert these, as user blah, we give the first command:

   mb2md -m

The main Maildir directory will be created if it does not exist.
(This is true of any argument options, not just "-m".)

   /home/blah/Maildir/

It has the following subdirectories:

   /home/blah/Maildir/tmp/
   /home/blah/Maildir/new/
   /home/blah/Maildir/cur/

Then /var/spool/blah file is read, split into individual files and
written into /home/blah/Maildir/cur/ .

Now we give the second command:

   mb2md  -s oldmail -R

This reads recursively all Mbox mailboxes and creates:

   /home/blah/Maildir/.fffff/
   /home/blah/Maildir/.ggggg/
   /home/blah/Maildir/.xxx/
   /home/blah/Maildir/.xxx.aaaa/
   /home/blah/Maildir/.xxx.bbbb/
   /home/blah/Maildir/.xxx.cccc/
   /home/blah/Maildir/.xxx.aaaa/
   /home/blah/Maildir/.yyyy/
   /home/blah/Maildir/.yyyy.huey/
   /home/blah/Maildir/.yyyy.duey/
   /home/blah/Maildir/.yyyy.louie/

 The result, from the IMAP client's point of view is:

   Inbox -----------------
       |
       | fffff -----------
       | ggggg -----------
       |
       - xxx -------------
       |   | aaaa --------
       |   | bbbb --------
       |   | cccc --------
       |   | dddd --------
       |
       - yyyy ------------
            | huey -------
            | duey -------
            | louie ------

Note that although ~/Maildir/.xxx/ and ~/Maildir/.yyyy may appear
as folders to the IMAP client the above commands to not generate
any Maildir folders of these names.  These are simply elements
of the names of other Maildir directories. (if you used '-R', they 
whill be able to act as normal folders, containing messages AND folders)

With a separate run of this script, using just the "-s" option
without "-f" nor "-R", it would be possible to create mailboxes which
appear at the same location as far as the IMAP client is
concerned.  By having Mbox mailboxes in some directory:
~/oldmail/nnn/ of the form:

    /home/blah/oldmail/nn/xxxx
    /home/blah/oldmail/nn/yyyyy

then the command:

  mb2md -s oldmail/nn

will create two new Maildirs:

   /home/blah/Maildir/.xxx/
   /home/blah/Maildir/.yyyy/

Then what used to be the xxx and yyyy folders now function as
mailboxes too.  Netscape 4.77 needed to be put to sleep and given ECT
to recognise this - deleting the contents of (Win2k example):

   C:\Program Files\Netscape\Users\uu\ImapMail\aaa.bbb.ccc\

where "uu" is the user and "aaa.bbb.ccc" is the IMAP server

I often find that deleting all this directory's contents, except
"rules.dat", forces Netscape back to reality after its IMAP innards
have become twisted.  Then maybe use File > Subscribe - but this
seems incapable of subscribing to folders.

For Outlook Express, select the mail server, then click the
"IMAP Folders" button and use "Reset list".  In the "All"
window, select the mailboxes you want to see in normal
usage.


This script did not recurse subdirectories or delete old mailboxes, before addition of the '-R' parameter :)

Be sure not to be accessing the Mbox mailboxes while running this
script.  It does not attempt to lock them.  Likewise, don't run two
copies of this script either.


Trickier usage . . .
====================

If you have a bunch of mailboxes in a directory ~/oldmail/doors/
and you want them to appear in folders such as:

~/Maildir/.music.bands.doors.Jim
~/Maildir/.music.bands.doors.John

etc. so they appear in an IMAP folder:

   Inbox -----------------
       | music
             | bands
                   | doors
                         | Jim
                         | John
                         | Robbie
                         | Ray

Then you could rename the source directory to:

 ~/oldmail/music.bands.doors/

then use:

  mb2md -s oldmail -f music.bands.doors


Or simply use '-R' switch with:
  mb2md -s oldmail -R


Stripping mailbox extensions:
=============================

If you want to convert mailboxes that came for example from
a Windows box than you might want to strip the extension of
the mailbox name so that it won't create a subfolder in your
mail clients view.

Example:
You have several mailboxes named Trash.mbx, Sent.mbx, Drafts.mbx
If you don't strip the extension "mbx" you will get the following
hierarchy:

Inbox
     |
      - Trash
     |       | mbx
     |
      - Sent
     |       | mbx
     |
      - Drafts
             | mbx

This is more than ugly!
Just use:
  mb2md -s oldmail -r mbx

Note: don't specify the dot! It will be stripped off
automagically ;)

Download:

Version: mb2md-3.20 (requires TimeDate perl library!)


Hope you like it. Comments are welcome: Juri Haberland <juri-at-koschikode.com>