Saturday, September 18, 2010

Don't expect to use Gmail's web interface and IMAP together... very well.

Recently, my office transitioned our email provider to a Google Apps account.  Finally, a way to access all  my email remotely.  I've been a big Thunderbird user, and was looking forward to the IMAP access.  I've also been using Gmail for my personal email for about five years, so I was looking forward to having a web client with equal access to my email history.

Two weeks later and I've learned one major lesson: it's impossible to organize your email such that it makes sense in both the Gmail and IMAP interfaces at the same time.

My old, POP based email organization had evolved over time.  I had a Clients folder, and many of them had specific project folders under them. Some even had another level of specialty folders for things such as form submission data.

Gmail doesn't have folders, it has Labels (called Tags by everyone else in the industry).  Google's IMAP interface translates its labels into folders, which sounds straight forward enough... until you try to use it.

I figured I'd start by importing all my old mail and testing the waters. Unfortunately, even this was problematic.  Gmail's way of handling nested folders is to use a single, longer label for each step down the tree.  For instance, if the folders look like "Clients/Foo Electronics/fooelec.com/orders" then you probably have labels "Clients" "Clients/Foo Electronics", "Clients/Foo Electronics/fooelec.com", and "Clients/Foo Electronics/fooelec.com/orders".

There is a Labs tool to collapse nested labels into neat little trees, so that was a good thing.  However, there's one big technical problem here: Gmail only allows for labels to total 40 characters.  That means my poor nested folder structure above won't work, because it's 42 characters long (the separator character counts against you, of course).

Thus, I was forced to mangle some of my folders/labels.  For instance, my Clients/Archive folder used to contain all the clients I no longer work with... adding 8 more characters to the length of their labels. I was forced to replace "Clients/Archive/" with just "CA/" to make things fit, making them not particularly intuitive.  Some specific clients also had to have folder names cut down for space.

But that's all cosmetic.  Making these changes I could now access all my email via both Gmail and IMAP.  I was planning on altering how I organized my email anyway, so this was just a minor, temporary inconvenience.

Unfortunately, this turned out to be the easy part.

Two of Gmail's greatest assets worked together to make IMAP organization a struggle: labels and conversations.

Both of these provide conceptual power to Gmail's interface, but those concepts break down over IMAP and even start causing additional conceptual problems.

For example: In Thunderbird, I can search for all messages in my Clients/Foo folder and all its child folders.  Gmail has no way to search through "sub labels" since it doesn't really have a concept of such a thing... to Gmail, all labels are flat.  Thus, having a more organized, nested structure is a detriment in Gmail.

The obvious next step is to have separate labels for the client and the project. Thus a single  message might be labeled "Foo Electronics", "fooelec.com", and "orders"... and maybe even "Clients".  That's... a lot of labels, especially since most of them imply the previous one in the chain.  ALL emails about "Foo Electronics" will be related to a client, and everything for "fooelec.com" will be for the client Foo!  What a waste to have all those separate labels.  Not to mention, that Orders label will need to be shared with all clients whose projects generate orders.  What a mess.

The problems get magnified when you look back at the labels-to-IMAP translation.  Each and every message appears in the "All Mail" folder, and is copied into every folder that matches each of its labels.  Thus, in the scenario before, all fooelec.com order emails would be in, at least, five folders.  Five separate copies of one email, just to be "properly" (if over zealously) labeled in the Gmail interface.  There will be more copies if any of them are Starred, in your Inbox, marked with a generic label like "Important", etc. (each represented by their own folders)

At this point, it's taking 5-6 times as long to even download my mail into my IMAP client, it's taking up that much more disk space, AND it's harder to search.  Thunderbird now will find many copies of the same message if I do a search that isn't restricted to one of those labels.

Making matters worse, Gmail's filter system is far behind Thunderbird's. To compensate, my original hope was to use Thunderbird's filters to organize mail... but that required my IMAP client be running to organize mail for the web interface, which makes no sense.  And trying to manipulate the labels of Gmail via IMAP has some... pitfalls.  This article (Gmail's Buggy IMAP Implementation) talks about some of the technical limbo Google imposes on "properly" using IMAP with its email service.

All of these issues I could find ways to work around, but I had lost faith that I could teach other, not-as-technical people in the company to use IMAP.  The final straw came when I realized the implication of how this IMAP structure would interact with Gmail's threaded conversation feature.

Say I've been communicating back and forth with a group of people for a while, and I Archive the messages.  My Inbox is empty, but I have copies of those messages in All Mail and maybe other places based on labels.

Now, one of the participants sends a new reply. In Gmail, the entire conversation pops up in my Inbox, but when I open it, I see only the messages that were new to the conversation (but have easy access to the earlier ones.

Under the hood, Gmail is smart enough not to move ALL message in the thread back to the Inbox, but that exposes just another layer of complexity... individual messages can have different labels from each other!  Just because I put a message in a folder in Thunderbird, doesn't mean ALL the messages in the conversation end up there... even though the Conversation as a whole does have that label in the Gmail interface.  The simple act of moving a single message into a folder causes the organization between the two interfaces to fall out of step.

The end result of all of this is that I have to abandon my plan to use Google's IMAP connection at all.  Despite the power of Gmail, there's many things that a native client can do better (like sort and scroll though messages quickly without having to create a search for every little thing), but with Gmail, it would either be POP or nothing... the IMAP is useless.

No comments:

Post a Comment