IMAP IDLE

So I have been working on adding IMAP IDLE to the email scheduler.  My test server is First Class which fully supports IMAP and IDLE.  I have been testing this for the past week, and other than some minor issues, it has worked pretty good.  I was actually about to release it, then thought I better try some other servers to make sure all is good.  I tried a Microsoft Exchange Server with IMAP, and it seemed to work ok.  But then I tried Gmail.  For some reason, Gmail’s SSL/TLS system doesn’t like the library that I am using for the connection.  I will work to get this fixed up, but my question to you is, is this a deal breaker?  How many of you use email other than gmail that supports IMAP?  Would you like me to release this knowing that GMail will not work?  Post a comment here and let me know.

Now, a small write up on the Pro’sand Con’s of IMAP IDLE, and why I bet you won’t use it.  If you don’t know what IMAP IDLE is, google it, and you will find ton’s of info.  It is neat in that it gives you instant notification of email hitting your inbox (much like what people think of what a blackberry can do, only the way they work is much different).  If you NEED your email instantly, this will work, except not many email clients support it on Windows Mobile (pocket outlook does not support it).  One of the biggest problems with IMAP IDLE is you need a constant connection with the server.  On a desktop computer, this isn’t a problem, but on a phone, when your connection is not used for a certain amount of time, it will timeout (standby mode if you will), to save battery.  I am on Verizon and I get the timeout after 5 minutes of inactivity.  After 5 minutes, the connection is lost, which means I must reconnect to the server, and start the IDLE over again.  This is murder on the battery.  This is no different than setting the schedule to poll for email every 5 minutes!  There is no way around this since all imap servers use Sockets to connect to them, which means if the network is lost, so is the connection. 

I now understand why Microsoft implimented ActiveSync and Exchange Server the way they did.  ActiveSync uses the HTTP protocol, which you don’t need a constant connection for.  When the server wants to talk to your phone, it will send out the message to your IP (phone), which will take you out of Standby, and all is good.  But, even ActiveSync runs into this problem, as after say 15 minutes of inactivity, you will lose your connection (radio will shut off, and you may lose your IP address) all together.  ActiveSync has an algorithm to maximize the connection, and preserve battery.  If you use ActiveSync and Exchange, you can see your connection timeout times by looking in the registry, at HKCU/Software/Microsoft/ActiveSync.  There are several values to look at, but the most useful is MaxRecordedHeartBeat.  This will tell you the longest it has been able to go successfully.  They are recorded in seconds; Mine, again on Verzion, is 1200 seconds or 20 minutes.  That is Fantastic!  That means unless I get emails, my phone will only use the net once every 20 minutes, compared to IMAP IDLE needing it every 5 minutes.  You can read more about ActiveSync’s workings HERE.  So, to sum up, here is a table listing the Pro’s and Con’s:

Pro’s
Instant Email notification
Most email servers support IMAP IDLE (non-free accounts)
Push-like email built into an email standard

Con’s (for Mobile Devices)
Not supported by most Mobile Clients
Must have constant connection
Most providers ’sleep’ data after a short time (5 minutes) which kills the connection which makes this a battery hog.

So, in closing, do you really need that email RIGHT NOW?  polling every 10 minutes on my HTC Titan (Verizon xv6800) leaves me with at least 50% battery at day’s end.  Using IMAP IDLE, I am left with around 20%. 

Hope this helps explain things a bit, and why I bet you won’t use IMAP IDLE.  Questions?  Post them here.

Gudy

Like my work?  DONATE!

9 Responses to “IMAP IDLE”

  1. Stanislas Elie - February 2nd, 2009

    Hello.

    I’ve been using Imap Pusher Service found here http://www.codeplex.com/Wiki/View.aspx?ProjectName=ImapPusherService for a while now, and it works pretty well on my phone (HTC P4000 with Telus in Canada).

    So, I know it works, I get my emails instantly, and it doesn’t kill my battery.

    However, it is unfinished software, it sometimes crashes without telling you and there is no way to know that it is running aside from a task manager (no icon or window). And when it looses it’s connection, because you are on the phone or for some other reason, it sometimes just quits instead of reconnecting.

    Also, it is coded in .net and is a memory hog.

    So, I’ve been looking for something better.

  2. Stanislas Elie - February 2nd, 2009

    I forgot to mention, I’m using Gmail as my mail account.

    And my phone is the exact same as yours.

    And I don’t mind having only 20% battery at the end of the day, but I usually end up with at least 40%.

    Thanks!

  3. betonpoaltie - February 3rd, 2009

    Here another using Gmail imap; simply the easiest solution for email to my oppinion.
    I’ve tried several pushy kind of programs/providers, but still gmail is my favourite.
    Also gave imappusherservice a try, I think this comes close to what you are trying to implement, but I found the batterydrain and bandwidth usage more than enough reason not to continue with it.

    I hope your implementation will be more efficient and friendlier ;-)

    I just installed 0.60 release because 0.56b was not working for me; hope this will do better…
    Will certainly come back for the Imap implementation though (provided it will work with Gmail :-) )

    Thanks for your efforts!!

  4. JO - March 4th, 2009

    http://www.codeplex.com/ImapPusherService/Release/ProjectReleases.aspx?ReleaseId=17672

    I use this too and it has a ssl dll in the zip you might consider.

    I tried the idle service with desknow mail server and it works great but not with gmail unfortunately.

  5. mgudenka - March 5th, 2009

    JO-I have looked at the ssl lib that he uses. It does work, but it has several issues (blocks, dead thread, lack of events). Part of the reason I wrote this piece of the program was because IMAPPusher has these limitations. Otherwise, I would just use that service!

    Thanks

    Gudy

  6. Paul - October 16th, 2009

    Nice post Gudy,

    So after reading this I don’t really like the idea of using idel on my phone.

    What about allowing more than one exchange account? There has to be a way to do this considering it is an option on palms. Personally I don’t understand why Microsoft didnt include this option in wm 6, or 6.1, or event 6.5 but then again untill I read this I didnt understand why they didnt include idel support…

  7. Fixup - November 21st, 2009

    This article is wrong. Like IMAP, ActiveSync direct push also needs an open connection, otherwise the phone has to listen to port 443 all the time like a HTTPS server which drains out battery in no time. The connection remains inactive, but cannot be closed.

    In my area, Verizon too closes inactive sockets after 5 minutes. For AS to work properly, MinHeartBeat must be changed from the default 480 to 300 (5 min). You can find out the actual heartbeat your phone is using in Windows\ActiveSync, the Ctrl0 and Ctrl1 files.

    I modified your ImapPusher source code to deal with this 5-min socket error and other disconnects, now it works very reliable as long as I set refresh to 5-min. I posted xImapPusher on xda-developers.

  8. Jake - September 2nd, 2010

    Why do you say imap idle requires a connection every 5 minutes? This is actually a property of the imap server. For most servers, the time is 30 minutes, but for some (e.g. fastmail) it’s 60. I connect via imap idle to my fastmail account and set my mobile so that it keeps the connection alive every 50 minutes. On a Motorola Milestone (Android 2.1) witj k-9 email, this means that in standby, receiving about 10-12 emails a day, I use up 20-30% of the battery from 8 am till midnight.

  9. mgudenka - September 10th, 2010

    Jake–You are correct, however, with Windows Mobile, if you do not use the data connection for a X amount of time, it will shut off the radio. This then closes the IMAP connection, thus losing real-time email notifications. It would be like using a client on your home computer, using IMAP, then taking out your network card. You lose that connection as well, even if you replace the card.

    Android doesn’t do this, it keeps the connection alive at the tower level, meaning, even though the radio might be in idle, it can still hold a connection without killing the battery. At least, this is the way I understand it.

    Thanks for the feedback!

    Gudy

Leave a Reply

You must be logged in to post a comment.