[Bulk] Re: [wplug] Anyone Wanna Try A New Email Client?

[Rob Knapp]

> On Tue, 2007-09-11 at 12:07 -0400, Rob Knapp wrote: 
> > Right now I'm working with SQLite, with plans to offer a msyql or 
> > postgres version in the future. (If I do that, I might implement it 
> > as a RESTFUL api so that I can still access my emails when traveling, 
> > without setting up a VPN :) ) 
> 
> No, no, SQLite is great and scales pretty well! You can always have an 
> "upload/sync" feature to another. SQLite "just works." 
> 

Oh, don't get me wrong I like SQLite... actually quite a bit. However as far as I can tell it does have some disadvantages.  For example, when I tried to connect to a 500M over an NFS mount on a wireless network.... that was not pretty.  By the time I stopped the process there were bits of broken photon all over my game room. :)

[snip]
> >  There is  about a 2-3 second delay when I click on that tag, and about a
> > 6-10  second delay when I click on "All". (This is only if I view all 
> > messages, when viewing only new messages, everything is immediate.) 
> > Doing a search accross all emails takes about 10-20 seconds... so 
> > there is a lot of room 
> > for improvement! 
> 
As a result of your email, I went back into the code and was able to find an issue (a loop that was supposed to stop at 50 wasn't stopping) and tightened things up.  I can bring up the  list for 47K emails in under 2 seconds....  (then they get added to the ui element as needed....)

> I'm about 10GB of IMAP-stored mbox right now. 
> 

That is going to take a very long time to import, even if you are pulling over a home network. :)   (SInce I am doing searching on a sql db, clearly I have to have all the data I'm going to query in the db....   Hey! does that qualify for the most obvious statement of the year?) 

Until I have the mbox importer written, I'd suggest that you might want to play around with an empty account, like from gmail or something...that is unless you are more patient then myself.

> > I've profiled the code, and it appears that the biggest delay is in 
> > actually retrieving the records, so a more robust db solution might be 
> > required for really big repositories. (Or my db indeces are 
> > wrong...which is possible.) 
> 
> Yeah, there are various ways to address that. Hashes to start for 
> various functions. Things where the hashing in the DB don't help your 
> application directly. 
> 

Heh.  Stupid profiler...made me look bad.  Yeah, as I mentioned the problem 
wasn't the speed fo the query.   Well kinda was in that I was pulling all the records into memory, rather than pulling them one at a time. But once I stopped doing that, I found that the bigger problem was the loop I mentioned.

> > The other issue (and this might affect you) is that the intial import 
> > of messages from the server can take some time. (About 20 minutes for 
> > 8K mails last time I checked.) After that things are pretty swift, 
> > though. [Also there is currently no support for importing from an 
> > existing maildir or mbox repository...I simply haven't written it 
> > yet.] 
> 
> Can't be worse than Evolution! ;) 
> 
> > Yeah, i was starting to hit the same problem before I started work on 
> > this.... they each have about a couple hundred thousand emails dating 
> > back to 2004. 
> 
> I've been using the same e-mail address since 1993. Although I only 
> have from 1999+ in Evolution. I've also archived many mbox files (zcat 
> when I need it ;). 
> 
> 
> 
> -- 
> Bryan J. Smith Professional, Technical Annoyance 
> mailto:b.j.smith at ieee.org http://thebs413.blogspot.com 
> -------------------------------------------------------- 
> Fission Power: An Inconvenient Solution 
> 
> 
> _______________________________________________ 
> wplug mailing list 
> wplug at wplug.org 
> http://www.wplug.org/mailman/listinfo/wplug 
>