[Bulk] Re: [wplug] Anyone Wanna Try A New Email Client?

[Bryan J. Smith]

On Tue, 2007-09-11 at 12:07 -0400, Rob Knapp wrote:
> Right now I'm working with SQLite, with plans to offer a msyql or
> postgres version in the future.  (If I do that, I might implement it
> as a RESTFUL api so that I can still access my emails when traveling,
> without setting up a VPN :) )

No, no, SQLite is great and scales pretty well!  You can always have an
"upload/sync" feature to another.  SQLite "just works."

> The db for my main account has 37.425 messages, with an additional 20K
> or so coming from other accounts.  (Basically all emails between Jan 1
> and now). That DB totals about 530 Megs, sqlite claims to be able to
> handle dbs "smaller than 10s of gigs"... so I haven't hit their limit
> yet.  The main issues I've run into is that that my biggest tag has
> about 30K messages in it (the python user mailing list).  There is
> about a 2-3 second delay when I click on that tag, and about a 6-10
> second delay when I click on "All".  (This is only if I view all
> messages, when viewing only new messages, everything is immediate.)
> Doing a search accross all emails takes about 10-20 seconds... so
> there is a lot of room 
> for improvement!

I'm about 10GB of IMAP-stored mbox right now.

> I've profiled the code, and it appears that the biggest delay is in
> actually retrieving the records, so a more robust db solution might be
> required for really big repositories.  (Or my db indeces are
> wrong...which is possible.)

Yeah, there are various ways to address that.  Hashes to start for
various functions.  Things where the hashing in the DB don't help your
application directly.

> The other issue (and this might affect you) is that the intial import
> of messages from the server can take some time.  (About 20 minutes for
> 8K mails last time I checked.)  After that things are pretty swift,
> though.  [Also there is currently no support for importing from an
> existing maildir or mbox repository...I simply haven't written it
> yet.]

Can't be worse than Evolution!  ;)

> Yeah, i was starting to hit the same problem before I started work on
> this.... they each have about a couple hundred thousand emails dating
> back to 2004.

I've been using the same e-mail address since 1993.  Although I only
have from 1999+ in Evolution.  I've also archived many mbox files (zcat
when I need it ;).



-- 
Bryan J. Smith         Professional, Technical Annoyance
mailto:b.j.smith at ieee.org   http://thebs413.blogspot.com
--------------------------------------------------------
        Fission Power:  An Inconvenient Solution