[TransWarp] making progress - examples in cvs

[Ulrich Eck]

hi out there,

for any interested people an update on our work:

in our mission to build a scriptable user-management
(ldap-accounts + imap-mailboxes) we make good progress using
the latest and greatest peak-capabilities.

you can have a look at our code at:
http://cvs.net-labs.de -> apps -> acmgr
(or check it out and play with it)
cvs -d:pserver:anonymous at cvs.net-labs.de:/data/cvsroot login
passwd: anonymous
cvs -d:pserver:anonymous at cvs.net-labs.de:/data/cvsroot co apps/acmgr
(in case the name doesn't resolve: IP 217.89.141.27)


what has been done so far:
----------------------------------------------------------------
IMAP:
- NamingProvider for IMAPConnections
- IMAP Elements (Folder, Message, ACL) -> Model
- Datamangers (EntityDM's for data-access, QueryLinks/QueryDMs
               for folder/message listing, LazyLoader for MessageParts)

open question:

message-listing of a folder is a QueryLink to a MessageListQuery
that retrieves the ids of the messages in this folder.

i tried 2 implementations:

  1. list only the ids and initialize the PersistentQuery with
     [ dm[id] for id in message_ids ]

  2. list the ids and fetch all headers and then initialize the
     Persistent Query with
     [dm.preloadState(<oid>, <somestate>) for xx in message_ids ]

  the first solution gives back a list of message-objects fast
  and allowes using slices of the list (without knowing the content)
  but it is fairly slow because it needs to contact the imap-server
  for every single mail to fetch the header-info (dm._load(xxx))

  the second solution needs some time to get loaded but has
  preloaded objects with message-headers ready for listing.
  the second solution is good for mailboxes with up to 1000 messages,
  the first solution is better for mailboxes with more messages.

  possible solution: create a policy that switches the implementation
  based on the message-count.

----------------------------------------------------------------
SIEVE (a mail-filtering-language for cyrus-imap):
- NamingProvider for SIEVEConnections
- SIEVE Elements (SieveScript) -> Model
- Datamangers (EntityDM's for access, QueryDM for scriptlisting
               LazyLoader for Script-Download)

----------------------------------------------------------------
LDAP-Account (work has just started and is not yet ready):
- Basic Model (User)
- UserDM for load/save/new of User-Objects.

open questions:
(a lot) but the most important:

what is the oid for the ldap-stored objects -> the DN ??
the DN is the only unique identifier for objects in ldap.

if we use the DN, we need a way to build the DN when creating
objects. we thought of using a similar way as in our IMAP solution:
e.g. an User-Object should be stored in a OrganizationalUnit,
then we would for example do this:
<begin> ...
ou = ouDM['ou=people,dc=net-labs,dc=dev']
user = userDM.newItem()
user.cn = 'Some User'
user.sn = 'User'
user.givenName = 'Some'
ou.addChild(user)
<commit> ...

the newly created user-object would be stored using the following _p_oid:
def _new(self, ob)

    oid = 'cn=%s, %s' % (ob.cn, self.ouDM.oidFor(ob.parent))

where ou.child is a model.Collection
and user.parent is a model.Attribute with proper referencedEnd declarations.

and there comes another question what should "def _thunk(self, ob)" do ??

if i would say in the above example:

    oid = 'cn=%s, %s' % (ob.cn, self.oidFor(ob.parent))
                                ^^^^^^^^^^
then i need to implement the userDM._thunk(self, ob) method.

any hints for this one ??
----------------------------------------------------------------

some example-code for using the imap/sieve stuff is in the
acmgr/src/acmgr/connection/tests directory.

i'm pretty impressed that a non trivial thing like accessing
IMAP in a nice python-object-tree is fairly simple using
peak.storage and it seems to have a really good performance
due to its load-on-demand design.

i like it very much :)))

comments on concepts and code are welcome.


Ulrich Eck
------------------------------------------------------------------------
net-labs Systemhaus GmbH
Ebersberger Str. 46
85570 Markt Schwaben
fon:   +49-8121-4747-11
fax:   +49-8121-4747-77
email: ueck at net-labs.de
http://www.net-labs.de