[TransWarp] PROPOSAL: Split "SEF" into "CIS" and "SEF"

[Ulrich Eck]

Hi Phillip,

> Comments and questions welcome, especially if anybody has any better ideas
> for names of things.  Also, if you're actively developing based on the
> current preview releases or CVS, and your work would be painfully disrupted
> by this, please let me know.  I'd like to make this change *before* the 0.2
> final release, because I can't write the 0.2 tutorials correctly otherwise.

We are actively developing our NLL BaseLibrary with 0.2 preview and the
changes mentioned above will definitly break most of our code  ...
but the earlier changes that make sense take place the better it is i think.

I've run into several inconviniences when developing the Database stuff 
recently and would like propose changes there as well:

I don't find it very usefull to have all connection-types in one file, if you 
want to supply more than just a few connection-types in longer terms.
Also several Models need more than just one File and should be placed into
seperate directories:

for my Development i use a structure like:

NLL.Database:
 \-Connections.py		# defines QueryResult and ManagedConnection
 |-DataModel.py
 |-SQL:
 |  \-Connection.py	# defines SQLResult,SQLConnection and DBI
 |  |-SimpleSQLModel.py
 |  |-MySQL:
 |  | \-Connection.py	# db-specific things like fieldType-check/conversion
 |  |-sapdb:
 |  | \-Connection.py	# db-specific things like fieldType-check/conversion
 |  ... more SQL-DB-drivers here 
 |
 |-LDAP:
 |  \-Connection.py	# defines LDAPResult, LDAPConnection
 |  |-LDAPModel.py
 |  |-Schema.py
 |
 |-MiniTable:			# uses TW.Utils.MiniTable for InMemoryStorage :)
 |  \-Connection.py	# defines MiniTableResult, MiniTableConnection
 |  |-MiniTableModel.py  
 |
 |-ADSI:
 |  \-Connection.py	# defines ADSIResult, ADSIConnection
 |  |-ASDIModel.py
 .... More Models here


I have a working LDAPModel which covers most of the things that are needed and 
works with OpenLDAP,IBM DS yet. A few issues are still open e.g. what to do
with RecordType-Inheritance and attributes like loadAttrs, keyFields etc.

Yesterday I finished a first draft of a SimpleSQLModel, that is heavily 
inspired by by PyDO. It uses a RecordType Definition like this:

## DataLayer.py
from TW.API import *
from NLL.Database.SQL import SimpleSQLModel
from NLL.Database.SQL.MySQL.Connection import MySQLConnection

class Database(SimpleSQLModel.Database):
	
	class TestTypes(SimpleSQLModel.RecordManager):
		
		rootTypeName = 'TestType'
		
		class TestType(SimpleSQLModel.RecordType):
			
			tableName = 'testtable'
			allKeys = (('id',),)
			
			loadAttrs = (
					('id', 'tinyint(4)' ),
					('name', 'varchar(255)' ),
					('description', 'varchar(255)'),
						)
	
	class SQLConnection(MySQLConnection):
	
		Host='localhost'
		Database='dbtest'
#####

and is made for simple ObjectType->SQLTable/View mappings. everything that 
goes beyond this is probably not doable in a performant way ...  (looking 
forward to see your WarpCORE stuff :).
One can specify sequenced attributes as well and the queries are build with
parameters(bindVariables). So I had to change/extend the signature of 
SQL/DBIConnection object a bit.
It still lacks on docs .. but you know the concepts .. so you'll know what I'm 
doing.

All this stuff you can get/view at:

http://cvs.net-labs.de/viewcvs.cgi/NetLabsLibs/src/NLL/Database/

or per anonymous cvs-checkout:

#cvs -d:pserver:anonymous at cvs.net-labs.de:/data/cvs login
password: <enter>
#cvs -d:pserver:anonymous at cvs.net-labs.de:/data/cvs -z9 co NetLabsLibs

It would be good if you could take a look at NLL.Database.* and tell me, if 
you think it fit's into your concept and if this work could make it into 
TransWarp itself. I tried not to touch any of the Interfaces (oh yes .. they 
need to be written as well :) you defined except the SQLConnection .. that i 
needed to extend for parameter-usage.

Perhaps there is another thing of interest:
NLL.Utils.InstancePool:
a first draft for transparent InstancePooling that
has a first implementation: TransactionAwareInstancePool.py. It creates a
number of Instances  (e.g. Database) and blocks other threads until a 
Transaction has finished its work and an Instance is free for use again.
We talked about this shortly and an example can be found under:
examples/Util/InstancePool


don't expect polished code .. it's under heavy development but most of the
examples should work, if you have a pyro-nameserver running and start
the examples/Services/Transaction/NetTransactionMonitor.py to use
our Distributed TransactionMonitor. (you'll find some calls to get_nllCurrent 
, which is a Mehtod that is added to __builtins__ like ZopeTransactions does 
at well. Current is the context of an incoming Pyro/corba call and it's 
created per thread. It also holds the TransIdentity.)


Thanks

Ulrich Eck
net-labs Systemhaus GmbH
ueck at net-labs.de