[03:46:26] --> MikeT has joined #peak [09:39:53] --> jack-e has joined #peak [10:40:00] * jack-e is gone (12:40PM): .. autoaway .. [11:16:50] --> jack-e has joined #peak [12:09:05] --- jack-e is now known as jack-e|away [12:29:36] --> teef has joined #peak [13:10:00] * jack-e|away is gone (3:10PM): .. autoaway .. [14:03:17] --> teef has joined #peak [14:03:59] Any one want to talk about MOM (Ulrich?) [14:06:21] I have written my own Middleware server... Ok it is really simple but I have a messaging service and I heard some one here might want to talk about MOM's? [14:08:35] My credentials are I write EJB code all day at work (Jboss/Jetty/Postgres) using JMS/CMP/SessionBeans ect... This stuff is way to complex and is actually stupid sometimes there should be a better Python alt. [14:09:54] --> Maniac has joined #peak [14:21:20] --- jack-e|away is now known as jack-e [14:21:26] teef: i [14:21:31] 'm ulrich [14:21:40] .. just got back to my office [14:38:30] w/b jack-e [14:45:19] heya maniac [14:46:38] sorry just got back from a meeting [14:48:35] hello [14:48:49] Have you been working on a MOM/ [14:48:52] ? [14:49:16] yes [14:49:57] I have been doing a little dabling myself. [14:50:10] moment please be right back .. [14:51:18] back [14:51:25] teef: have you been working with peak ?? [14:51:43] jack-e: nope [14:52:26] ok .. peak has a fairly simple to use but powerfull implementation that is fairly similar (in its functionality not complexity) with JNDI [14:52:29] jack-e: I had started writing my own one a couple months ago and then I stumbled on PEAK. [14:53:15] i'ld like to simplify JMS specs, put away all the overhead to get a similar package usable for peak for interchanging messages between business objects [14:53:35] teef: i'm working on that at the moment .. [14:54:10] jack-e: Here is the my message syntax. [14:54:25] prc = PRMCClient() mp = prc.lookup('MessagePit') mp.publish('log/logger', 'TESTING THE logger is really cool') [14:55:19] lookup returns a Queue-session ?? [14:55:27] DDDDDDDDDDCCAAABBBB [14:55:49] jack-e: Acually this one is a topic. [14:56:18] i think i need to restart my client . i don't see any messages anymore .. brb [14:56:39] --> jack-e has joined #peak [14:56:49] re [14:57:10] the last i saw was you example for publishing a message [14:57:16] jack-e: sorry [14:57:27] somehow i didn't see any messages anymore [14:57:28] This is a topic example. [14:57:36] ok [14:57:55] that shouldn't be too different from a Queue one, right? [14:58:13] No not at all. [14:58:38] i like the idea of unifying these two message-systems [14:58:53] Topic many subscribers get the same message where as a queue only one subscriber gets the message. [15:00:08] there are more differences behind the scences but the interface how they'll be used is very similar [15:00:09] I use xml to configure everything. So I can put a queue MessagePit and a topic MessagePit or anything else just different lookup names :) [15:00:50] I think the interfaces should not be much more complicated than my current example. [15:01:01] right [15:01:11] is your project opensource ?? [15:01:13] JMS i have been using and it is a nightmare!!!! [15:01:34] It has been a hoby so far but I plan on opensourcing it. [15:02:02] It is starting to get useful. [15:02:43] I can package up the source for you if you want to take a look. I think I will GPL or LGPL all the code which every make people happier. [15:04:09] It would be really easy to write different types of MessageServices topics and queues. All you have to do is change the package name in the xml file :) [15:05:06] ok .. i'm working on several components that would form a Message Queue/Topic Server/Client [15:05:44] do you have the interfaces or some code? [15:06:17] i'm doing all my development opensource and it is available at cvs.net-labs.de (apps/nll/src/nll/messaging is the part we're talking about) [15:06:42] i have the following goals with the messaging system: [15:07:20] - should work locally and remote (using different Network-Transports e.g. Pyro, Jabber, Twisted.spread, Corba) [15:08:00] - simple interface for using Point-To-Point (Queue) or Point-To-Multipoint (Topic) systems [15:08:41] - possibly integrate other (probably commercial) systems via a "driver" to be able to integrate into larger systems [15:09:28] - mix different implementations (e.g. MemoryQueue and networked, persistent Queue) without the need to use them differently [15:09:34] does this make sense ? [15:09:57] Yes i agree [15:10:22] in nll.messaging i just started with the JMS Interface and try to simplify where it makes sense [15:10:33] JMS got cought with the commercial integeration and that is how I beleive it got screwed up just as a note.. [15:10:55] because of that i started writing some very simple implementations of different backends (Filesystem, Memory, bsddb3= [15:11:02] EJB for that matter is broken because of the Oracle,IBM and Sun [15:11:27] i'ld love to learn from the mistakes made by others ;-))) [15:12:58] i don't need to have all features available by other commercial messaging systems .. i just wanted to say, that my implementation will seperate the queue backend from the Infrastructure implementation [15:13:08] to make it possible to build drivers/connectors [15:13:49] My plan was to have the message service be an interface. So anyone could write there own. My current implemementation is very simple but I have implemented datasources so it would be easy to have a database driven message service and the datasource. Easy to implement a Memory or even filesystem. [15:14:42] The driver I use by changing the xml file. [15:15:52] [15:15:58] i've a similar concept, but i wanted to have one MessageService implementation and configure the available Queues and their implementations/datasource/backend independently [15:16:47] is class the queue implementation or the message-class ? [15:17:27] THat is the queue the message class is in a seperate deploy.xml file. THe datasources are in a seperate datasource.xml file [15:17:44] log/Logger net.axiomis.contrib.LoggerEgg [15:17:50] That came out bad [15:18:05] np [15:18:11] name, class, queue name where the import parts [15:18:48] the configuration-system you use, is self-made or did you use some existing packages ?? [15:19:04] name is the lookup name, class is the implementation and queue is the name of the queue to subscribe from. [15:19:15] I made it myself. [15:19:36] I am a firm beleiver in XML for system configuration. [15:19:46] ok [15:20:49] i'ld like to study your concepts/code and have some more talks about this issue [15:20:50] It doesn't really matter the configuration mechanism but it seemd the simplest I could figure out. [15:21:29] i'm using ZConfig and the peak-config system for that [15:21:31] Most of my code and concepts come from EJB but I have added some sugar . [15:22:27] I think this shouldn't matter. The interfaces are the important part. How the queue gets its information is up to it. [15:23:12] Sorry that didn't make sence how the queue gets it configuration is up to the implementation but there should be an interface that describes the method calls and the data need. [15:23:49] you're right [15:24:27] when i talked to Phillip Eby about the interface, he just said: you need a get and a put method for a Message Queue [15:24:36] My wacky ideas about configuration and all are just because I stare at xml all day so it easy for me to configure servers that way. [15:24:53] What does that mean? [15:25:15] when talking about the Interface of a MessageQueue system for peak [15:25:53] so what you really need for a MessageQueue would be a put(message) and a get() ;-) [15:26:38] I used publish and process but that is just me. I used subscribe to add to the list of subsribers for a queue and publish for pushlish a message on the queue and process to process the messages. [15:27:51] so your system can be used in eventloop or multi-threaded sytems ? [15:28:05] This is fine. I don't actually think that describes what really is going on though. Put yes but get not really when a message is message listener gets a message it chews on it and does somthing but I also wrote it. [15:28:42] Yes it is multithread but not a good implementation. [15:29:10] I have to run for coffee let me get back to you in 10 minutes I have to have a coffee chat with some on in the office. [15:29:33] sure [15:29:35] shure .. [16:08:09] sorry I am back [16:08:36] ok [16:08:58] jacke-e: One of the things I found intreasting when I was writing my server. [16:10:23] There are a lot of different was to process messages. And I didn't want to limit the possible ways to do it for example schedule process, priority processing and possible even distributed proceessing. [16:11:00] was == ways ?? [16:11:14] Because of this I want to make a very abstarct interface.. (I didn't make the interface but just the implementation but I ment to) [16:11:22] ways [16:11:26] sorry [16:13:01] Some queues might get burst of messages so you want to queue when you have a peak and process a lot of message when you have a low. This started me thinking that threading was better to provide for more flexability in the message processing. [16:13:20] i haven't yet made it that far ;-) [16:13:45] I speak a lot of theory but have very little implementation. [16:13:59] ok .. same for me mostly ;-) [16:14:55] Implementing the mom made me have ideas about the interfaces specifically exaclty what that actually do but this may not make sense for the black box developer. [16:16:41] there is an interesting mail from phillip eby on the transwarp mailinglist: http://www.eby-sarna.com/pipermail/transwarp/2002-August/000187.html [16:16:48] and some follow-up discussions [16:18:34] about peak messaging and requirements .. this is more or less the requirements-catalog for my work [16:20:44] That is pretty heavy. [16:21:26] yep .. i know .. and i'm trying to sort out stuff that i don't need right now .. but keep doors open for later implementation [16:21:43] does your system has persistent and transaction support ? [16:21:54] s/persistent/persistence [16:21:58] Seeing as though CRITICAL: Messages must be secure (and) * CRITICAL: Asynchronous "fire-and-forget" . This would be really hard trying to get performance would be hard [16:23:32] i've had some discussion about security with phillip .. i think i'ld suffice to have ssl-encrypted communication between the queue and clients for example [16:23:35] Transactions are tied to the threads and this really needs some work. I have tested it but I know I need to do a little core code redesign to make it more maintable. Transactions a supported from a db perspective. [16:24:03] you use a SQL-db as backend for your Queue ? [16:24:29] This is optional. Right now it is in memory but it would be really easy to make in sql [16:24:53] or filesystem. My implementation is very simple. [16:25:44] My idea is that you could just change the package name of the class and inmplement 3 methods and you could have any kind of persistance that you wanted. [16:26:02] same for me [16:26:48] i want to use as many parts from peak as posssible e.g. Datamanager for storing stuff in DB/FS/..., peak.naming to resolve stuff, etc [16:27:14] This would be cool to be able to have plug and play implemintaions that could work in different server enviornments. [16:28:36] that's what i want to achieve .. i'm working on a simple persistence backend that uses berkely-db [16:28:58] you can then reconfigure your queue-manager to use a SQL-Backend [16:29:16] and on the front-side you can choose from several transports [16:29:18] I was using postgresql but that is just because I had it install I wanted to try and use sqllite or something simple like that. [16:30:41] I was planning on writing a sql messaging service which would work with and SQL db and just change the datasource name to change the database for storage. [16:32:43] i want a modular setup .. e.g. you compose a Messaging Server from a Queue-Manager, a Queue-Backend and a transport [16:32:59] i think we have similar goals [16:33:44] are you interested in sharing some ideas about the overall setup ? [16:33:53] Defiantly have simular goals. I haven't looked much at the modular setup is it just subclassing? [16:33:57] implementation and the like ? [16:34:02] Definatly [16:34:33] peak works with late-binding of components [16:34:44] I believe that we could build a really nice spec/interfaces . [16:35:02] that would be a good start :) [16:36:03] How does peak do it naming is it simular to mine m = prc.lookup("MessageService) ? [16:37:20] I think this is the most important part before hacking out a spec for the MOM. [16:38:00] you can define your own providers that care about urls (messaging.queue: could be resolved by a naming-provider - a call naming.lookup(context, 'messaging.queue:MessagePit', adaptTo=IMessageQueue) would get you a MessageQueue instance) [16:38:01] I use the lookup for finding datasource and well just any resource including Control classes. [16:38:30] you can do a naming.lookup(IMessageService) to get some utility that is found in the Component-Tree [16:38:43] That is very cool [16:38:53] adaptTo is the interface? [16:39:56] to create an SQL-Connection you'ld do a naming.lookup(ctx, 'postgres://user:passwd@host/dbname', adaptTo=ISQLConnection) [16:40:33] yup adaptTo tells the mechanism to adapt the resulting URL-Object to a Isomething (you need to provide an adapter for this) [16:40:52] this is very cool what is in the ctx? This is I assume the url/port of the nameing service? [16:41:07] no there is no naming-service basically [16:41:13] ctx is a context [16:41:31] What is in the context? [16:41:35] a peak application is composed from components that know their parents [16:41:49] so the place where you are (the component) is your context [16:41:59] there can be local-configuration for a component [16:42:14] and a component has a parent [16:42:16] Ahh the tree. [16:42:19] so when you lookup something [16:42:29] it looks up the local configf [16:42:48] if nothing is there that matches it wanders up the tree till it reaches a root-config-context [16:43:02] this is defined mostly though the peak.ini configuration [16:43:16] you can provide global utilities in a ini-style config-file [16:43:25] so ctx could be a myApp.ini file ? [16:43:43] no context is the place (in most cases it's "self") [16:44:32] there are methods that allow you to lookup and create if necessary objects (peak.binding) [16:44:45] have you worked with zope ?? [16:44:55] Tried and faild [16:45:01] :P [16:45:14] they have something similar called Aquisition [16:46:07] did i explain it in a way that you can imagine how it could work ?? [16:46:13] I have read the PDF Eby wrote I will have to review it again to try and get a better idea how PEAK works. [16:46:30] Yes you did I just had a different idea of it after reading the docs [16:46:48] when you go to cvs.eby-sarna.com [16:47:19] PEAK/examples/bulletins/src/bulletins [16:47:23] I also have to change the way I think about middleware to figure it out. [16:47:24] look at app.py [16:47:35] :) [16:48:55] are you there ? [16:49:00] yep [16:49:09] jack-e: where would an example of naming.lookup(ctx, etc.) be found? [16:49:57] maniac: you don't use naming.lookup() in most cases .. you use self.lookupComponent(url) on peak components usually [16:51:06] what is this bulletins.storage:BulletinDM [16:51:07] teef: the whole app consists mostly of configration [16:51:41] jack-e: ok cause i have some grasp of lookupComponent [16:51:55] binding.New is a binding function that creates a new instance when the attribute is first used [16:52:17] it would create an instance of the class BulletinDM from module bulletins.storage [16:52:37] how do you change the datasource? [16:52:58] look at /examples/bulletins/bulletins [16:53:32] this is the main application configuration file [16:53:52] jack-e: when you do a offerAs=[blah] you are (hopefully stating the obvious) giving ability of other components to perform lookups and obtain that object [16:53:56] ? [16:54:43] Maniac: yes .. the late-binding requires to be explicit in this case .. you need to "offerAs" some Interface/Property to be able to look it up later [16:55:02] teef: are you at the configfile ?? [16:55:30] yep [16:55:48] you see a [bulletins]\ndatabaseURL = 'someurl' there [16:56:26] this is a Property called 'bulletins.databaseURL' that can be looked up with the config-system (e.g. binding.bindToProperty('bulletins.databaseURL') [16:56:30] I am trying to figure it out. Are configurations per package? [16:56:38] no [16:56:44] Acaully I don't see the databaseURL? [16:57:02] are you in the bulletins package ?? [16:57:10] sorry wrong place I was in bulletins.ini [16:57:12] http://cvs.eby-sarna.com/PEAK/examples/bulletins/bulletins?rev=1.2&content-type=text/vnd.viewcvs-markup [16:57:15] now I see it [16:57:28] you can "layer" config-files [16:57:42] later loaded config-files can overwrite earlier loaded [16:57:59] what is the order of the loading? [16:58:25] can be defined custom .. normaly only the peak.ini file is loaded [16:58:37] when you create an app [16:58:50] you can create a root-context with config.makeRoot() [16:59:15] this method takes argument iniFiles=(list of inifiles) to control this [16:59:33] you can also define a env-variable PEAK_CONFIG for that [16:59:40] you can also declare PEAK_CONFIG='myIni.ini' [16:59:47] :) [16:59:47] yup [16:59:57] back to configuration [17:00:51] you have defined the property 'bulletins.databaseURL' [17:00:55] in the ini-file [17:01:40] go back to the src/bulletins/app.py file [17:01:52] you see the db=binding.Once(... ) construct [17:01:53] ok [17:02:25] there is a call to self.lookupComponent(self.dbURL, adaptTo=storage.ISQLConnection) [17:02:30] this means the following: [17:03:06] get self.dbURL -> this is an attribute defined in the config-file as name 'bulletins.databaseURL' [17:03:28] when you first say self.dbURL the config-system is asked to load that property [17:03:46] you defined this one in the bulletins file we say before [17:03:56] s/say/saw [17:04:37] lookupComponent now has a string-url 'sqlite:///tmp/bulletins.db' [17:04:52] for the first attribute (the thing that should be looked up [17:04:53] ) [17:05:10] the algorithm for looking up something is roughly like that: [17:05:17] lookupComponent allways passes the interface? [17:05:18] 1. check if it's an url [17:05:49] 2. if it's an url (urls are found by searching for a : in the string, and then check if there is a provider for the protocol) [17:06:06] then the naming.lookup(ctx, url) is called [17:06:28] ctx in this case is the component you have this .. ergo: the App-instance [17:07:13] 3. if it's not an url, and it is a string, the system tries to find an attribute with this name [17:07:41] 4. if it is an Interface, it tries to find an provider for this interface up somewhere in the tree [17:07:58] lookupComponent does not need an interface [17:08:32] but you can say adaptTo in many methods of peak to declare that you want the result of your call be adapted to the interface you specified [17:08:47] so in our case the following happens: [17:09:08] url is resolved to a SQLite-URL instance [17:09:38] the SQLite-URL instance is adapted to an ISQLConnection which results in a connection-object ready to connect to your sqlite-db [17:09:55] it's very simple to use but still powerfull and customizable [17:10:21] could you understand my explanation ? [17:10:35] hold on a second [17:11:08] ok [17:12:22] sorry some one poped in the office for a second [17:12:38] no problem [17:12:38] yes it makes sense [17:13:25] The lookup is what I am most instreasted in for compatability. [17:13:49] yes it decouples the parts of a system [17:14:29] Weather it is local or remote lookup the client should be the same and I am trying to figure out how my lookup could be the same as PEAK [17:15:15] what do you mean by the "lookup be the same as PEAK" .. why not use peak ?? [17:15:50] you can have some config entry that says if you need to get a remote-proxy to your queue or have it local .. right ?? [17:16:28] My application server has a container that includes control object, schedule object, persitance objects and ours truely message objects which use lookups [17:17:55] how do you communicate with your clients ?? [17:18:43] I have remote clients that talk over sockets and the local clients talk dirrectly to the objects in the vm [17:19:07] Neither client has a clue if it is local or remote. [17:19:20] so you implemented your own protocol for communication ? [17:20:19] the client gets some configuration that says if its local or remote right ? [17:20:22] errrr. Not by choice I just did to get it working so I could see my app server work and now it seems to be working alright but I would swithc in a second [17:20:49] ok .. i'ld like to evaluate Twisted's rpc-system [17:21:15] Yeah sorta I have gone back and forth about this. [17:21:20] it works asynchronous and has a really nice integration into an event-loop [17:21:29] same for me :)) .. [17:21:40] yes I was think about this exact same thing. [17:21:50] i cannot really say that i like twisted, but they seem to have done great work in some areas ... [17:22:14] i used pyro a lot recently (and contributed) [17:22:19] Now that I actually want to start using and sharing my code I would be happy to replace component that for lack of a better word suck. But I don't want to lose the usability. [17:22:47] it is a very nice rpc-system for prototyping and simple systems .. [17:22:52] I was planning on using pyro myself but it got complicated I thought. [17:23:13] it has the drawback that it uses pickle that cannot be made totally secure [17:23:25] i found pyro really simple to use [17:24:17] pickle over ssl would work? [17:24:53] yes .. that's why we implemented ssl for pyro :)) .. but if someone steals your identity, he can really screw up the server and you have no way to stop him [17:25:40] the twisted serialization protocol is fast, interoperable (java/emacs client) and has no problems like pickle .. [17:25:41] I didn't think pyro would be running over public network? [17:25:49] right :) [17:26:07] you can run it, but i wouldn't do so .. [17:26:55] All the J2EE servers can only have the web stuff accessable from public otherwise they would be as easy as nfs to hack. [17:27:12] :) [17:27:53] I was planning on using webservices for exposing functionality to the public network. [17:28:26] Anyway I think I would like to wrap my simplisity up or just use the twisted framework because it is just better. [17:29:45] I had fun playing with the sockets anyway and I now know a lot more than I did about import things. I only have to rewrite two methods to use the twisted framwework.. (I hope) [17:30:04] fine .. [17:30:55] I am trying to move from the experiment stage to the this might actually be useful stage of my project. [17:31:04] would you be willing to send me your current stuff that i can play with it abit and see what you've done ?? [17:31:45] I will send it write now.. Give me couple minutes to doc it up a little. [17:32:01] i'ld really like to help you peakify the system ;-) [17:32:14] ql .. my stuff is all available in our cvs.net-labs.de server [17:32:15] Hey I am open to anything. [17:32:30] if you want to checkout stuff to your disk: [17:32:44] cvs -d:pserver:anonymous@cvs.net-labs.de:/data/cvs login [17:32:52] password: anonymous [17:32:55] Wintel or Unix? [17:33:03] linux [17:33:24] what are you using ? [17:33:38] linux [17:33:44] sometimes freebsd [17:34:02] but I don't have a wintel start script that is why I ask. Just start.sh [17:34:16] :) [17:34:31] cvs -d:pserver:anonymous@cvs.net-labs.de:/data/cvs co apps/nll for example [17:34:57] the nll (net-labs libs) is in experiment status as well currently [17:35:25] so please do not expect that all the stuff works as expected .. i have many open todo's on it ... [17:35:39] need to do that. [17:36:13] so .. i need to get some stuff done for today .. it was really nice and interesting to chat with you [17:36:41] i'm looking forward to look at your stuff and hope that we keep in touch [17:36:46] I will catch you later after I get your stuff down and get some time to look it over and get a better feel how PEAK works. [17:36:56] I can't seem to co app/nll [17:37:03] apps/nll [17:37:27] yeah that one [17:37:33] cvs -d:pserver:anonymous@cvs.net-labs.de:/data/cvs co apps/nll [17:37:48] does it work now ? [17:38:03] nope [17:38:22] moment [17:38:56] but you get the viewcvs pages when browsing to http://cvs.net-labs.de ??? [17:39:14] yes [17:40:16] from a server outside in the internet tested: [17:40:19] p15106492:/tmp/test# cvs -d:pserver:anonymous@cvs.net-labs.de:/data/cvs login [17:40:19] Logging in to :pserver:anonymous@cvs.net-labs.de:2401/data/cvs [17:40:19] CVS password: [17:40:19] p15106492:/tmp/test# cvs -d:pserver:anonymous@cvs.net-labs.de:/data/cvs co apps/nll [17:40:19] cvs server: Updating apps/nll [17:40:29] works [17:40:37] what ip do you get for cvs.net-labs.de ?? [17:42:27] hang on [17:43:11] cvs.net-labs.de. 604229 IN A 217.89.141.27 [17:43:31] hmm .. that is the old ipaddr .. i moved the server last week [17:43:49] it has now the 217.89.141.28 [17:43:55] that's still the ip i get [17:44:04] you can try the ip instead of the hostname [17:44:23] are you surfing through a web-proxy ?? [17:44:31] no bad isp [17:44:40] :) [17:45:31] thanks I will send you and email sometime today with my crap I got to actual get to work. Mettings and all [17:45:47] same for me .. bye [17:45:51] cho [17:46:42] well, i restarted my dnscache and it resolves ok.... [17:47:07] ok [17:48:51] --- jack-e is now known as jack-e|away [17:56:12] Howdy-- are any of the MOM folks still around? The idea of a messaging system using PEAK sounds pretty cool, and I could probably help out a bit w/ that. I'm writing a python jabber server using twisted, and I'd like to PEAK-ify it if possible to allow easy composing of protocol handlers. [17:57:17] jolby: i need to get out of office now .. but i think i'll be back later (need to check this with my girl-friend ...) [17:57:27] Along those same lines-- has anyone here PEAK-ified a simple twisted server? [17:57:49] jolby: another think i'm interested to talk about :) [17:58:08] jack-e|away: ok cool, I should be around later. If not later today, then tomorrow. [17:58:25] fine .. i get into touch with you .. cu later [17:59:32] jack-e|away: ok, sounds good. [18:02:50] jolby: a twisted-jabber server? [18:03:07] jolby: how about a twisted jabber client? [18:08:07] Hi Maniac-- I'm working on a client as well in parrallel w/ the server. [18:08:50] I'm not sure if I'm going to use twisted for the client tho-- because of some reported probs using twisted + wxPython... :-( [18:15:23] just a question, why are you writing your own wxPython client? [18:16:07] is your server code publicly available? [18:19:15] The core of the client+server will be open-source (probably LGPL), but we will have extensions that will be proprietary [18:20:45] The extensions + the core + doco and support will be our product. [18:21:21] ah ic [18:21:25] so nothing right now? [18:25:55] I think the core server could be popped out into public pretty soon. I would need to decide where I would have it hosted and do all that setup. SF would be the easy path, but they've had some performance probs recently... [18:44:54] sf has intermittant cvs problems that are aggravating [18:59:01] * jack-e|away is gone (8:59PM): .. autoaway .. [20:02:00] * jack-e|away is gone (10:02PM): .. autoaway .. [20:13:12] * jack-e|away has returned... [atl] [20:13:16] --- jack-e|away is now known as jack-e [20:13:28] re [20:15:15] re? [20:15:23] i'm in mental lock [20:15:30] reentrance .. [20:15:34] ah [20:15:54] is a habbit from my earlier irc-days ;-) [20:16:06] i've only been an IRC 'junkie' for about 2 years [20:16:15] i do have a quesitno tho [20:16:31] go ahaed [20:16:33] although it's pretty stupid [20:16:59] since in PEAK you just lookup components. what happens if two of the 'same' components advertise themselves as the same thing [20:17:30] last come will serve i think [20:17:37] latest [20:18:42] i have an application which records 'customer concerns' i need a way to link 'contacts' to customer concerns. I'm trying to think of a good way to perform that linkage with different 'backends' [20:18:46] that means, if you declare something in code after some other definition of the same interface or propertyname the last will win (it's just a dictionary of configkey->value that is filled and locked after the first read) [20:19:30] you have 2 element-classes with features already defined ?? [20:19:59] 2 element-classes? [20:20:43] if you store something in a backend (e.g. sql) you want persistent baseclasses that have features .. e.g. subclass model.Element and give them model.StructuralFeature s) [20:20:56] let's look at the bulletins example again [20:21:49] ok, no i have not defined them in terms of DM's yet. nor model's [20:22:08] i tend to overthink things in terms of future problems (that will probably never arise) [20:22:13] how do you achieve it then (persistence, backend, etc) [20:22:14] ? [20:22:42] so what hinders you to use DM/Element ? [20:22:44] brb [20:22:48] it's not peakish yet. based on some zope scripts [20:23:36] ok [20:23:55] but you started to use peak (using the binding stuff and properties) ? [20:24:27] zope scripts and peak ??? [20:24:30] i have started to use (on a very minimal basis) some config and naming [20:24:57] no, it's zope now, i have tiny test scripts pulling hte info from postgresql which is entered in a zope web UI [20:25:04] how would zope (i assume v2.x) work with peak ?? [20:25:25] go ahead . [20:25:31] in other words, they are not working (nor will ever) together [20:25:45] i'm attempting to re-architecht [20:25:48] ok [20:26:09] so you have a postgres-db as backend and try to re-architekt you app [20:26:26] then i think it's time to dive into peak.storage :) [20:26:27] correct [20:26:31] and peak.model [20:26:37] the idea being in the long term i want to be able to swap out postgresql for some aspects [20:26:48] replace with what ? [20:27:14] i.e. I want a contact-manager that supports connect/disconnected modes, so a central postgresql-db and a disconnected [somethingelse] [20:27:32] but again, probably over-thinking things [20:28:32] these are imho more arguments to use datamanagers :) .. you can link several dm's together [20:28:50] in terms of dm chaining? [20:29:01] yes [20:29:05] or fallbacks [20:29:16] chaining is possible now [20:29:54] fallback is not implemented but i think it wouldn't be too hard if you chain a "fallbackDM" with a set of configured backendDMs [20:30:31] or have some config entry which DM is used currently and a sync-option to keep them in sync [20:30:51] ok, i'll ask you 'cause your a smart guy. It's not peak specific. [20:31:09] who told you that i'm a smart guy ??? [20:31:21] i can tell [20:31:31] :) [20:31:33] ok .. [20:31:39] i accept that :) [20:32:11] i have a 'customer complaint' that i want to map linkages to 'contacts' 'customers' 'products' etc. the best i've come up with so far is a table-> app1id,app1link,app2id,app2link [20:32:22] and then search that table for linkages [20:32:25] it's very ugly [20:33:21] so you have an mxn-relation where n is of some unkown type .. right ?? [20:33:48] basically yes [20:34:12] i think these problems are allways hard to solve in a non-ugly way :) [20:34:59] but the concern is allways of the same object-type or can it vary as well ?? [20:35:12] concern is the same object-type [20:35:19] BUT i want these linkages to go both ways [20:35:39] so customer can link to concern as well ? [20:35:42] i.e. if i lookup a contact i want to spit out all related concerns. If i lookup a concern spit out all related contacts etc. [20:35:43] yes [20:36:25] you can allways do this with looking up the inverse relation (e.g. query for contracts where this customer is linked) [20:36:44] not contracts but concerns [20:37:21] but can one concern relate to more than one object e.g. product, customer, .. ? [20:37:27] oh yes [20:37:47] product, customer, notes, etc etc. [20:37:50] ok [20:38:38] btw jack-e, not to distract you but this is cool: http://www.nat.org/dashboard [20:40:23] you need some table like you had .. i'ld try to come-up with something like columns: CONCERN_ID, CLASS_TYPE, CLASS_ID where you have some configuration in your code that maps a class-name to the responsible DataManager [20:40:29] does that make sense ? [20:41:19] s/class-name/CLASS_TYPE [20:41:43] i think that's inline with my thoughts [20:41:47] i was thinking [20:42:53] that doing SQL lookups on CLASS_TYPE might a performance hit on large tables vs integers [20:43:49] but again, i was thinking a 'generic' link interface that would allow aritrary objects (contacts, concerns, notes, etc.) to link to each other [20:44:29] you could then just say: LEFT_CLASS_TYPE, LEFT_ID, RIGHT_CLASS_TYPE, RIGHT_ID [20:44:36] basically the same as you did before [20:44:44] yea, basically what i was doing, but probably prettier [20:44:52] i could think of some other way to do it: [20:45:21] if linkage is a central part of your application, you could do the following: [20:45:33] but by using CLASS_TYPE i can perform lookups in a PEAKish way (via a config file) etc. [20:46:16] every object has one entry in a common table and provides a unique id over all objects. [20:46:40] every object has then a table for its values, and a link to the OBJECT_TABLE [20:47:04] then there is one Link-Table: OBJECT_ID_LEFT, OBJECT_ID_RIGHT [20:47:12] to map arbitary objects together .. [20:47:50] there is just one part missing, how to find an OBJECT by it's id :-) where you'ld probably end up with something simliar .. pointing to the table where it is stored [20:48:17] but i think this way makes more problems than it solves [20:48:50] certainly seems more complicated :O [20:49:10] i think i like LEFT_CLASS_TYPE,LEFT_ID,RIGHT_CLASS_TYPE,RIGHT_ID [20:49:53] so you need some configurable mapping from which you define which Datamanager is responsible for a CLASS_TYPE [20:50:02] yes [20:50:58] the assumption being (unless there's some sort of syncing) that once you specify a DM in that mapping the LEFT_ID and RIGHT_ID would probably lose relevance if you 'swap' backends [20:51:01] then you need a QueryDM (the implementation will retrieve a record, check the CLASS_TYPE, get the responsible Datamanager and load CLASS_TYPES_DM[RIGHT_ID] [20:51:53] if you plan to be able to switch backends you should really take care to have unique-identifiers for all your objects i think [20:52:29] what hinders you to have a unique-integer value for your objects in an SQL vs. XML/ZODB database ?? [20:52:45] i have no idea [20:52:47] i don't see a problem .. your backend just needs to be able to query for them [20:53:11] peak.util has some GUID stuff no? [20:53:49] right .. but it calculates GUIDs (globally unique ids) .. they might be quite expensive to generate in masses [20:53:54] the primary keys should be globally-unique-interger values [20:54:11] leaf globally ;_) [20:54:13] out [20:54:34] just have one SEQUENCE in your database that you ask for a new value everytime you create an object [20:55:03] if you do that, how do you maintain uniqueness across DM's [20:56:02] one question in between: will several users have their systems offline at the same time and create new objects while they'r offline and then must be able to sync them back into the main database ??? [20:56:10] yes [20:56:17] or at least i want to support that [20:56:27] that makes it definitly harder i think .. [20:56:36] yes, it does, syncing is hard :) [20:56:37] you cannot "take the sequence" with you [20:56:52] which is why GUID's are floating in my head [20:57:00] so you'll probably end up with GUID's [20:57:16] certainly in terms of 'CONTACTS' i would want various disconnected modes [20:57:19] they're a mix of mac-addr-info, time, and random [20:58:06] it's at this point that i start thinking i'm over-designing [20:58:22] :) .. [20:58:34] ... and enter a mental lock [20:58:39] you could start simple with a DB-SEQUENCE (it's unique in it's domain) [20:58:58] when you're at a point where you want to enable the offline-mode [20:59:26] you could update all integers to unique-identifiers and go ahead [20:59:47] if implemented in a good way, your application shouldn't be changed [21:00:07] but if you can call yagni on it, you make your life much easier [21:00:32] [you aint gonna need it?] [21:00:36] yup [21:00:44] i [21:00:49] would start without :) [21:01:04] yeah, and just use db-sequence for now [21:01:52] so .. i interrupted when i was talking about how to implement the link-system [21:02:41] you're Element-Classes would have a Feature (LinkedObjects(Collection):referencedType="model/Any") [21:03:41] this feature's state is filled with a QueryLink that executes a query on the LINK_TABLE and returns states for Different objects (decision made with CLASS_TYPE and the configured mapping) [21:03:55] is this a bit clearer now ?? [21:04:17] i have the basics, if/when i get around to doing it i may ask for hand-holding :) [21:04:29] * Maniac is cutting out this chat-buffer and inserting into notes [21:05:09] to start i need to look into peak.model and peak.storage and get my stuff using DM's [21:05:30] i can use pje's example to wrap my head around that as his example is SQL based [21:05:36] then grow from there [21:06:02] and then call yagni as much as possible, but i do like the basic ideas your've outlined [21:06:22] :) .. just look at the bulletins thingie e.g. model.py Category->bulletins and storage.py BulletinsForCategoryDM and CategoryDM -> stateFromRow look at QueryLink part [21:06:36] ok [21:06:42] good luck [21:06:50] thanks :) [21:07:07] it will be slow going as i (as i've said before) am not a programmer i'm a wannabe [21:07:33] i've learned everything by doing as well .. [21:12:29] i think in terms of sync, that i would let the disconnected client maintain a mapping of GUID to db-sequence and push stuff anyways [21:13:14] or the server could maintain some sort of mapping and come up with a proper sync interface [21:14:31] the problem allways occurs, if two offline users change some linkage/create objects and link/delete objects and unlink while their both offline [21:14:45] and then come back an resync [21:15:22] to avoid having duplicates you need to ensure they never are able to create the same UID that's why you'ld need a GUID [21:15:26] yes [21:16:28] it would be sufficient, if you save what UID's you created locally while you've been offline [21:17:03] you would then want to update with server-side UID's [21:17:03] and then ask the server for a sequence of new (UID in the domain of the appserver) UIDs and convert them while syncing back [21:17:09] no [21:17:21] you would maintain local-UID's while you're offile [21:17:41] i agree, convert them when syncing back [21:17:43] and when you sync back, your UID's change to the ones that you got from the server [21:17:47] yep [21:17:55] that's what i ment by update with server-side [21:18:16] ah .. ok [21:18:38] i think that'ld be a fairly straight forward solution [21:20:23] thanks for your input jack-e [21:20:51] you're welcome .. [21:21:55] so many problems to solve so few hours in the day [21:22:02] btw did you take a look at dashboard [21:22:15] that would be cool implemented in python [21:23:14] what do you mean .. the dasboard, or the stuff they'r talking about ?? [21:23:59] the dashboard, i.e. an app that sits there and pulls up relevant information based on what your currently working on [21:24:26] isn't that basically a blog ?? [21:24:32] i.e. Maniac is chatting on IRC with jack-e and dashboard pulls up last x emails from jack-e, last cvs-commits from jack-e etc. [21:24:50] ok .. intelligent agents ?? [21:24:58] that's basically what hte project is [21:25:02] ok [21:25:07] that site is the 'dashboard' blog updating status [21:25:33] the screen shots have say, talking on IM and last emails come up on dashboard, contact infomration, related documents etc. [21:25:44] it's very cool [21:25:46] imho [21:26:19] there is something called narval from logilab.org available .. i've played with it a bit but cannot judge it [21:26:24] written in python [21:26:47] they have agents that are configurable with rules [21:26:56] sounds somehow similar [21:27:13] it is somewhat similar but it doesn't have the notion of context (afaik) [21:27:21] ok [21:27:35] anyways i thought it was cool :) [21:27:45] :) yeah [21:29:02] constructing models seems pretty straight forward [21:29:26] you can build some constraints into a model [21:29:39] when you take care about them later :) [21:30:04] if you mean defining upper/lower-bounds or types for example you're right [21:30:20] yes [21:30:58] * Maniac notes it's almost 'quittin time' [21:31:21] i'm far over that .. it's 23:30 here [21:32:10] you seem to work very late [21:32:24] yes .. i get up late an i work long :) [21:35:18] http://cvs.gnome.org/bonsai/cvsblame.cgi?file=dashboard%2Ffrontends/dashboard.py&rev=&root=/cvs/gnome [21:35:52] there is some python-frontend already for dashboard .. though i do not know what that means ;-) [21:36:15] yes, it means it can send clue-packets :) [21:36:46] or i mean request [21:36:47] to the dashboard, that does something i clued ?? [21:36:58] s/i/it [21:37:37] well, for example gaim is a frontend which sends information to the backend. the backend is searched and relevant information is updated to the dashboard as i understand it [21:38:34] I have to go home now. later (my home irc client will stay :) ) [21:38:42] <-- Maniac has quit #peak [21:38:54] cu [23:10:12] --> MikeT has joined #peak