[15:01:39] --> You are now talking on #peak [15:01:39] --- Topic for #peak is PEAK development (http://peak.telecommunity.com) [15:01:39] --- Topic for #peak set by jack-e at Fri Jun 20 11:58:28 [15:06:45] hi aloisius [15:06:54] jack-e: hi [15:13:42] jack-e: i have just sent email to transwarp list [15:14:20] Does anybody keep log of this channel? [15:15:32] nope .. no log yet .. but this should be done .. you're right [15:17:02] aloisius: reading your mail [15:18:32] my IRC client runs 24x7, so i can publish logs when some interesting discusion appears [15:24:45] aloisius: we'll see how frequently someone shows up here ;-) [15:27:04] jack-e: do you understand my email? I have a suspicion that some sentences must not be understandable for native english speakers :-) [15:28:12] uhm i'm not a native english speaker (i'm german) .. but i still have some questions regarding your problem :) [15:28:39] are these configurations for developers or end-users ?? [15:31:24] it seems that you want a seperate configuration namespace for every app .. right ?? .. there is no component of one class twice instantiated within one app with different parameters .. right ?? [15:33:46] for the cache of a DM i'ld just do a cache = binding.New('import:some.path.to:Cache') to make it an instance variable [15:36:44] configurations are for developers and i want separate configuration for every application with sections for every path (component usage in hierarchy) [15:37:54] what i could envision is, that you use a seperate custom-ini file (loaded with config.makeRoot(iniFiles=[std+yours])) [15:38:38] then use some xyzAttr = binding.bindToProperty('app.attrs.xyz', default) to configure your app [15:39:16] and then create your app-components with different configuration-roots .. there should be something like app-configurations as well, but i haven't yet used them [15:39:57] this means, that a property of a name "app.attrs.xyz" would have a different value depending on what ini-file you specify for your app-context [15:40:47] would that suffice your needs ?? [15:41:16] satisfy ... [15:42:05] it could be problem, because one library component can be used on more places in one application hierarchy. 'app.attrs.xyz' should be defined in python module. Then I have no chance to address particular component that is used more than once. [15:42:24] right [15:43:14] did you play with ZConfig for that ?? i haven't yet had the chance to dive into .. but what i've heard, it might be helpful in this context ... [15:44:30] --> pje (~pje@n00.bcrtfl01.us.wh.nameservers.net) has joined #peak [15:44:38] 'ello [15:45:01] hello :) [15:45:16] i have read intro to ZConfig and it could be sufficient for my needs [15:45:16] --- ChanServ gives channel operator status to pje [15:45:24] pje: hello [15:45:29] hi :) [15:46:57] like i said, i havent yet played with ZConfig and i don't know if it allows to compose a hierarchy configured components .. [15:47:05] Yes, it does. [15:47:24] aloisius: then this would probably the way to go for .. [15:47:31] There's a sample in 'peak.running'. [15:47:54] The 'EventDriven.xml' schema configures an app component with task components as children. [15:50:07] pje: what i do not understand in the EventDriven.xml .. no actual values are set somewhere .. types, descriptions and names are set, but no values .. where do they come from .. e.g. for stopAfter [15:50:36] That's a schema, not a config file. [15:51:04] It is to a config file, as a model.Element class definition is to an instance. [15:51:30] The end user config file would just say "StopAfter 10" [15:51:37] (for example) [15:52:23] For the subobjects, there are sections like, \n RunEvery 30m\n ... [15:53:11] ok .. and that would be then loaded with some peak zconfig.schema:myAppSchema MyConfig to run it = [15:53:13] ? [15:53:27] Yes. [15:53:34] ok .. sounds fairly easy :) [15:53:42] And the config file can have a #! line at the beginning to invoke it, even. [15:54:00] Here's a sample: [15:54:03] RunEvery 5 [15:54:03] Priority 1 [15:54:03] CheckURL file:/cygwin/home/pje/PEAK/foobar [15:54:03] RestartWith shellcmd:touch foobar [15:54:03] [15:54:03] [15:54:05] Priority 1 [15:54:07] Directory . [15:54:09] RunEvery 2 [15:54:11] DeleteIfOlderThan 2 [15:54:13] MatchFiles foobar [15:54:15] [15:54:27] Oops. Something got deleted off the front... [15:54:36] #!./invoke wpython scripts/peak EventDriven [15:54:36] StopAfter 10 [15:54:36] [15:54:36] RunEvery 5 [15:54:36] Priority 1 [15:54:36] CheckURL file:/cygwin/home/pje/PEAK/foobar [15:54:38] RestartWith shellcmd:touch foobar [15:54:40] [15:54:42] [15:54:44] Priority 1 [15:54:46] Directory . [15:54:48] RunEvery 2 [15:54:50] DeleteIfOlderThan 2 [15:54:52] MatchFiles foobar [15:54:54] [15:54:56] There. [15:55:01] ok [15:55:21] That's a sample config file using the 'EventDriven' schema. It creates a file 'foobar' and deletes it, over and over. :) [15:56:11] yeah :) [15:56:23] this can be done easier in python *eg* [15:56:33] though maybe not that flexible :) [15:56:33] You're right, it can. [15:56:47] ZConfig is for *end user* configuration. [15:57:00] In other words, for people who don't know Python. [16:00:52] If your configuration is being done by someone who knows Python, an .ini or Python file is a better choice. [16:00:52] i've done all configs till yet with ini-style files .. but will try the ZConfig-way soon with my acmgr [16:00:52] ZConfig is also strongly typed, so you have to be relatively sure of what you want. [16:00:52] Actually, I mean statically typed. (Strongly too, but so is Python.) [16:00:52] hm, gotta go.. [16:00:52] cu later.. [16:00:52] adios [16:00:52] cu [16:00:52] <-- DrTopf has quit ("Client Exiting") [16:01:34] pje: we exchanged some mails about a messaging subsystem for peak a few months ago .. right now i'm at a point with my development where i will need a solution (point-to-point messaging implemented with jabber) to go ahead. [16:02:03] pje: will you have some time to discuss the issues about this topic during the next weeks ? [16:02:27] Probably. [16:03:03] But I don't really expect to put messaging in PEAK proper for some time. [16:03:37] It's more of a priority to implement "shared spaces" for collaboration. [16:04:16] that does not really matter for me now .. i'll put it into apps/nll/src/messaging, but i would like to share ideas with you and probably serve a playground for evolving such a package [16:04:55] Have you looked at JavaSpaces, or Linda, or any similar collaboration systems? [16:05:39] yup .. i actually implemented a spacesdm for testing (but not distributed yet) and very experimental .. [16:06:15] it works with ***duck*** Mysql and my SQLEntityDM/QueryDM ... [16:06:55] the most important part is as you stated the cache invalidation - right ?? [16:07:38] Locking, really. [16:07:58] err .. right .. [16:07:59] To do a transactional space, you need to be able to tell the other participants that you have "taken" [16:08:01] an item. [16:08:13] So that they don't try to "take" it also. [16:08:53] but that functionality is hidden with the SpacesDM from the Application. [16:09:09] But, the locking needs to expire if the transaction fails or a participant disconnects. [16:09:45] I wrote an experimental lock manager for the Python Spread module that did this at one time. [16:10:20] to get it right: you have different processes running, all suited with some SpacesDM that access one backend (e.g. SQL) [16:11:10] then you have some network enabled locking mechanism that prevents from taking one item more than once, transactionally. [16:11:13] right ?? [16:11:36] Right. [16:11:51] I was using Spread as the communications mechanism for locking. [16:12:21] In principle, though, if the data was transient, you could use only Spread, and have each DM just keep a cache. [16:12:35] As long as there is at least one participant, the data persists. :) [16:14:16] you mean, let spread send one item's serialized form over the wire to all participants and a take-notice as well ?? [16:14:43] Yep. [16:14:53] Radek... I'm not following your e-mail... [16:14:59] What do you need to cache? [16:15:16] but then clients that "come late" could not know about all items in the space .. right ? [16:15:46] pje: i need caching database records on DMs [16:16:02] So just use an *instance* attribute on the DM. [16:16:45] Ulrich: the existing members have to send the data to them. [16:17:01] For my lock manager, I had each member send the list of items it had locked. [16:17:06] but there can be more instances of one application and i want share cache between them [16:17:28] If you want to share cache between them, then you need to create a cache that's thread-safe. [16:17:58] pje: i know, i could not be problem. I can use spinlocks for speed. [16:18:15] s/i could/it could/ [16:18:17] Okay, but still don't make it a class attribute. [16:18:54] Just create an "application context" component that you put into each instance in the app pool. [16:19:37] You can then put whatever shared items you want in it, and then offer them downwards from each app instance. [16:19:49] pje: how do you envision to "serialize" your components ?? [16:20:38] Don't know yet, Ulrich. Really, I expect to do what you're doing. That is, put the data in an SQL database and just use the comms channel for locking. [16:20:56] Although there are other ways to do locking, certainly. [16:21:19] Currently, some of our apps do outbound mail via queue directories... and that is effectively a shared space. [16:21:34] pje: that was my first solution :-) But problem is that one component can be binded to /app_context/my_private_cache but this component can appear more than once in one application. Now both instances use same cache and it is undesirable. [16:22:48] You've lost me there, Radek. You have two instances of FooDM, but each wants its own cache, right? [16:23:00] pje: yes [16:23:04] But that's in *one* application instance? [16:23:15] Not multiple instances of the same app? [16:23:15] pje: yes [16:23:33] pje: ok .. i'm definitely interested in spaces as well .. but i'ld like to get some notification (i need some kind of agent-framework) when new items matching a filter arrive ... [16:23:33] So why don't you offer a different binding from their parent components? [16:23:38] pje: and it will be frequent scenario [16:24:53] If you need to provide a per-location cache service, you could always define a property rule that looks at the 'getComponentPath()' of the 'forObj'. [16:25:04] pje: this context must be unique to application and "path" [16:25:59] Ulrich, depending on your licensing needs, you might want to look at Elvin. [16:26:51] afaik Elvin's license does not really fit our needs .. we're looking at jabber for point-to-point messaging [16:27:02] Radek.. rules for computing a property value have access to the object that is asking for the property. [16:27:12] And they can get its path, using binding.getComponentPath() [16:27:33] pje: so i can have global dict with app:/path as keys and contexts as values and use them in my own PropertyRule. [16:27:41] Exactly. [16:28:10] Actually, it works for any configuration key, not just property names. But they're easiest to configure in an .ini right now. [16:30:03] Why jabber instead of say, Spread or BEEP? [16:30:11] pje: i am not sure if this solution have not some drawbacks and why I reject before [16:30:51] pje: spread makes some assumptions, that a "domain" needs to be configured on every server and is mostly static, right ? [16:31:02] True. [16:31:31] Radek: I'm not sure what drawbacks it has that aren't inherent to using cross-thread caches. [16:31:33] i've looked at the beep-specs a couple of times but i haven't yet used it. [16:32:04] do you have any experiences with BEEP (afaik the Chandler Project will/wants to use it) [16:32:04] Once the components hook up to the cache, they are pointing to it, and the binding machinery is no longer involved. [16:32:16] No experience yet. [16:32:41] I was asking, not to recommend a protocol, but more to understand your requirements better. [16:33:19] I find that understanding the choices *not* made is often more useful to understand someone's requirements than the choices they *did* make. :) [16:35:06] ok .. we're still targetting on a networked Account/Host-Managing Application with 1-n wxPython Gui Clients connected to one(or more) Controlling Processes (probably using some workflow-cababilities and a busines-object-model). [16:35:38] then there are agents on different hosts/os that get their jobs via Spaces. [16:36:08] all communication should be done asynchronous e.g. though subscribing to a Space, take items, put items [16:36:08] Who's the user? [16:36:49] varies from Teacher at school who adds a new user to administrator how configures host-settings of a server/client-machine [16:37:31] So why wxPython GUI as opposed to say a web app? [16:38:05] i really tried to build such a system with Zope2 about one year ago .. [16:38:29] but .. we we not able to find a good UserInterface that was fast and easy to use [16:38:44] e.g. there are between 1000-2000 accounts to manage .. [16:39:15] only if you do a lot of Javascript-stuff you get a UI that really works [16:39:36] Ah. [16:40:13] have you recently lookt at our nll libraries ?? [16:40:31] Well, your agents sound like something that could be done with AdaptiveTasks. [16:40:36] No, I haven't. [16:41:05] We use the older version of AdaptiveTasks (MetaDaemon) for a variety of queue-driven and replication tasks. [16:41:18] nll is a collection of all sorts of modules (database/net/gui/messaging) that i build for peak apps [16:41:46] URL? [16:41:57] its all in cvs.net-labs.de apps/nll [16:42:01] close to acmgr [16:43:04] acmgr/bin/acmgr_gui.py is an experimental gui-client-dummy-app that shows the current state and has a treeview and a listcontrol filled with some items .. [16:43:36] Ah. [16:43:50] the nll/database package has our LDAPEntityDM and SQLEntityDM implementations [16:44:44] and Sapdb-Schema from which you can generate a complete Element-Model (configured Element-classes, EntityDMs and QueryDMs) [16:45:42] example output: http://cvs.net-labs.de/viewcvs.cgi/apps/nll/bin/HELLER.py?rev=1.1&content-type=text/vnd.viewcvs-markup [16:47:28] Impressive. Although I don't understand it yet. :) [16:47:40] what .. the gui stuff or the schema ?? [16:49:17] The generated model I just glanced at. [16:49:49] It's pretty immense. :) [16:49:58] right .. [16:50:10] in the same directory there is TST.py [16:50:40] it still has lots of tables .. but much less [16:50:50] i create Element classes for each table first [16:51:02] then i create configured Datamanagers with Configured QueryManagers [16:51:22] in the end i put them together to a component that represents the whole database at once [16:52:27] Ah. [16:52:29] SQLEntityDM is a configurable Datamanager that creates SQL-Query in a metaclass (will change this to a late-binding) from field/index/table-spec [16:53:31] if the schema is detailed enough you can even create the element-associations automatically (foreing-key definitions needed) [16:54:12] I'm a lot more interested in the reverse direction: peak.model -> SQL [16:54:30] this would be the next step i think [16:54:39] for now i only inspect the database [16:54:50] * pje nods. [16:54:51] but the whole schema stuff is made with datamanagers [16:55:25] so one would need to implement save/new for schema with corresponding SQL-Commands to make this work [16:55:35] then read a Model and create the Database from it [16:56:29] Ah... I think I get you now. [16:56:31] i tried to implement the basic schema stuff as db-independet as possible [16:56:53] so my sapdb-connection has a schema-dm-implementation [16:57:29] so could a postgres or other rdbms .. using the same structure of Tables/Columns/Sequences/etc [16:57:56] Have you looked at CWM? [16:58:16] not really .. would it help there ? [16:58:17] CWM is somewhat to databases as UML is to code. [16:58:25] ahh .. ok [16:58:34] and it's stored in XMI right ? [16:58:46] or could be stored .. [16:58:52] It can be, yes, but that's not my point. [16:59:02] It's more that it has a very robust schema model. [16:59:19] And, that the metamodel can be generated by PEAK. [16:59:30] ok .. i was searching for something like that .. [16:59:34] If you run 'mof2py', it will generate CWM10 and CWM11 metamodels. [16:59:52] I recommend skimming through the CWM specs. [17:00:02] It deals with tables, columns, indexes, views... [17:00:06] i'll have a look at them [17:00:10] Tons and tons of stuff. [17:00:30] And it aligns with MOF, so it's within the realm of possibility that you could use 'mof2py' to generate peak.model code from a schema. [17:00:45] But of course you've already written all this, so it's probably moot now. :) [17:01:52] i think it should not be to hard to refactor my Element-model (which has Tables/Columns/Indexes/Sequences/etc) to CWM .. it would be nice if there were some Design-tools available, that output CWM in XMI .. [17:02:10] Yes. That's really the sticking point. [17:02:35] In theory, PMW (Python Modelling Workbench) should be able to be hacked to edit CWM, since it's a metamodel-driven editor. [17:02:40] And it supports XMI. [17:02:56] But it's not a high enough priority for me at the moment. [17:03:05] what are you using to create XMI (except Notepad) ?? [17:04:11] Heh. [17:04:17] :) [17:04:19] MagicDraw; it's a commercial UML editor. [17:04:42] And of course the XMI files for UML and CWM are from the OMG website. [17:05:01] ok . i bought ObjectDomain a year ago .. but their XMI-Export is still not finished [17:05:38] for this i have a question: [17:05:55] how do you intend to use XMI-Files (e.g. containing UML-Data) ?? [17:06:21] err .. i think we talked about this via email lately .. [17:06:24] For UML, code generation. [17:06:30] ok [17:06:40] But more usually, for application data exchange. [17:07:01] would this be a "way" to serialize components as well ?? [17:07:01] In principle, any peak.model schema can be mapped to an XMI schema. [17:07:41] Yes. I see it as a way to create data sets for testing/prototyping that can be inspected by human eyes. [17:07:50] (I'm not very good at reading pickles!) [17:07:56] :) me neither .. [17:08:16] It's also a way to transport initial setup data to initialize the contents of an application's database. [17:08:24] yes [17:08:26] But I'm not doing any of this just yet. [17:10:19] ok [17:12:39] you have once stated, that you have an implementation for a filesystem-based queue working in your projects .. is the source-code for this package available for my design work on a messaging system ?? [17:13:44] i really would like to come up with some similar flexible thing like peak.database/naming for messaging where you can delay the desicion which transport you wanna use [17:19:37] No, it's not available. [17:19:41] It's not open source. [17:19:48] ok [17:20:00] But something similar will get written for PEAK at some point. [17:20:23] I must prepare for my weekend journey (pack tent :-) but I stay connected to log all interesting debates. Thanks for advices. Bye! [17:20:43] cu .. have fun :) [17:20:48] * aloisius is away: mangiare le scarpe [17:21:21] Bye [17:21:46] pje: do you think we can work on a draft for the "messaging-backend" interface together during the next weeks ? [17:22:34] i need to get uptodate in office the next week .. but i'ld like to start work on that after that (in july) [17:23:13] I don't know. [17:23:13] Considering that to me, that equals "DM plus locking" [17:23:13] That is, a DM with "put" and "take" methods. [17:23:45] Personally, I think that is all that is needed. [17:23:50] hmm .. you say basically a space ;-) [17:24:08] Yes. [17:24:19] And it may be that you don't even need 'put', just newItem(). [17:24:29] and then commit [17:24:43] Right. And 'take' might be a query + lock. [17:25:05] So, really, you could have a 'take' method that you passed an item to, after querying to find it. [17:25:18] And 'take' would basically lock it, and flag to delete it from the space on commit. [17:25:33] Voila. You now have "messenging". :) [17:25:39] so as far as i understood Spaces, they "hold" items that have attributes (in java any Serializable object can be stored in a space afaik) [17:25:46] Pretty much, yes. [17:26:08] But I prefer a somewhat more strongly-typed space. [17:26:13] ok [17:26:21] But DM's can be either way. [17:26:24] otherwise it would be really tricky to implement [17:26:42] *shrug*. You could always store pickles. [17:26:51] right .. [17:27:00] The tricky part is actually the locking. [17:27:01] but how do you query then :. [17:27:18] Pickles aren't very good for querying, obviously. [17:27:23] yes ;( [17:28:01] So add columns for things that need to be queriable. Or have a mapping from model classes to table names. [17:28:06] Do whatever you want. :) [17:28:22] The point is that DM's form the basis for the abstraction, whatever your implementation is. [17:29:00] ok .. i'll go that way a bit further and report my experiences [17:29:05] To me, this is more a "pattern" than a library. Until I've implemented the pattern more than once with PEAK, I won't know how (or whether) to turn it into a library. [17:29:46] yeah [17:58:08] I need to go get some lunch; and then I should probably get back to work on PEAK. :) [17:58:32] See you later. [17:58:36] cu .. i'll be offline to prepare for my 30. birthday 2morrow [17:58:44] Happy birthday. [17:58:46] Ciao. [17:58:48] cu on monday :) [17:58:52] <-- pje has quit () [17:58:59] <-- jack-e has quit ("gone") [21:03:59] --> _Maniac (~User@h24-77-230-121.wp.shawcable.net) has joined #peak [22:27:48] --> azaad (~jxj@sv-fw.looksmart.com) has joined #peak