[10:49:42] --> jack-e has joined #peak [10:49:52] re [10:59:50] jolby: are you online ?? [12:54:05] --> Maniac has joined #peak [12:54:20] --> queuetue has joined #peak [12:55:10] Hi there. Could someone give me the 100-words-or-less explanation of why I should investigate PEAK? I don't even really understand what it *is*... [12:55:57] queuetue: pje (the main peak develoepr) is not in at the moment [12:56:12] however, jack-e could probably do that if he's around [12:56:37] my 3 word explanation: 'cause it's cool' isn't very technical [12:57:09] I guess I'm tryign to figure out what problems PEAK is trying to solve... [12:57:27] you've looked at the webpage? [12:57:44] queuetue: reusability, configurability, performance of mid-to-large python-projects [12:58:03] jack-e: How, exactly? [12:58:13] * Maniac thanks jacke [12:58:15] Maniac: Yes, I have. [12:59:08] there is an abstraction for object-persistence (Datamanagers can store stuff in several backends e.g. SQL/XML/whateveryouimplement - the application does not need to be changed when storage-backend changes) [12:59:43] there is a late-binding system that saves you much thinking and typing e.g. when you want to know you parent [12:59:59] or use utilities that you exposed via an interface [13:00:17] there is a naming system (based on the ideas of JNDI) [13:00:23] Ok, so it's another object broker? [13:00:44] It promises to stop having to invent architectural wheels... What sorts of architectural whells are we talking about? Just persistence-related ones? [13:00:58] structural mostly [13:01:25] ... Can you give a concrete example? [13:01:28] you compose your system from components rather than subclassing and def __init__(self, parent, bla, blub) stuff [13:01:43] do you know zope3 [13:01:48] jack-e: Why is that a benefit? [13:02:10] jack-e: Not much - I find zope fairly unpleasant, from a python perspective. [13:03:21] regarding zope2 .. you're probably right .. zope3 introduces the Component Architecture [13:03:42] it will be much more friendly to the python-programmer than zope2 i think [13:04:10] so .. what does peak.binding offer you .. hmm .. have you read the first pages of the tutorial ?? [13:04:33] The PDF one? [13:04:38] yup [13:05:12] are you familiar with model-driven-architecture ?? [13:05:13] I've read it... I see a lot of promises... I still don't see anythign technical or concrete... [13:05:25] We're still talking about ways to help developers that don't want to manage persistance, right? [13:05:41] peak is not only about persistence .. [13:05:52] it's a way of coding your whole app [13:06:09] How does a "component architecture" give me anything more that "import myComponent"? [13:06:59] "component architecture" is from zope3 .. it has a similar approach as components in peak, but they do not really have any implementation in common [13:07:17] a component has context (you build a tree of components) [13:07:48] a component may implement interfaces, there is a system for registering and using Adapters [13:08:04] Yes, but why? [13:08:16] I read the whitepaper - I'm trying to figure out why this is necessary. [13:08:35] s/whitepaper/tutorial/ [13:08:53] because it reduces overall complexity of code i think [13:09:06] It introduces a massive amount of complexity and constraint on the developer - what does he get in return? [13:09:17] Reduces complexity? [13:10:05] it adds some initial requirements of knowledge to the developer, but - like in so many cases - it helps you to get your job done easier and faster [13:10:32] Could you perhaps give a single concrete example of where it might? [13:10:57] i've created a user-management application (unix cmd-line) with peak within one week [13:11:17] And you don't think you could have in straight python? [13:11:29] (cause I know I could have... :) ) [13:11:31] it was able to create/update accounts in LDAP (pam/nss_ldap), winNT4 Pdc (via remote), IMAP Mailboxes, homedirs, etc [13:11:51] Right - what did PEAK actually *do* to make that possible? [13:12:20] it gives me a framework for building applications [13:12:37] We keep coming back to the same points - what does that *mean*? [13:12:42] more than a if __name__='__main__': run() [13:13:03] i think i cannot describe this in a way you expect .. i'm not a native english speakr [13:13:44] I'm starting to get the impression that this is a tool to force junior developers to write readable code... [13:14:27] give me an example of a large python project where you find the code readable ?? [13:16:03] We recently wrote a charter airline management and booking engine. It was pretty readable to my guys, and topped somewhere near a million SLOC, not including xslt skins... I'm trying to figure out what PEAK would have added to that effort, and it look slike just a lot of typing... [13:17:40] you just used pure-python or any framework ? [13:18:23] Pure python - no framework other than python itself. [13:18:41] WHat would we have needed one for? [13:19:12] usually a framework saves you from doing things others have already implemented .. right ? [13:19:39] Sure. What has PEAK implemented? [13:20:01] WHat is the technical value of using an inteface, for example? [13:20:12] (other than it allows me to use PEAK.) [13:20:42] be right back [13:21:48] Maniac: WHat are *you* using PEAK for? WHat benefit are you getting from it? [13:23:34] back [13:24:20] interfaces can be used a classification for objects (you don't tie your code to a ClassXYZ but to an interface and can exchange the class behind as you like) [13:24:40] How does that differ from inheritance? [13:24:59] you define an ISQLConnection but you don't care if its an OracleSQL-Conneciton or a MySQLConnection [13:25:19] Right. How does that differ from inheritance? [13:26:39] in inheritance you need to subclass from something and therefore you're bound to that baseclass - using interfaces does not require this .. just declare that you class/object (maybe taken from another project) implements that interface (and .. [13:26:49] therefore fullfills this contract) [13:28:02] you can everything do without peak, interfaces, etc .. but with peak it will be more reusable, and probably in longterm easier to grow with [13:28:37] peak is a very young project with less developers yet .. all this has not been proved yet .. so there are no promises :) [13:28:41] Is there a concrete example of how interfaces make it cheaper to implement code, or this just an opinion? It seems to me that if you design a clean "interface" then there' sno difference between using interface sor inheritance... [13:29:23] Except one is fully supported by Guido and the crew in #Python and one isn't. [13:30:00] what do you mean by that ?? [13:30:09] Which part? [13:30:27] your last sentence [13:30:36] who is not supported ? [13:30:45] people using interfaces C? [13:31:28] peak has some ideas that are very close to AOP [13:31:43] and programming in interfaces is not python specific [13:31:50] I mean that if I have a problem using PEAK's interfaces, who do I turn to for help? The PEAK guys (or single guy, I think.) If I have a problem without them, I can rely on the entire python community. I'm willin gto go it on my own, if there's a tangible benefit, but what is it in this case? [13:32:04] as far as how i use it ? I've starteed playing with peak.naming [13:32:16] queuetue: that's a valid point. As jack-e said it's a young project [13:32:27] If you design a clean api, then subclasses don't care about how the super implementations are implemented. It seems you're adding syntax with no real benefit, but I want you to prove me wrong... [13:33:00] the point is (from my perspective) is I don't have time nor skills nor intelligence to build my own framework everytime i want to build an application. I *always* look for someone who has built that framework already [13:34:14] whether that framework is twisted for networking or peak for enterprise tools (simular to but different from J2EE) [13:34:23] i nice docu/info about using interfaces and adaption: http://peak.telecommunity.com/protocol_ref/module-protocols.html [13:35:17] So, it's a junior developer support tool? If so, then that's all I needed confirmation on. [13:35:54] from my point of view, peak implements a lot of very usefull design-patterns [13:36:07] jack-e: COuld you tell em what one of them is? [13:36:14] s/em/me/ [13:36:19] http://www.eby-sarna.com/pipermail/transwarp/2003-June/000505.html [13:37:50] adaptor, command for example [13:38:33] Maniac: That looks like more persistance stuff... For all the discussion about protocols and objects and interfaces, it looks PEAK is just a way to avoid learning SQL... [13:38:46] jack-e: And how do you use adaptor with peak? [13:38:56] queuetue: that answer was biased towards a certain question (which was based on persistence) [13:39:03] (not that I love SQL, but it;s darned fast...) [13:40:21] Ok, obviously I'm too dumb to get this. [13:40:33] Thanks for trying. [13:40:46] .. if you're interested, put your questions onto the mailinglist .. [13:41:05] queuetue: yet again i say. you need (if your seriously interested) to talk to pje (Phillip J. Eby) the lead developer for a better explanation :) [13:41:09] phillip eby will be able to answer your questions, clarify these issues [13:41:55] Well, my interest was limited to "is anyone using it" and "what does it do for them?" It sounds like "not really" and "allows them to completely change thier coding style." So, I think I'm out of interest. :) [13:42:51] .. from your point of view an understandable argument .. come back in a few month and look again [13:43:00] s/month/months [13:43:05] If I'm looking at a 10x increase in the time it takes to develop a project, I better be lookign at at least a 10x increase in the value I get for my time. I can't see that happening with this technology. I'll come back in a while. [13:43:27] <-- queuetue has quit #peak [13:43:32] puh [13:44:00] we should forward this talk to the mailinglist and ask for comments .. should we ?? [13:44:22] pretty judgemental [13:44:39] judgemental ? [13:44:55] all that extra typing (defining interfaces etc.) helps in the long run for large projects i would think [13:45:04] so do i [13:45:11] 10x increase 10x payoff blah blah blah [13:45:23] btw: i have new ideas today :) [13:46:09] i cannot imagine that he has managed to build a python-monster with millions of LOCs without running into trouble in the areas of configuration, refactoring, binding etc [13:46:25] but anyway .. he's off :)= [13:46:27] the thing with peak is it's not ONE thing [13:46:34] peak.naming peak.config [13:46:37] etc. [13:46:41] each has benefits [13:46:47] yep .. so what are your ideas ?? [13:47:02] using peak.config and not having to write your own config parsers is benefit enough :) [13:47:07] as for idears [13:47:14] yoo [13:47:34] i was thinking that each 'object' should manage it's own links (rehashing yesterday) [13:47:52] 'object' == 'model.Element' ?? [13:48:37] have you read the posting from pje today ? [13:48:53] mmm, (my terminology is not good) i guess so [13:48:59] he wrote someting about finding DM's for Classes :)) [13:49:13] the 'functionality gap?' [13:49:18] yup [13:49:25] yeah i read that last night [13:49:57] that would solve the "mapping" problem between the element-class and the responsible datamanager [13:50:24] ok .. back to each object should have a linkedElements feature [13:51:03] ( i suck at explaing this stuff) [13:51:05] i was thinking of that as well, i want to try to build something like a zope-folder with peak-datamangers (it is basically the same as you want with your linkedObjects) [13:51:50] you have a common model.Element class for all your Elements, this class defines a feature linkedElements(model.Collection) [13:53:16] this feature is filled like we discussed yesterday .. when you have a table ltype, lid, rtype, rid you can link every object in some others linkedElements feature by just saying someEl.addLinkedElement(otherEl) [13:53:20] * Maniac grabs a coffee [13:53:31] mee too .. and i need to finish some project-calculation .. [13:54:40] the basics of the idea is this: [13:55:07] each object (in this case i'm thinking of a 'concern' or 'contact' as an object) maintains it's own links [13:56:25] if you access a 'concern' it has a table which has CONCERN_ID,CLASS_TYPE,CLASS_ID [13:57:02] ok .. so you have one "link-table" per "object-table" [13:57:21] now a 'contact' wants to find out which CONCERNS links to it, it goes out and queries the 'CONCERN' and the CONCERN returns the CONCERN_ID and some sort of description [13:57:33] jack-e: yes [13:57:40] what's the benefit of this solution ? [13:57:54] (in opposit to RTYPE, RID, LTYPE, LID) [13:57:55] hopefully better modularity [13:58:04] i doubt [13:58:14] you need 2 configurations for each object then [13:58:23] 1. classtype -> responsible datamanager [13:58:28] i'm thinking in terms of a CONCERN talks to some backend and says 'find me everything that links to me' [13:58:35] this can be local, remote etc [13:58:48] 2. linkedElements -> responsible datamanager [13:59:26] in this case, you need to query every "link-table" for information about "Type = and ID = [13:59:38] yes [14:00:04] in the case where you have only one link-table, you just query this one [14:00:34] i'ld prefer one link-table .. you can create a generic implementation with a little configuration [14:01:54] you just add to some element a LinkedElementsFeature and and take care in the DM's methods that the value is saved/loaded [14:02:15] (by asking the one linked-elements-dm in your application) [14:03:02] the linked-elements-dm is not more that a facade to the real entity-dm's with the ability to store this kind of relation [14:03:33] i admit this was 'inspired' by dashboard in which you just 'plugin' different 'backends' and they all respond to a basic query [14:10:21] it's basically that way, you edit one mapping entry for your new class and you're ready :) [14:10:38] oh started to make some 'models' i guess there's no referencedType = DateTime [14:10:57] not yet .. copy it from bulletins [14:11:02] yeah i saw that [14:11:55] i've build some in nll.database.schema.sqltypes i think (sapdb-specific) [14:12:06] s/build/built [14:57:37] oh yes, we should forward this to the mailing list. My guess is that pje, however, at least until things mature more was not interested so much in 'selling' PEAK. [14:59:34] ;-) .. [15:47:20] {global notice} Good afternoon everyone, apparrently one of our main rotation servers has crashed, and the problem is being looked at. Any further messages will be given in wallops. [15:49:31] * joel (jolby) finishes scrolling... [15:49:46] Hi jack-e -- I'm here now. [15:49:49] hi jolby [15:50:29] i'm here as well, but somehow busy, so i cannot really get into a deep discussion about twisted and peak right now [15:50:45] I should be around most of the day if you want to talk about mixing peak + twisted. I was thinking a good place to start would just to port the echo server. [15:51:09] echo-server with twisted.spread ? [15:51:27] or simple echo-server? [15:51:46] No-- just the plain simple echo-server (Just to play around w/ the reactor) [15:51:56] yes .. sounds goot [15:51:58] good [15:52:45] I'll leave this window open. and check in every once in a while. [15:53:04] i haven't yet done much with twisted [15:53:35] I've been doing a bit the past few weeks. I like it, but I'm not sure if I'm doing things the "twisted way" yet. [15:54:02] have you looked at the Eventdriven class [15:54:03] ? [15:55:02] I just saw it in the cvs changelog a while back. Hmmm... maybe pje has *already* done some simple porting to get that class to work. [15:55:23] I should ask pje on the maillist. Maybe this has already been done before... [15:56:04] i've done some examples with Eventdriven and Tasks [15:56:55] How did you "wire" up your event driven objects to the event generators? [15:59:21] there is a BasicReactor configured in peak.ini (you should probably replace it with a TwistedReactor) [15:59:51] the peak system in peak.running defines a TaskQueue in scheduler.py and an AdaptiveTask [16:00:59] Hmmm.. I'll check that out today. [16:01:11] you could probably take AdaptiveTask and create a specific task-class for e.g. getting some request and handle it (e.g. urlchecker example in daemons.py [16:01:36] err. AdaptiveTask and TaskQueue are in daemons.py in peak.running [16:03:37] can peak.config *write* out config files? [16:03:37] ok. cool. I wonder how twisted + peak will work together in regards to threading. twisted is single-threaded, and needs to be in charge of the main thread for it to work correctly I think. [16:04:04] you would have an EventDriven component and put some child-objects (from type AdaptiveTask e.g. URLChecker, FileCleaner), when a child of Type AdaptiveTask is composed, it looksup the taskqueue and registers itself [16:04:19] the taskqueue plays nicely with the TwistedReactor [16:04:50] ok, cool. [16:05:10] the only thing that i haven't yet found out is, how to make e.g. a protocol a peak component [16:06:00] maniac: i don't know [16:06:04] :) [16:06:42] jack-e: maybe just inherit from twisted.Protocol, and Component? [16:08:25] probably .. it's just that you usually create a component with a certain signature, and probably a protocol as well, the protocol initialization should be done when the component is bound in the context i think .. but i'ld have to look at this mor [16:09:06] i don't know exactly when i'll be able to work on this with you .. i'll try to get some time for it 2morrow evening (CET) [16:11:26] ok, cool. No problem jack-e. I think I'll pop an email to the PEAK list to see if we can enlist any brain cycles of pje as well... :-) [16:12:55] btw-- I just read on the zope3 maillist that Guido is leaving the company that makes zope for another company in california. [16:13:34] I think their zope3 architecture is pretty well along tho-- so it shouldn't have to big of an impact. [16:15:32] i've read guidos mail as well .. he was not too active in zope3 (he mainly built the fssync-part) [16:16:10] but it's bad that he'll leave the zope3-team .. i worked with him in rotterdam a bit .. it was really nice [16:21:21] I bet that would be fun. I hope to attend a python sprint sometime. They look like they would be a blast. I'm pretty new to python (java/j2ee refugee), and I *love* the great community spirit that python has. Makes programming fun again... [16:22:49] yeah .. i've been on 2 zope3 and one plone sprint .. and it' so much fun .. [16:23:06] when the peak community grows a bit, perhaps we can do some sprints on that as well [16:24:56] That would be great! I imagine attending one of those sprints, you would learn more in a day than you would in a couple weeks poking around by yourself. [16:26:57] that's true for zope3 as well for example .. it's so much easier to explain and ask if you sit together and work on something [16:28:24] Yeah, face-2-face is a very dense informational broadcast medium. [16:28:38] definitly :) [16:33:23] what kind of company is guido going to? [16:35:17] http://mail.zope.org/pipermail/zope3-dev/2003-July/007598.html [17:24:42] --> _jpl_ has joined #peak [17:31:33] hello john :) [17:31:51] <_jpl_> Hi Ulrich, how are you? [17:32:23] fine .. (except a fairly busy day) .. and you ?? [17:33:22] <_jpl_> Likewise... finishing up a complete refactoring of my project using PEAK [17:33:33] <_jpl_> At least what I can understand of it so far. :) [17:33:49] sounds good .. do you like what you've done so far ?? [17:34:30] <_jpl_> Yes, I'm really happy with it. It's far far better than what I had originally. [17:34:53] we had one guy in here today how poked on us why we're using peak .. we couldn't really convince him to look at peak more into deep :) [17:34:56] ql [17:35:07] _jpl_: may i ask, what is your project [17:36:23] <_jpl_> It's an agent/service system for Unix/Linux/Windows configuration management and monitoring [17:37:55] i'm really looking forward to talk with you about your agent-stuff - we had some interesting talks about message-queue implementation here yesterday [17:39:38] <_jpl_> That's great, any conclusions reached? How goes the JMS-to-Python translation? [17:41:06] lateef jackson (nick: teef) sent me a prototype of his messagequeue implementation - i need to look at it more closely [17:41:47] jolby is working on a peak/twisted based JabberServer (which is basically very similar to a message-queue) [17:42:02] <_jpl_> Is it based on any particular communication or remote object mechanism? [17:42:24] teef's work implements its own network protocol [17:42:52] an is heavily inspired by java .. so there is much to refactor . but he has good ideas i think [17:43:37] <_jpl_> Hrm [17:43:48] i think we need to work on a proposal for peak.messaging .. i'm about to make a prototype for a Queue that uses the berkley-db's Queue-features (should be fairly straight forward to implement) [17:44:05] i still want to divide the Transport layer from the Queue implementation [17:44:27] hrm ?? [17:45:05] <_jpl_> Seems like what we need is a layer which provides generic remote object functionality (into which a developer could plug in any RO framework they like), on top of which things like messaging frameworks would be built. [17:45:24] yes .. that's my thinking [17:46:01] i've had some talks with a guy from the munich technial university last weekend [17:46:43] <_jpl_> Is he doing something similar? [17:46:49] they use corba to build an augmented-reality-framework .. he had good ideas for communication-layer abstraction as well .. i'll meet him again next week i think [17:47:04] (they use java) [17:48:10] i'll get the details next week and then i can say if it's interesting and worth to look at [17:48:41] if we "only" want to have RPC .. that would not be too hard to unify [17:48:46] <_jpl_> I've become quite interested in CORBA of late, especially after seeing how easy it is to use with omniORB and Python. It seems like we could wrap some of the most common operations using peak.naming and make it even simpler to deal with. [17:49:17] i've worked with omniORBpy for a while .. it's still much more typing that e.g. using Pyro [17:49:59] i've had some thoughts on connecting the Type-Framework from peak to the Corba-Types and using IDL-Interfaces with Adaption .. [17:50:08] <_jpl_> Yes, but that's what I think we could wrap with PEAK, making it as simple as instantiating through any other naming mechanism [17:50:26] <_jpl_> I've been wanting to look into that as well [17:50:37] ql .. so we are 2 now :) [17:51:10] i still want to have a python-only variant (probably with pyro) to deploy simple apps without any large dependencies [17:51:29] but for *real* communication .. corba is a fairly good choice i think [17:51:33] <_jpl_> I was asking ejb the other day about how to setup 'corbaloc:' and 'corbaname:' schemes... seems like it should be pretty straightforward. [17:52:24] yes .. we could probably do that in less than one day .. but we need some ORB available and setup in our process before we can get the naming-context [17:52:52] pje .. ejb could be easily mixed up with enterprise java beans :) [17:53:30] <_jpl_> Yes, with some sort of neutral middle layer it should be easy enough to support PyRO for Python-only uses, and CORBA for cross platform [17:53:38] <_jpl_> Oops, yeah, pjb :) [17:53:47] that's what i want to come up with .. [17:54:21] <_jpl_> omniORB seems like the obvious choice for a reference platform [17:54:36] for that reason i started with defining an interface in NLL (IRemoteObject) that i can use for urls and adapt to that interface to get a real instance of the RemoteObject to use [17:54:45] omniORB's python-bindings are excelent [17:55:14] so i have an url e.g. "pyro.client:/somecontent" and a "corbaloc:/someloc" [17:55:37] <_jpl_> Dealing with asynchronous communication might get a little tricky. [17:55:51] if i parse these urls i get a PyroClientURL and a CorbaLocURL that can be adapted to IRemoteObject [17:56:30] yes .. we will need to think about these issues (e.g. integration in the Eventloop, multithreading) [17:57:01] <_jpl_> What does your IRemoteObject interface look like? [17:57:20] it's only a marker interface, that i can define adapters for it [17:57:53] i think it now has a call method that takes a methodname and arguments .. but that's bogus [17:58:12] <_jpl_> By the way, what is the new URL for your CVS web interface? [17:58:44] i updated the dns-data but forgot to update the Serial .. so it was not distributed .. [17:58:57] <_jpl_> Oops [17:59:12] we fixed this today and it schould be ok now (with the usual delay) [17:59:39] the cvs-server (not web - cause of virtual-hosts) is available at 217.89.141.28 [17:59:41] <_jpl_> Found it [18:00:05] is it already updated ?? [18:00:22] <_jpl_> I'm still getting the hang of adaptation, so wanted to look again at where you're using it. [18:00:50] for now i use it where i have to :) .. [18:00:54] <_jpl_> cvs.net-labs.de seems to get me to the right place [18:01:11] i had one very usefull place: [18:01:19] <_jpl_> Ah, so you're using ZConfig now? [18:01:47] <_jpl_> I've been using it outside of PEAK, but not yet within PEAK [18:01:58] i defined adapters for wxPython Event-Classes to convert them into a standart format .. this is a very practical way of doing such a conversion [18:02:33] I'm on my way to use ZConfig .. i understood it finally last week and try to modify acmgr to make use of it - but is not finished yet [18:06:02] hey-- jack-e-- what do you mean by converting wxPython events into standard format? [18:06:35] jolby: i'm working on a component-model for wxpython apps [18:07:02] that means, that you load an xrc-file and get a peak-component-tree that represents tha gui hierarchical [18:07:33] the components can connect events that are sent to some IGuiEventHandler (utility) [18:08:00] to be able to serialize them (e.g. with our upcoming message-queue) i need a known format, [18:08:19] look at cvs.net-labs.de (hopefully your dns-server already has the new ipaddr) [18:09:04] at apps/nll/src/nll/gui/wxgui/model_event.py [18:09:38] there i define converters and adapters for wxCommandEvent and TreeEvents [18:10:38] the idea is to create thin-client-applications that use some rpc (corba/jabber/whatever) to communicate with some application-server [18:10:59] and to ease the creation of wxPython-apps in general [18:11:16] Oh, wow-- that sounds fantastic. I like wxPython, but I don't really like their event mechanism-- to C++sy for my taste. This would be really cool! [18:11:56] i see .. we should make a peak/nll tutorial in this channel :)) [18:12:22] I'd sign up for it! :-) [18:12:34] <_jpl_> Ulrich, I see you're using adaptation extensively in OmnisImport. By adapting a backend object to various protocols in sequence, is the assumption that a backend could support multiple protocols? [18:12:54] you can checkout our stuff with cvs -d:pserver:anonymous@cvs.net-labs.de:/data/cvs login , passwd: anonymous , co apps/nll [18:13:03] jpl: yes absolutly right :) [18:13:24] jack-e: thanks-- I'll check it out. [18:13:36] <_jpl_> I've been thinking for a couple of days on doing something very similar [18:14:01] jolby: most of it is very experimental, try python test.py .. there are some unittests and report if something is broken [18:14:14] I just sent a PEAK+twisted email to the list to hopefully tickle some other PEAK'ers brains... [18:14:25] jpl: it has proven to be very usefull [18:14:32] jolby: yeah :) [18:14:58] <_jpl_> I want developers/users of my project to be able to put together their own "configuration modules", which could provide any of a number of sets of data or behavior (e.g. user data, file specifications, partitions, software packages, etc.) [18:15:50] <_jpl_> So I was thinking of running each configuration module through adapt() to see what it supports, then load the information into the model objects accordingly [18:17:20] jpl: very similar to my approach, i deliver some template-class with a backend, that creates the needed information from some common data (omnis is a database-system for schools) .. if some backend implements an interface it supplies the template [18:17:26] <_jpl_> Is it expected that adapters will sometimes need to use introspection to determine whether an object supports a given protocol? [18:17:28] class that is configurable per backend [18:17:53] jpl: have you read PyProtocols doku ?? [18:18:01] <_jpl_> Where could I find an example of your templates? [18:18:18] <_jpl_> I read through it once quickly, and this morning began re-reading it slowly. :) [18:18:39] i think it's worth to read through completely [18:19:17] <_jpl_> (My approach is also heavily template-driven) [18:19:37] it only needs 2 methods: __conform__(self, protocol) on a component (optional) and __adapt__ on types/interfaces/etc (not optional) as far as i got it [18:21:03] if you look at acmgr/conf/acmgr.conf and omnisimport.py and the backends you see my current approach (not yet finished): [18:21:22] (for templates) [18:22:22] i have a property-namespace e.g. acmgr.omnisimport.schueler.templates where i define my template-strings [18:22:53] then there is a TemplatedStruct in omnisimport that has a create-method [18:24:08] the create-method takes a property-namespace (e.g. PropertyName('acmgr.omnisimport.schueler.templates').of(self), a list of dict-like data-objects and optional keywords [18:24:30] the class of that is a peak.util.Struct.struct [18:25:03] <_jpl_> Have you ever worked with Cheetah? [18:25:09] nope [18:25:13] what's that? [18:25:19] <_jpl_> www.cheetah-template.org [18:25:30] <_jpl_> Object-oriented templates in Python :) [18:25:43] <_jpl_> s/-// [18:26:00] --> taj has joined #peak [18:26:08] <_jpl_> Hi Taj [18:26:31] uhm .. what i meant with template is a template for a certain string-value e.g. how is a user-id made up from the data i have [18:26:49] hey [18:26:55] you think of templates more ... how does a named.conf file look like .. right ?? [18:27:01] hello taj [18:27:22] i can't get to cheetah-template.org :( [18:27:32] <_jpl_> s/-// [18:27:36] ahh :) [18:27:38] <_jpl_> i.e. cheetahtemplate.org [18:27:59] the current context was Object-oriented :) [18:28:01] * Maniac notes #peak has become a very busy channel [18:28:09] <_jpl_> Yes, my main application for templates is for wrapping things like named.conf and the like [18:28:47] so we're speaking of different bits in practice .. they're not so far from each other in threory i think [18:28:55] who is the developer of peak? [18:29:07] the project founder I mean? [18:29:11] Phillip Eby .. he is not online at the moment [18:29:18] <_jpl_> And I really like how Cheetah templates become real objects with inheritance, etc. Now I'm trying to figure out the best way to integrate them into PEAK apps. [18:29:19] pje@eby-sarna.com [18:29:20] cool thanks [18:30:25] jpl: i'm looking at the docs at the moment .. [18:31:20] * taj is wondering how PEAK uses OMG modelling languages [18:31:39] <_jpl_> Take a look at peak.metamodels [18:31:56] taj: the work has begun, there is a possiblity to read XMI Files into an object-model [18:32:25] <_jpl_> I'd love to see an example of how that's done. :) [18:32:27] from that you can generate python-code .. but all that is ***very*** experimental [18:32:49] peak/storage/test/xmi.py [18:32:54] a very basic one ;-) [18:33:34] jpl: cheetah templates look abit like dtml-methods from zope [18:33:40] hmm I've written something very similar too, but my base model is MOF [18:33:54] taj: in python ? [18:34:04] I've got some minor pythonics going on to manage references etc [18:34:08] jack-e: yes 100% python [18:34:18] taj: for what purpose ? [18:34:33] <_jpl_> Something like DTML, but more Pythonic I thihnk [18:34:47] I've written a metamodelling app (model/generate/reverse multiple languages) [18:34:53] <_jpl_> Taj, I'd be interested in seeing that [18:35:07] brb [18:35:38] I havent released it yet but soon enough. I'm just looking around at what else is available as I've been underground for the past year working on this ;) [18:35:43] peak seems cool though, I might be able to contribute [18:37:16] <_jpl_> It would definitely be great to bring together all such projects under one enterprise Python project. [18:37:25] bbs [18:38:20] i think the irc-channel is a good thing (tm) for bringing all these people together .. we all have similar goals .. [18:39:35] jack-e: I just pulled the apps/nll from cvs (lots of stuff!) and tried running python test.py, and got a no module named csv. Is that the csv from object-craft? If so, I'll install it. [18:39:58] jpl: so what i've called templatedStruct would you use as context for a cheetah-template to create a config-file .. right ?? [18:40:08] <_jpl_> Definitely, and there are ways in which people could be contributing to PEAK directly, maybe once pje gets things to a more stable point [18:40:27] jolby: yeah .. that's one of the weak-parts of my test-framework (shamelessly stolen from zope3) [18:40:45] i need to get all the imports done within the test-classes [18:40:50] <_jpl_> I need to go look at your templatedStruct now. :) [18:40:51] jack-e: hehe-- ok, I'll install it. thanks [18:41:10] one of the most important PEAK things now is the example pje is working on to shake out some feature gaps IMO [18:41:20] <_jpl_> csv is also coming built-in for Python 2.3 [18:42:09] jpl: a peak.util.Struct.struct is a datatype that has a defined set of Attributes (defined in the class by a class-attr: __fields__ = ['some', 'field', 'of', 'astruct',] [18:42:49] it behaves like a list if you iterate over it, like a dict if you call __getitem__ and exposes all attributes via getattr and is immutable [18:42:57] (e.g. you cannot add unkown attributes) [18:43:18] so what my TemplatedStruct does when you call the class-method create is: [18:44:10] it tries to get a property-value PREFIX + attrName from the config-file e.g. a login = "fm%04(id)d" [18:45:00] then it takes the data-dicts and if the data-dict has a key (attrName) template % data[key] is set in the templatedStruct [18:45:08] Maniac: I agree completely. Gets stuff out of the abstract realm and into the realm of using the toolkit to fix real problems. [18:45:27] so you can specify for every attribute how it looks like after "conversion" [18:45:32] It looks like jack-e has already done a bunch of stuff w/ PEAK tho'. Its nice to see examples. [18:45:57] jolby: i've already suceeded the migration from Transwarp to peak :) [18:46:26] the NetLabsLibs dir in the cvs-root is build upon the latest Transwarp releases [18:46:37] but depreciated now [18:47:00] jack-e: I missed out of the transwarp phase completely (except for seeing it in google occasionally when searching for python + AOP) [18:47:11] :) [18:47:31] jpl: can you follow .. or do i write nonsense ? [18:48:59] <_jpl_> I think so, but I'm looking at the source now to see the specifics. :) [18:49:57] jpl: yes .. look at omnisimport.py at the bottom of the file there are 2 concrete TemplatedStructs defined SchuelerData and LehrerData [18:49:59] <_jpl_> What is the 'ns' variable in TemplatedStruct.create? [18:50:09] these are used in the omnisimport commands as dataClass [18:50:47] it's the proepery-context that is created like this (inside a method of a component=self) [18:50:49] : [18:51:03] ns = PropertyName('some.path.to').of(self) [18:51:10] jack-e: The tests mostly worked with the exception of 3 failures due to not finding the file: ./src/nll/database/csvdm/tests/schema.xml./src/nll/database/csvdm/tests/schema.xml [18:51:19] Is that to be expected? [18:51:43] whoops-- I copy-pasted that filepath twice. [18:51:53] then you can say ns.get('name') that would retrieve a property with name 'some.path.to.name' [18:52:24] jolby: you saw some windows flickering while running the tests ? [18:52:33] <_jpl_> Ah, that's good to know [18:53:02] Yes-- window came up and kind of froze-- I had to kill it. Was that the wxPython stuff? [18:53:27] jolby: hmm yes it's the wxpython-stuff .. what version do you have installed ?? [18:54:08] they should be closed after the tests as well .. [18:54:50] jack-e: 2.4.0.1 -- maybe too old a version? [18:55:33] nope, i've 2.4.0.1 (debian) installed on my machine [18:55:40] err 2.4.0.8 [18:55:46] but that shouldn't matter [18:55:51] jack-e: I'll run them again. I may have gotten impatient. [18:56:14] what's the exact error with the schema.xml ? [18:56:49] IOError: [Errno 2] No such file or directory: './src/nll/database/csvdm/tests/schema.xml' [18:57:00] does this file exist ?? [18:57:14] <_jpl_> So Ulrich, where does the actual data come from that uses the templates? [18:57:32] jack-e: Ok-- the wxPython tests ran fine now. They poped up, flickered a bit, and then closed themselves. That's what is supposed to happen right? [18:58:30] jpl: i say = SchuelerData.create(myns, [{'id':123, 'lastName': 'blub'},], password='123') [18:58:47] jack-e: Nope-- it doesn't. Just not checked in to cvs I guess. I do that all the time w/ my unittest test data... [18:59:49] <_jpl_> Interesting... so where does the list of dicts come from? [19:00:47] jolby: i'll check this in asap [19:00:54] <_jpl_> Gotta run, be back in a bit [19:01:04] jpl: i load them with my csv-datamanager from a csv-file [19:01:26] (it's not exacly a dict but again a struct that behaves like one) [19:01:41] jack-e: cool. Thanks. [19:02:08] jack-e: Woohoo-- got a pretty nice reply to the twisted Q from pje on the maillist. Looks like enough to run with. [19:02:33] Think I'm going to dig into this for the next couple hours. [19:02:48] schema.xml is checked in [19:03:08] if you want a fairly simple example of how to write an app with nll.gui goto [19:03:20] apps/acmgr/bin/acmgr_gui.py and acmgr_wdr.xrc [19:03:43] if you checkout acmgr and cd acmgr/bin [19:04:12] python acmgr_gui.py -d should bring up a explorer-like window and a pycrust window for exploring all that [19:04:58] gui-subcompontents can be reached e.g. panel['pnlTree'] (take the structure from the xrc-file) [19:05:05] jack-e: Thanks for the tip on the wxpython app, I'll check it out. [19:05:13] * jack-e is reading the peak-list [19:05:50] jack-e: That checkin fixed all the probs in the test except I'm getting a no module named ldap-- which ldap module is it looking for? I'll install it [19:06:29] python-ldap 2.0 (pre something) [19:06:55] you don't need to install it fi you don't use ldap i think [19:07:48] --> pje has joined #peak [19:07:53] Hola. [19:08:02] heya phillip :) [19:08:09] jack-e: ok, think I'll skip that one since it looks like I'd need to install openldap as well. [19:08:20] jolby: right [19:08:47] pje: Hi-- Phillip-- I'm just reading your response to my Twisted+PEAK email to the list. [19:11:11] * pje is catching up on the log [19:11:40] it was a busy day today ;-) [19:11:57] --> jolby_chatzilla has joined #peak [19:12:38] Ahhh-- this is much nicer. I think I'll use the chatzill client for a while. [19:13:27] i need to cleanup my desk abit and get prepared for takeoff to a birthday of a friend .. [19:14:06] jack-e: ok jack-e. it was nice talking to you. Thanks for all the pointers. [19:14:39] i'll be back 2morrow to get ahead .. i'm really interested in a peak-based jabberserver that i can customize ;-) [19:15:08] me too! [19:15:21] :) cu all soon [19:21:19] So, jolby... how did my example work? [19:21:46] * pje has only proven it correct, not actually tested it. ;) [19:25:07] pje: I'm still digesting your email and looking thru PEAK code, but your email makes sense so far. Think I'll have a couple Q's in a little bit tho' [19:29:47] I don't have Twisted installed at the office, or I'd test it myself just out of curiosity. [19:36:42] pje: this channel has been smoking for the last two days :) [19:36:57] I noticed. [19:36:57] great productivity sapper :P [19:37:12] Jolby: I just tested the code I posted... works like a charm. [19:38:00] I didn't do the ZConfig stuff, just the "quick launch" version. [19:38:24] Maniac: that's why I space out my visits a little. [19:41:15] Mmmm-- I just got back from gulping down a sandwich and an iced-coffee-- and pje has already tested the twisted/PEAK sample app. Sweet! [19:41:55] Mmmm... food. [19:42:04] Now you're making me hungry. [19:42:28] I think I'm going to try the ZConfig stuff. Although I am wondering why you would suggest using the Zconfig approach rather than the .ini approach you did in the bulletins example. [19:42:53] They're different things. [19:43:38] Typically, if you have a server, you want to be able to install various components in it. [19:43:42] ZConfig is ideal for that. [19:44:04] Bulletins, on the other hand, might *be* a component in such a server. [19:44:51] In general, .inis are for developers, ZConfig is for users/admins. [19:44:58] Ok, so is ZConfig used to configure the *container* of components, and the .ini for configure of the component parts? [19:45:42] The exception is that sitewide admins may also create the PEAK_CONFIG .ini file, to configure access to enterprise resources and installed components. [19:45:52] Jolby: not necessarily. [19:46:13] But sometimes that will be the case. [19:46:20] It's possible you could have it the other way, too. [19:46:59] The ZConfig format is ideal for creating a component *tree*, but requires a predefined schema. [19:47:19] .ini files are lousy at making component trees, but don't need a schema and can contain executable Python. [19:47:40] I think I'm beginning to see what you're talking about. I've done quite a bit of J2ee/jboss programming (ugh...), and the ZConfig looks a lot like web.xml, ejb.xml etc... [19:48:02] .ini files can also easily set properties that control PEAK itself. [19:48:12] pje: can peak write out zconfig files or .ini files or only read in? [19:48:26] Only read. [19:48:32] ok, thought so [19:49:04] so, if you were going to 'config' which channels you wanted an IRC bot to join you would use Zconfig (as a bot user) [19:49:06] For ZConfig, write capability might make sense, at least in the sense of having a tool to edit the files. [19:49:19] Maniac: yes. [19:50:54] so how much zconfig do i have to digest to use it in peak.... (somewhat rhetorical) [19:51:03] * pje shrugs [19:51:13] Ok, so .ini files are sort of just constrained python scripts, mainly to be used for the config/setup phase of an app. [19:51:13] --> pyniac has joined #peak [19:51:20] %spell rhetorical [19:51:24] Exact match: rhetorical [19:51:37] <-- pyniac has quit #peak [19:51:51] Maniac: To use it in PEAK, just know that you can use 'someComponentClass.fromZConfig' as a datatype in a ZConfig schema. [19:52:24] And that you can use 'zconfig.schema:fileOrpkgfileURL' to reference a ZConfig schema and use it as a command interpreter. [19:53:13] Thus, e.g. "peak zconfig.schema:pkgfile:peak.running/EventDriven.xml somefile" will attempt to read 'somefile' according to the EventDriven schema, and then execute it as a command. [19:53:53] Jolby: not really python scripts... the code is mostly executed on demand, not when read. [19:54:22] Defining a property 'foo.bar.baz = 1/0' will not cause an error unless you getProperty('foo.bar.baz') at some point. :) [19:55:00] pje: ok. Its starting to make sense. [19:55:28] I sort of think of .ini as a really souped-up Java properties file... [19:56:00] The main enhancements being (lazy) calculation, and not having to type 'foo.bar.baz.' over and over again if you have a bunch of foo.bar.baz properties. [19:56:51] pje: I have a couple of practiacal questions for using the PEAK toolkit-- here's the first: What would be the "best practice" way to add particular directories to the sys.path. [19:57:56] My impulse reply is use PYTHONPATH. [19:58:08] Or .pth files on the path. [19:58:20] But I don't know *why* you want to add them... so maybe you can explain that. [20:00:19] Yes-- that's (PYTHONPATH) what I did to get your bulletins example to work, but if I can, I'd like to be able to avoid setting system-wide python-path settings. From my experience w/ java, that led to all sorts of probs when working w/ different versions of frameworks and libraries. [20:00:52] You can do it in the #! line of an executable... [20:01:03] #!invoke env PYTHONPATH=whatever peak etc... [20:01:35] <_jpl_> Speaking of 'invoke', will it be added to the setup.py at some point? [20:01:49] I myself run bulletins as 'PYTHONPATH=src bulletins etc...' [20:02:04] JPL: are you volunteering? ;) [20:02:18] Patches gratefully accepted. :) [20:02:20] <_jpl_> Sure :) [20:02:49] The distutils can in principle compile an executable, and in theory install it as a 'script'. [20:02:54] pje: Ok, that makes sense. Thanks [20:03:06] But, there are no distutils commands that are written to do it, that I know of, anyway. [20:04:57] ok, i have no idea how to use Zconfig :) [20:05:25] in my .ini file i have [20:05:31] [Log] [20:05:40] directory = '/some/directory' [20:05:45] Maniac: .ini and ZConfig are two different things. [20:05:45] <_jpl_> I'll have to learn a bit more about distutils... [20:06:04] to use this in zconfig i would? [20:06:28] Ah, sorry. [20:06:44] Maniac: you need to define a ZConfig schema first. [20:06:59] The schema says what settings people have, and what data types they are. [20:07:12] Er, not people, objects. :) [20:07:51] A ZConfig file is converted to a single object, of the class specified by the schema's top-level "Datatype" [20:08:19] So it's a completely different ball of wax than .ini files. [20:08:36] .ini files are mostly used to define PEAK configuration rules, not instantiate individual components. [20:08:38] <_jpl_> Maniac, you'd need a "sectiontype" for your [Log] section first. [20:09:07] <_jpl_> Within that sectiontype you'd have a "key" called "directory". [20:09:39] So, ZConfig is used when you want a user to be able to configure certain object instances directly, especially if they need to plug in more than one sub-component or have to supply lots of option information. [20:09:43] <_jpl_> (this is all in your schema, btw) [20:10:31] Btw, keep in mind that ZConfig comes from Zope, so feel free to inquire about ZConfig details in the Zope community as well. [20:11:04] The stuff I said earlier about the 'fromZConfig' constructor and using zconfig.schema: URLs are really the only PEAK-specific things you need to know. [20:11:13] The rest is just off-the-shelf ZConfig. [20:12:09] <_jpl_> With ZConfig you have the bonus of type checking, so your "directory" key could be of type "existing-file", for example, so that ZConfig would automatically check for the existence of the file during the validation of the configuration. [20:12:21] Brb... [20:13:54] <_jpl_> So that schema might look like this: [20:13:57] <_jpl_> [20:13:57] <_jpl_> [20:13:57] <_jpl_> [20:13:57] <_jpl_>
[20:13:58] <_jpl_> [20:14:12] ok so loading a schema is *seperate* from loading a config [20:14:30] <_jpl_> Right. To use it, you'd do something like this: [20:14:44] <_jpl_> schema = ZConfig.loadSchema(someFileName) [20:14:59] <_jpl_> config = ZConfig.loadConfig(schema, someOtherFileName) [20:15:14] <_jpl_> someFileName is the path to the schema file, while someOtherFileName is the actual configuration file. [20:16:18] <_jpl_> Actually, ZConfig.loadConfig returns a tuple, of which you usually want only the first item. [20:16:39] so that's straight Zconfig stuff. is there anything special to do this from PEAK? [20:17:01] <_jpl_> But continuing from the above, you could then reference "config[0].log.directory" to get the log directory value. [20:17:21] <_jpl_> That's where what pje was saying comes in [20:17:50] Back. [20:18:27] <_jpl_> Take a look at EventDriven.xml for an example [20:18:39] Hm... I think you meant '
' [20:19:01] <_jpl_> Oops, yes. :) [20:19:29] <_jpl_> Looks like I dropped a ">" too. [20:20:16] <_jpl_> Phillip, is there an example config file that uses EventDriven.xml? [20:20:32] Hm.. no, not at present. I should probably add one to the examples. [20:20:48] I have one that I use for simple testing... [20:21:04] It includes two tasks: one that checks that a file exists, and if not, creates it... [20:21:06] <_jpl_> So is your preferred use of ZConfig to have separate config files for each component you want to setup? [20:21:15] The other task periodically deletes the file the first task is looking for. :) [20:22:11] JPL: no, not really. [20:22:27] Right now, the preferred use of ZConfig with PEAK is to make a single, executable configuration file. [20:22:53] In fact, there's not really any way in PEAK currently to "just load" a ZConfig file. [20:23:02] ah, ok [20:23:21] <_jpl_> From which sub-components would be defined using datatypes and such? [20:23:30] right now (at least in this context) i'm only looking to load configuration parameters [20:23:36] JPL: yes [20:23:40] i.e. "just load" [20:23:59] using .ini is working fine in my little script [20:24:04] Maniac: consider that these "parameters" are probably associated with some component (perhaps your app itself) [20:24:26] So, if you define a top-level schema that instantiates your top-level app with the specified parameters... [20:24:34] it all comes out the same. [20:25:13] well, the root of the problem, then is everything i'm doing is still: >python myscript.py [20:25:20] vs: peak someMagicStuff [20:25:32] Your schema could be something like [20:26:10] Maniac: yes, that would be a problem. :) [20:26:27] <_jpl_> What would be the equivalent to "#!invoke peak runIni" and "[Load Settings From]" for ZConfig? [20:26:50] JPL: #!invoke peak zconfig.schema:schemaURL [20:26:59] and %include... [20:27:38] Maniac: note that you can use root.lookupComponent('zconfig.schema:whatever') to retrieve a 'ConfigLoader' for that schema. [20:27:42] pje: i have not yet wrapped my mind about that yet [20:27:51] <_jpl_> Could config.fileNearModule be used in the %include? [20:27:55] So, if you insist on using 'python myscript.py', you can still load and run the file. [20:28:00] JPL: nope, sorry. [20:28:21] <_jpl_> No problem, probably not necessray anyway. [20:28:23] pje: heh, i dont' insist i just don't know any other way currently [20:28:24] If you want to distribute config like that, why not just use pure Python? [20:28:30] <_jpl_> s/ray/ary/ [20:28:55] i dont' grok that magic yet [20:29:18] i'm very much at the : peak is usefull for sysadminish scripts currently [20:29:21] Maniac: just implement one of the running.IExecutable sub-interfaces. [20:29:26] <_jpl_> Maniac, just copy what Phillip has done for the bulletins example. Works like a charm. [20:30:14] Here's an example ZConfig file using the EventDriven schema: [20:30:14] #!./invoke wpython scripts/peak EventDriven [20:30:14] StopAfter 10 [20:30:14] [20:30:14] RunEvery 5 [20:30:14] Priority 1 [20:30:16] CheckURL file:/cygwin/home/pje/PEAK/foobar [20:30:18] RestartWith shellcmd:touch foobar [20:30:20] [20:30:22] [20:30:24] Priority 1 [20:30:26] Directory . [20:30:28] RunEvery 2 [20:30:30] DeleteIfOlderThan 2 [20:30:32] MatchFiles foobar [20:30:34] [20:30:50] This is the one I mentioned earlier, with the two duelling tasks. [20:30:57] <_jpl_> Nice. [20:32:03] Maniac: the simplest way to convert an existing script to a "command", is like this: [20:32:11] <_jpl_> So the keys within those sections are automatically checked against the appropriate class somehow? [20:32:16] from peak.running.commands import AbstractCommand [20:32:23] class MyCommand(AbstractCommand): [20:32:27] def run(self): [20:32:44] # whatever you'd have done in the script, but without having to [20:32:57] # set up a configuration root, since 'self' has such a root as its parent [20:33:16] And then you can run your command with 'peak import:mycmd.MyCommand' [20:33:36] And its 'self.argv', 'self.stdin', 'self.stdout', 'self.environ' etc. will be set. [20:37:32] pje: ok that worked [20:40:48] <_jpl_> Phillip, I'm still trying to figure out how that EventDriven config file works. EventDriven.xml's multisection uses type running.Task, which I can't seem to find anywhere. [20:41:31] JPL: notice the 'import package='peak.running'' [20:42:05] pje: what do you mean by 'without having to setup a configuration root' [20:42:07] Maniac: you can of course define bindings in MyCommand for whatever you want... bulletins uses this technique to create a base App class, that the various commands then subclass. [20:42:31] Maniac: you don't need to do a 'root = config.makeRoot()', since the commands bootstrap will do that for you. [20:43:50] <_jpl_> I'm still not finding a class called just "Task" anywhere in peak.running. [20:44:37] JPL: 'import' in a ZConfig schema loads 'component.xml' from the specified package. [20:44:49] So read 'component.xml' [20:44:58] <_jpl_> Oh! [20:44:59] (in peak.running's directory) [20:45:25] <_jpl_> Got it, thanks. :) [20:45:27] pje: Trying to work out the PEAK+twisted example you posted to the list using the ZConfig and EventDriven.xml. In the email you suggest I define a new Zconfig schema based on EventDriven.xml -- so does that mean I should create a file (EchoApp.xml?) That has everything EventDriven.xml has + my schema for the EchoRunner component? would I then place this EchoApp.xml file in the root dir of the app to run it, or does peak search for co [20:46:13] Jolby: not necessarily "everything"... I think you can include EventDriven.xml into your new schema. [20:46:16] hehe-- wow-- pje is getting hit from all sides w/ PEAK questions today! [20:46:45] Also, to run it, you would use #!invoke peak zconfig.schema:wherever/EchoApp.xml [20:47:36] Or, your sitewide PEAK_CONFIG can define a [peak.running.shortcuts] value for EchoApp, so that you don't have to type out the whole long #! line. [20:48:13] pje: Ahhh-- ok, dim rays of light are starting to illuminate my brain... :-) [20:49:58] pje: If I were to include EventDriven.xml in my extended schema, would I have to list the full path to the file, or would it automagically find it since I have installed peak? [20:50:47] Jolby: er, alas, I think you'd have to have the path. [20:51:08] Although... you might be able to use pkgfile: [20:51:28] I seem to recall hacking PEAK's ZConfig support so it uses PEAK's URLs, not just Python's. [20:51:52] hmmm. Ok, that's not really that big of a deal. I could just have a config dir where I'd put all the ZConfig schema files. [20:54:22] pje: ok i usually do root=config.makeRoot() [20:54:45] pje: then db=root.lookupComponent('config:MyDB.database/') [20:55:13] Maniac: so now you can just add: db = binding.bindTo('config:MyDB.database/') [20:55:27] in the body of the MyCommand class... and refer to self.db to use it. [20:56:33] Jolby: a quick test reveals that you *can* use pkgfile in ... [20:56:49] But alas, it's useless... ZConfig doesn't seem to let you extend top-level schemas that way. :( [20:57:39] Apparently, you must define top-level schemas fresh every time. I'll have to write Fred about that. [21:01:39] pje: i can still do PEAK_CONFIG='myApp.ini' at top of file ? [21:01:41] pje: ok, thats unfortunate, but not insurmountable. I guess for each app it will have to have its own ZConfig schema repository [21:01:51] Maniac: no... [21:01:57] aha! [21:02:05] Here's what you do instead... [21:02:17] At the top of myApp.ini, put #!invoke peak runIni [21:02:20] And add this: [21:02:24] [peak.running] [21:02:39] app = importString('myapp.MyCommand') [21:02:55] Now, just execute 'myApp.ini' directly... [21:03:04] And you can drop the '.ini' extension if you like. [21:03:24] Now, it's not necessary to set PEAK_CONFIG, but it will still be used. [21:03:47] PEAK_CONFIG is really intended for the site admin to set PEAK configuration defaults; applications should leave it alone. [21:04:29] So, on my server, I'll have PEAK_CONFIG set up for local naming services, any special URL types, etc... [21:04:55] If I run your app, it should honor my sitewide settings, unless overridden for the app. [21:05:25] where does 'invoke' come from? [21:05:37] 'invoke.c' in the scripts directory. [21:05:45] If you have a BSD OS, you probably don't need it... [21:05:46] oh, so i must compile this [21:06:00] Mandark 9.0 [21:06:04] It's for OSes whose #! support is broken. [21:06:09] Ah, you'll need it then. [21:06:29] how do i use it? [21:06:37] Many OSes don't split command arguments in the #! line.... invoke fixes that. [21:06:48] i.e. complile / install [21:06:58] <_jpl_> gcc -o invoke invoke.c [21:07:03] 'gcc -o /usr/local/bin/invoke invoke.c [21:07:08] Heh. GMTA. [21:07:26] ok, it's in the README [21:07:30] Yes. [21:07:48] Actually, I think it's in INSTALL.txt, but who's counting? [21:09:08] yeah, sorry INSTALL [21:09:13] ok my script works now yay! [21:10:29] ok now this is getting cooler [21:11:21] * Maniac notes PEAK just keeps on getting cooler and pje exlains it to the mentally challenged [21:13:29] <_jpl_> Phillip, how would one duplicate the [peak.logs] section of the bulletins .ini file using ZConfig? [21:14:14] JPL: that's kind of tricky. [21:14:30] ZConfig doesn't know about properties, so you have to do it indirectly. [21:14:46] For example, you could create a component class for configuring individual logs... [21:15:10] And have them have a 'whenAssembled()' method that registers the config property with a parent component... [21:15:31] Or, more straightforwardly, you can just use 'offerAs=[PropertyName('peak.logs.foo')] [21:15:40] on attributes that will be configured by ZConfig. [21:15:42] <_jpl_> That's what I was thinking, yes [21:15:54] That will only apply to child components looking for those properties, of course. [21:16:06] <_jpl_> Oh, hmm [21:16:25] Well, the component itself, too. [21:16:36] It just won't carry up to parent components. [21:17:01] <_jpl_> It could just be specified in the top-level config, which is then parsed out by the app object. [21:17:14] Yep. [21:17:45] how much of this stuff works on win32? does it require cygwin? [21:17:55] Generally, though, I tend to see the logging being configured in the PEAK_CONFIG by the site admin... [21:18:17] <_jpl_> I haven't yet figured out how to use PEAK_CONFIG... :) [21:18:18] Maniac: I regularly run tests using plain win32 Python (not cygwin Python) [21:18:31] And that includes today's Twisted [21:18:35] 'Echo' example. [21:18:37] pje: but your little invoke trick [21:18:38] And Bulletins. [21:18:54] Maniac: yes, true... I use cygwin in order to make #! work. [21:19:08] The alternative is too painful to contemplate. :) [21:19:11] pje: I just got your EventDriven script you posted earlier to work-- but I'm sort of baffled as to why it works -- Specifically, how does the line: #!./invoke peak EventDriven know how to find the EventDriven.xml schema file? [21:19:22] <_jpl_> For win32 you'd probably just make a .bat with "peak whatever whatever"... [21:19:35] Jolby: see [peak.running.shortcuts] in peak.ini [21:19:48] JPL: yes, exactly. It's just a bloody pain, that's all. [21:20:51] so you could define a [peak.running.shortcuts] for sampleApp and do: peak sampleApp [21:20:52] ? [21:20:57] Maniac: yes [21:21:11] This is also an advantage of a site-wide PEAK_CONFIG. [21:21:21] and sampleApp could be still sampleApp.ini [21:21:34] JPL: PEAK_CONFIG=/path/to/my/sitewide.ini peak whatever [21:21:56] Maniac: Hmmm... not sure about that. [21:22:04] <_jpl_> Ah, which could then set various [peak.*] config settings... [21:22:04] hmm [21:22:20] Actually, yes... yeah... it's just a little awkward. [21:23:03] <_jpl_> So that's where log settings would sit most comfortably, it seems. [21:23:21] Something like sampleApp = importString('peak.running.commands.IniInterpreter')(argv=...) [21:23:31] <_jpl_> Now I see that the setting used in the bulletins .ini is also the default in peak.ini [21:24:14] Maniac: really though, for apps that need their own .ini file, it's simpler just to make the ini file the app. [21:24:28] i.e., 'sampleApp whatever' instead of 'peak sampleApp whatever' [21:24:53] ok, what i'm saying though, is that on win32 the #!invoke stuff wont' work [21:25:09] pje: Ahh-- ok. Starting to see the connections now. [21:25:10] afaik [21:25:18] Maniac: so you'll need a .bat file for each app. [21:26:06] how would you use the .bat file [21:26:13] @echo off [21:26:38] <_jpl_> Maniac, I mentioned that earlier... [21:26:46] --> queuetue has joined #PEAK [21:26:51] \Python22\python.exe \Python22\Scripts\peak whatever... [21:27:07] Hi Queuetue [21:27:11] Hi. [21:28:16] I'm trying to understand why peak is useful. Mind giving a quick lesson? (And expect me to be argumentative... :) ) [21:28:21] Sure. [21:28:38] If you're writing only *one* application, PEAK doesn't give you much. [21:28:45] (Well, it depends on the app, actually) [21:29:05] But if you're writing *more* than one... [21:29:13] It starts to pay off pretty quickly. [21:29:19] Specifics... [21:29:44] The main packages are peak.binding, peak.config, peak.naming, peak.storage, peak.running, and peak.metamodels... [21:30:08] Each of these is fairly competive with similarly-purposed independent Python packages. [21:30:30] Then why compete? [21:30:46] Superior integration. [21:31:23] If I get config package X and persistence package Y and command package Z and metamodel package Q... [21:31:38] Each has its own things expected as base classes, different ways to configure, etc. [21:31:51] Ever done any J2EE programming? [21:32:01] I guess my first "issue" is goign to be with the scope and size of PEAK. I have the same issues with Zope and Twisted, although I appreciate that Zope splintered ZopeDB off... It's tough for me to buy into a single-source monolithic framework... Especially one that expect em to stop writing python code and start writing framework code... [21:32:15] Very little. Let's say no. :) [21:32:38] I personally find Zope and Twisted "too big" also. [21:32:46] PEAK is a *heck* of a lot smaller than either. [21:33:02] I think about 1/10th the size, but that's an off-the-cuff estimate. [21:33:13] Smaller in footprint, but not in scope and overal affec ton my code... [21:33:20] <_jpl_> And PEAK covers a lot of application "bases" that neither Zope nor Twisted even consider. [21:33:23] * queuetue has a broken space key... [21:34:15] Q: that's why there's (for example) 'peak.util', which is all stuff that doesn't require other parts of the framework. [21:34:42] Lots of miscellaneous library modules that are used by the rest of the framework, but don't themselves require the framework. [21:34:54] Why do I have to change my coding style so dramatically to take advantage of a framework? Why can't "enterprise scalability" be managed like a mixin, instead of making me think about interfaces and such? [21:35:27] Q: because enterprise scalability is about *infrastructure*. I'll give you an example from my "day job"... [21:35:42] We have multiple web servers running a large Python app, backed on a large DB... [21:35:52] Sounds familiar. :) [21:36:00] We have test databases, live databases, training databases, replicated LDAP servers... [21:36:19] DB connections to multiple (non-Python) applications' DBs that get replicated... [21:36:49] We need to distribute knowledge of these things to our monitoring tools (in Python/using PEAK), [21:37:08] and to our various apps that parse mail, send mail, do periodic tasks... [21:37:24] So, a common infrastructure is important. [21:37:39] Your last staement doesn't follow. [21:37:54] * queuetue smashes his keyboard... MS junk. [21:38:17] Q: we *have* a common infrastructure... so it's important. :) [21:38:19] pje: A common *api* maybe, but aocmmon infrastrucure? Why? [21:38:26] <_jpl_> It follows if you have no interest in reinventing the wheel over and over. [21:38:36] Q: you're confusing architecture and infrastructure. [21:38:52] Infrastructure means that we have all these common services and resources. [21:38:52] Ah, I thought *you* were. :) [21:38:57] That's the common infrastructure. [21:38:59] Continue. :) [21:39:12] Our apps don't necessarily have common architecture... except that they're in Python. [21:39:18] Some are more PEAK-based than others. [21:39:50] At the lightweight end of the spectrum, we have a command-line app "db" that just looks up a "infrastructure name" for a database and connects us to a shell for it. [21:40:13] We have web apps, daemons, and command line apps... [21:40:29] Over time, we developed a library called "AppUtils" to do some of this stuff... [21:40:30] Why isn't that just in LDAP? Why a special naming mechanism? [21:40:55] Q: because we need names for the LDAP servers too... you have to start somewhere. :) [21:41:10] And we have a lot of LDAP servers. Also, LDAP's not that reliable. :) [21:41:21] so, the PEAK naming service is broadcast or something? [21:41:36] How do you avoid chicken-and-egg there? [21:41:36] No, PEAK wraps atop our other naming services. [21:41:52] There are a bunch that we have to deal with, actually, some legacy. [21:42:01] peak.naming puts them all under a common interface. [21:42:45] ... And how do you avoid the chicken-and-egg problem? [21:43:02] Our root naming service is file-based. [21:43:16] And pushed out to the cluster. [21:43:20] * Maniac leaves for home [21:43:24] So it could be LDAP anyway, because you already have your "start." [21:43:27] <-- Maniac has quit #peak [21:43:28] <_jpl_> (I'm thinking of writing an SLP component for peak.naming, btw) [21:43:47] * pje shrugs [21:43:52] SLP? [21:44:12] <_jpl_> Why would you want it to be *based* on LDAP, when not everyone will be interested in "starting" with LDAP? [21:44:42] <_jpl_> Service Location Protocol, RFC 2608 [21:44:49] Because I don't believe in recreating standards, and this smells like that to me. I jsut want answers - I'm not calling you dumb or anything. [21:44:54] <_jpl_> See also http://www.openslp.org/ [21:45:01] _jpl_: Tx. [21:45:24] God loves standards... look at how many there He created. :) [21:45:33] <_jpl_> peak.naming doesn't recreate any standards, it merely acts as a front end to anything you'd like to look up. [21:45:46] peak.naming is actually a Pythonic clone of Java's JNDI. [21:45:47] My next questions would have been about rendevous (sp?) but I think SLP is that, in standards form... [21:46:49] I guess if you have a problem keeping lists of resources straight, then a naming server makes sense. I don't see the point yet, but I'll accept that it's important. [21:46:54] But, unlike JNDI, it doesn't have its own special configuration system (as it shares peak.config), and it has a mechanism to auto-load names (via peak.binding) [21:47:25] Q: Keep in mind that this is an *enterprise* toolkit... and not in the weak sense of that buzzword. [21:47:57] <_jpl_> peak.naming is not a naming *server* at all! [21:48:20] So, if you don't work on big systems, you may not see some of these things as problems. [21:48:36] Here's the first case where we run into what I consider "forced interoperability." Why did you create your own config? Python has one, twisted has one, zope has one, why did you need your own, and why do I have to use it to use the other stuff? [21:48:48] Q: PEAK works with ZConfig. [21:49:03] Includes it, actually. [21:49:04] <_jpl_> And with Python .ini files. [21:49:11] It's the same, or it's acceptably close, or it works with either specifically? [21:49:17] Ok. [21:49:20] However, what 'peak.config' is really about is publishing resources *internally* to an application. [21:49:51] Q: it uses ZConfig code... subclassed to add support for PEAK naming system URLs; "normal" ZConfig only supports certain URL types. [21:50:12] Doesn't ZConfig store its data in ZopeDB? [21:50:18] Q: no... files. [21:50:21] K. [21:50:21] <_jpl_> peak.config makes it simple for various interconnected components to find the configuration options they need, whichever actual config file format you use. [21:51:01] Not just options, but other components/services. [21:51:25] So, by defining an attribute with an 'offerAs=[]' setting, you "publish" that attribute to subcomponents under the specified search keys. [21:51:27] <_jpl_> Those too :) [21:52:51] Is that it? [21:52:55] naming, config, and binding are all "horizontal" frameworks... meant to be used in any app. [21:53:16] storage, running, and metamodels are more "vertical" frameworks, specific to certain kinds of app. [21:53:27] running? [21:53:50] peak.running covers event loops, command line apps, logging, locking... [21:53:54] Think "process control" [21:54:20] <_jpl_> And note that you can use naming, config, and binding along with other frameworks like Twisted. [21:54:51] Then why not spin them off into seperate projects? Why maintain the monolith? [21:54:57] <_jpl_> I'm currently using PEAK with OSE (http://ose.sourceforge.net/) with great results. [21:55:17] Q: are you volunteering to maintain a bazillion different frameworks, each with its own release? [21:55:25] God knows I barely have time for the one. [21:55:32] You already are - you just tarball them together. [21:56:02] Q: just look at the setup.py and tell me that it'd be easier to maintain 6 or more setup.py files for this stuff... [21:56:11] <_jpl_> Q: There's not much of a monolith. Install PEAK and use just what you need... you're not going to run into any disk space problems. :) [21:56:26] And then tell me that my package rollout process across all my webservers is going to be easier when I'm tracking different releases of the packages... [21:56:39] <_jpl_> PEAK will also uses lazy loading of modules, so you'll only import what you actually end up using. [21:56:43] Seriously, this would be several times as much work for me to distribute separately. [21:57:03] The problem I'm hinting at is one of insulation, not disk space. [21:57:20] PyProtocols is the only subpackage I distribute separately. [21:57:36] THe point is that if "naming" is important enough, and atomic enough, then it could be a general project, and would (hopefully) draw it's own set of developers, and grow to be more generally useful than the one you are building as "part of peak." [21:57:53] <_jpl_> Q: I understand, but what's the problem with just using the parts you need? [21:57:54] Q: naming uses config for its config, and binding to build the components. [21:58:17] For my purposes, it's insanely more work *not* to use the pieces I've already written! [21:58:29] What was the point in writing them if I can't use them? :) [21:58:55] <_jpl_> Perhaps a "PEAK Lite" which includes only config, binding and naming would satisfy Q. :) [21:59:14] That's like saying the Python standard library should import things from other standard library modules... [21:59:19] Er, shouldn't. [21:59:24] :) [21:59:34] <_jpl_> And those three packages alone could certainly make a big difference for a lot of people. [21:59:39] THe poitn is that PEAK grows closer to what the community wants it to be - including you, not just what you think it shoudl be. Insulation is what keeps webware, zope, twisted and peak from gainign the popularity... [21:59:48] <_jpl_> Not that I think they need to be separate. :) [22:00:02] I'm not sure what you mean by insulation. [22:01:17] Python is a groupthink tool. It's designed by an active community that argues and builds a best-of-breed chunk of technology. [22:01:36] It seems to me almost that you're saying a package should only depend upon the standard library, not any other packages. [22:01:38] PEAK is built by you and your posse. And it's pretty good for you, and you try to make it generaly useful. [22:02:06] But it's hard to hit group-acceptability for multiple components. [22:02:23] The reason frameworks end up "big" (c.f. "sumo distributions"), is because Python packaging utilities don't deal with dependencies well. [22:02:54] If there was a Pythonic CPAN, I imagine Zope, Twisted, and PEAK would be a lot more "open" in the sense of lightweight component distribution. [22:04:05] I keep saying this in #python, but no one sees it. Pypi isn't even planned to have that functionality... [22:04:17] Right now, the ideal number of packages for a given maintainer to publish and distribute, is no more than 1. :) [22:04:42] Couple that with the Pythonic philosophy of "batteries included", and voila! [22:04:47] You get the current situation. [22:05:00] But, my point was that if instead of building a naming service, if PEAK used an existing one (even if you had to help develop that other one) then it would be much more appealing to me, and to a large group of developers, I think. The end result would be that you "feel like you're using python", with PEAK's help, not that you're uysing PEAK, which is based on Python. [22:05:33] Q: I think you've missed the point of peak.naming. [22:05:51] It's to have a common API for *any* naming service. (Think DBAPI for naming services.) [22:06:11] As I said, it's like JNDI: Java Naming and Directory *Interface*. [22:06:58] I suppose my point isn't heard. That's ok. It's your time, and you're free to spend it any way you want. [22:07:16] Q: I get your point... but if you *need* what PEAK does... [22:07:21] Then it's worthwhile. [22:07:32] If you don't... you're not really looking for what it does. [22:07:41] I agree with you re: Twisted and Zope... [22:07:49] And I've tried to keep a minimum of that in PEAK. [22:08:00] But there's only so far I can go that way. [22:08:17] That's why PEAK interoperates as much as possible with Zope and Twisted. [22:08:25] I don't want to reinvent perfectly good wheels. [22:08:52] For that matter, that's why peak.util is there... those are indepenent libraries. [22:09:23] Beyond that... it's hard to spin anything out into separate packages, because they start to be dependent on other packages. [22:10:20] If there were better distribution tools, I imagine I'd distribute finer-grained packages. [22:11:04] I don't really understand why it's more difficult to release 4 packages than to release 4 modules in one package. [22:11:12] WHat's the logistical problem you're having? [22:11:25] <_jpl_> (time?) [22:11:38] Queutue: every package needs its own README, INSTALL, setup.py CHANGES... [22:11:48] <_jpl_> README/INSTALL/CHANGELOG for four packages versus one? [22:12:10] Building the bdist and sdist files, tagging CVS... [22:12:19] You already have to cover all of the same information in the "consolidated" one now... Hwo could it take more time? [22:12:23] Uploading the dist files, making announcements to the appropriate lists... [22:12:28] Q: duplication. [22:13:26] OK, like I said - it's your time. [22:13:30] Just changing the stupid bloody version numbers across all those doc files and scripts takes a bunch of time right now. [22:13:32] <_jpl_> Q: Would you be willing to volunteer to be the release engineer for several smaller subpackages? [22:13:47] You know, I might. [22:14:15] <_jpl_> Because this would take a dedicated release engineer... right now PEAK pretty much *is* Phillip. He doesn't have a "posse" helping him out. [22:14:21] As long as that package was allowed to have a life of it's own. (and I understood the problem domain well enough to be a valuable contributor) [22:14:38] Depends on what you mean by "life of its own". [22:14:39] <_jpl_> More like a few groupies who watch the CVS directory every day to see what new goodies he's given us. :) [22:14:43] _jpl_: I was guessing that you were posse #1. [22:15:26] <_jpl_> Actually, I'm a complete beginner with PEAK. Just got my first app working last week, but it's made me a convert. [22:16:01] JPL: I suspect that Q views that as a negative. :) [22:16:11] <_jpl_> Just using peak.binding and peak.config, I've solved all the configuration difficulties I was having pre-PEAK. [22:16:19] <_jpl_> Hah, could be. [22:16:23] That is, I'm assuming you don't like "religion" in your software, right? :) [22:16:24] pje: Here's the crux. WHat I'm saying is that the best thing for PEAK is if (for example) naming was developed as a useful standalone app, which benefits many different projects. I'd introduce it to WebWare, to Zope, and we'd try to get as many hand son it as possible, making it generally useful - but this may require PEAK ot adapt to the community will. [22:17:07] Q: if you insist that every package depend on nothing more than the standard library, then the only development that will take place is of the standard library. [22:17:43] That's not what I'm saying. I'm saying that a package should not be dependant on it's dependants. (Does that make sense?) [22:17:46] I can't build a building if I'm not allowed to put the bricks on top of other bricks. [22:18:01] You're saying no circular dependencies, is that it? [22:18:34] Yes, but that's a degenerative case of the bigger problem - tight coupling. WHat microsoft (and I think you) call interoperability or integration. [22:18:55] * pje laughs [22:19:11] There's coupling and there's coupling. There's *made of*, and there's *requires*. [22:19:18] * Maniac- is a peak groupie not posse [22:19:25] heh [22:19:28] PEAK is *made of* PEAK. But PEAK doesn't *require* PEAK. [22:19:39] Is that making any sense? [22:20:05] Zope 2 *requires* Zope 2, and in many places, Twisted *requires* Twisted. [22:20:19] the other thing with PEAK adoption is it's just *very* recently started to publish actuall working examples :) [22:20:22] PEAK happily accepts things that implement (or can be adapted to) the interfaces it wants. [22:20:48] * pje can't release much working PEAK "example" code because its employer-owned [22:21:10] Here's the way to start this. I'm a startup "Striek team" consultant. Every time I'm called into a company by investors, I make them tell me the "bumper sticker" that the organization has on it's car. If it's too long to fit there, then you'r enot focused enough. WHat is PEAK's bumper sticker? [22:21:31] Python Enterprise Application Kit. [22:21:35] :) [22:22:01] If that was on a bumper sticker, could you jsut go "Oh, ok." and ge tit immediately? (assuming you're not you, of course. :) [22:22:24] Well, I'd figure it was some kind of kit for making enterprise applications in Python. [22:22:44] And that is pretty much what it's intended to be used for. Not what it *is*, granted, but what it's to be used for. [22:23:05] For ZopeDB, I'd say "Making objects Stick." For naming (if I understand it) ,I'd say "Helping you find services in the wild." [22:24:01] PEAK is too wide a target to understand, is my point. naming, I think I could understand, given a day or two. And I think coupled with SLP, it could be a kickass module, probably a candidate for standard library inclusion. [22:24:15] I imagine for PEAK as a whole, it would be "J2EE made simple and Pythonic". [22:24:52] If I didn't love Python so much (and Java performance didn't suck so badly) I'd probably be a J2EE-head by now. [22:25:21] Binding, Running, etc, are probably similar. Decoupled from the whole of PEAK, I bet they are all very attractive - to the community as a whole. As a "monolith" I think they turn people off, because there simply isn't enough community involvement... I'm only here because I see value. I'm not trying to piss anyone off... [22:26:06] <_jpl_> Remember PEAK is still in pre-release stage... [22:26:39] But I bet "naming" could be shored up to a 1.0 release pretty easily, as long as "supports what PEAK needs from it" isn't one of the 1.0 milestones... [22:26:46] <_jpl_> So there hasn't been much chance for community involvement yet. [22:27:03] pje: Noooooo! I'm a J2EE refugee, and I'm feeling pretty happy to be out of that prison. I hope PEAK stays much lighter-n-simpler than the abomination that J2EE has become... [22:27:20] Jolby: I did say J2EE made simple and Pythonic. [22:27:27] DO I just sound like a jerk, or am I making a *little* sense? [22:28:04] Q: All I'm hearing is that some people don't like large Python frameworks, which I already knew. [22:28:13] I'm trying to understand what else you're saying, and it's not sticking. [22:28:38] PEAK has roughly the same scope as J2EE, minus the bits that already exist for Python. [22:28:50] Think of it this way: If someone else build JNDI for python, would you have built your own? [22:28:59] <_jpl_> Q: I think you're being understood loud and clear, only naming/config/binding are all pretty tightly coupled... it's not always possible to write sub-packages in a loosely coupled way and still maintain the same level of functionality. [22:29:35] pje: hehe-- ok good. Although in J2EE's defense (and PEAK's) the idea of easily composable+reusable components is a good one-- its just that the real-life implementation *sucks* in J2EE's case... [22:29:55] Q: no, if it met all of my requirements. [22:30:02] _jpl_: Never, in my experience, has that been a true statement. It has always meant that the problem was misunderstood, or the implementation was not properly factored... [22:30:39] pje: Ar eyour requirements unorthadox, or are they pretty much what anyone else would need, too? [22:30:45] Right, the problem with J2EE is that unless you're an expert, it's trivial to create an unscalable application. :) [22:31:03] It's not that it's impossible to make a good app, just unlikely. :) [22:31:54] Q: That depends. [22:32:01] <_jpl_> Q, you might then want to become more familiar with the binding/config/naming code and see where PJE has gone astray and suggest how you might make things more orthogonal. [22:32:16] I don't want 90 jillion pieces patched together. I want a "PyEE". [22:32:22] darnit. :) [22:33:00] pje: Ok, then that's cool. I want seamless components - you want a consistent framework. That's fine, and I'll shut up now. [22:33:12] I've tried to make it easy - where it didn't conflict with other PEAK goals - for someone to use a non-PEAK component, or to replace a PEAK subsystem, etc. [22:33:41] Argh. [22:33:50] PEAK *does* seamless components. [22:34:17] It *never* requires you to subclass a framework class to achieve some functionality. [22:34:31] I don't mean PEAK compoennts - I mean all of the other modules, currently existing or simply future "pythonic". [22:34:33] For me the problem wasn't necessarily the scalability, but the complexity. All the xml-hell in all the descriptor files, and don't even mention the insane memory/cpu loads and the HUGE amount of time needed to bounce the server or even just redeploy a large app... blech... [22:34:54] <_jpl_> Seriously, get to know the code and if you can come up with a way to release a separate peak.naming without sacrificing functionality, the effort would probably be welcome. [22:35:24] Q: I don't understand how your approach would allow for any *synergy*. [22:35:43] binding+config+naming = greater than the sum of three disparate packages [22:36:40] I think that what PEAK "is" is a small, very difficult thing. And I think you're required to build a dozen or so semi-hard things around that core, just to bring Python up to speed. What I want is for those semi-hard things that aren't really PEAK - they're just missing python functionality - to be less coupled to the core that I think you want to build. I'm probsably wrong, but that's what I think... [22:37:36] I don't know if you're wrong or right. [22:37:54] I feel that those pieces, made solid and shared across multiple packages, would serve to bring the whole python frameworks community closer together. [22:38:17] Here's an example of something though, that might let you test your hypothesis... [22:38:35] <_jpl_> I think eventually PEAK could be the framework upon which other frameworks are built. [22:38:38] For event-driven applications, PEAK can either use a twisted.internet.reactor or its own simple reactor. [22:39:03] I had no desire to duplicate the "hard" work done in Twisted for dealing with different GUI event loops, connections, protocols, etc. [22:39:03] _jpl_: Twisted, Zope, and MetaKit all think so, as well. [22:39:29] * pje has no illusions about that. [22:39:51] So, there's an interface for that, and two alternatives for implementing it. [22:40:00] DB connections are "hard"... so I use DBAPI. [22:40:25] <_jpl_> But none of those kits do what PEAK does, and what PEAK does could be incredibly useful for those other kits. [22:41:04] pje: Ok, good examples. I've talked ot the twisted crew about splintering reactor. I think it's not a twisted module, but they have a much deeper version of the disease I'm accusing you of having... [22:41:11] JPL: the reality is that each kit specializes in certain things, and nobody is going to rewrite their entire framework to use PEAK. [22:41:32] <_jpl_> Agreed. [22:41:47] The best I can hope for (and have gotten) from the Zope and Twisted crews is occasional help with ensuring compatibility and avoiding toe-stomping. [22:42:27] Q: peak.running contains a 'BasicReactor' that does the simplest reactor-ish things for the simplest cases. [22:42:57] For apps that don't use more sophisticated Twisted abilities, you can use the PEAK reactor instead of Twisted's, as I didn't want PEAK to depend on Twisted for simple things. [22:43:07] <_jpl_> Q is lamenting on the lack of a general purpose framework upon which other frameworks would be built -- I think PEAK could probably fit the bill, but you can't *make* other framework builders use your tools, be they in one "large" package like PEAK, or in smaller, more focused packages like "naming", "binding", etc. [22:43:12] pje: I guess what I'm arguing for is to take the common denominators and free them as seperate projects. Zope has been unwilling, but has shown great promise with the zopedb issue. Twisted believes that being monolithic is part of the core of thier value proposition. There's a "that wa sinvented here" sicnkess in the python frameworks froup that I don't like... [22:44:03] Q: Zope 3 is much less monolithic. peak.web will use Zope X3's 'zope.publisher' and 'zope.i8n' packages, separate from the 'zope.app' monolith. [22:44:20] _jpl_: But, if JNDI-like-ability was available, uncoupled to any framework, then ther eis a much greater chance of all frameworks sharing it. You're already converging on a shard Interface standard... ZopeDB bring some value for persistance... [22:44:56] Shared interface standard? Ha. PyProtocols *forcibly* adapts Zope and Twisted interfaces to a "common" standard. :) [22:45:37] pje: Ah, then your marketing skills are strong. :) THat's not the way I read it off of your site, you were very diplomatic about it. :) [22:46:09] Oh, JimF and Glyph certainly know the score, there. [22:46:31] <_jpl_> Again, Q, look into peak.naming code and see if there's a way to separate it without losing functionality. You're probably right that if there was a PyNaming (like PyProtocols), it might be used more widely. [22:46:36] The real diplomacy is in not pointing out that Zope and Twisted interfaces only pass a fraction of the unit tests that PyProtocols do. :) [22:46:44] The point I keep beating here is that if something is generally useful, it shoudl be a seperat epackage, for PEAK's good, for the package's good, and for the community's good, IMHO. [22:47:11] It's just my opinion, and if it sucks, than it sucks. [22:47:39] Q: if the usefulness is diminished by separation, or it depends on another part of PEAK, then what? [22:48:11] Do I make 6 or 8 separate distributions, all identical but for the docs? :) [22:48:30] E.g. I make a 'naming' distribution that includes binding and config, but doesn't talk about them? :) [22:48:57] No. You define the interfaces better, and you (or someone - maybe it's me!) do the hard surgery to seperate them and keep both patients alive and well. [22:49:07] * pje sighs. [22:49:16] Okay... here's an example... [22:49:40] peak.naming includes base classes for naming contexts... think of it as if the DBAPI actually included some base classes for you... [22:49:58] Those classes derive from binding.Component, because they want access to configuration data. [22:50:11] Now, you *could* write your own implementation of a naming context, without using that base class. [22:50:17] Because the interfaces are well-defined. [22:50:27] But, if you want to do that, then why bother having the base classes? [22:50:44] peak.naming would then just be a bunch of interfaces, and no implementation. [22:50:49] What good is that? [22:51:10] And if I'm going to write the implementation, I'm going to write it using tools that make it easier to write... [22:51:14] First of all, how does JNDI handle this? Not that we're slaved to thier implementation, but just as a first indicator... [22:51:29] Q: JNDI has its own configuration machinery. Lots of ti. [22:51:31] er, it. [22:52:19] peak.naming just uses peak.config to define the structure, and peak.binding to access it. [22:52:43] Also, JNDI doesn't provide you with any bases... you have to write naming contexts completely from scratch. [22:53:08] Actually, with JNDI you either get your implementation from Sun or from a vendor who's taken the time to write their context from scratch. [22:53:11] Think about PEAK.naming from the perspective of Twisted. THink of it form the perspective of a person who's just trying to write an LDAP CRM all by themselves. [22:53:16] Either way, you have to deal with the crummy configuration system. [22:53:37] An LDAP CRM? Customer Relationship Management? [22:53:42] Yeah. [22:53:58] No framework - they just need naming support. [22:54:16] * pje nods. [22:54:36] For LDAP, why not just use the Python LDAP module? [22:54:56] Now, if the problem is that they need to know what LDAP server to connect to... [22:55:01] Thier LDAP server moves a lot - they want naming to help them find it. [22:55:05] And what login information to use to log into it... [22:55:08] Yeah, that's the problem. [22:55:12] And what DN the stuff is at... [22:55:19] Yup. [22:55:32] Okay... I guess I'd do this: [22:55:43] from peak.api import config, naming [22:56:04] myLDAP = config.makeRoot().lookupComponent('theLDAPServer') [22:56:34] Voila. No further PEAK needed, except the knowledge to setup a PEAK_CONFIG file with the necessary info to make this work. [22:56:55] Hmm... looks like I didn't even need to import naming. :) [22:57:14] Ok, then what is naming for? [22:57:32] lookupComponent() ends up calling naming.lookup(). [22:57:32] pje: Hi-- I'm still working on the sample twistedpeak app, and I'm trying to figure out how peak.running.component.xml gets loaded, and how I should load it (plus the additional stuff for the echoserver component) [22:57:41] Ah. [22:58:09] Jolby: in EventDriven.xml, there's an , that's what does it. [22:58:17] Q: I could've written it as: [22:58:22] root = config.makeRoot() [22:58:36] myLDAP = naming.lookup(root, 'theLDAPserver') [22:58:57] This is roughly equivalent to the two-line Java incantation: [22:59:10] env = new Properties(); [22:59:20] ctx = new InitialContext(env); [22:59:41] myLDAP = (ldapConn) ctx.lookup('theLDAPserver'); [22:59:47] Er, make that three-line. [23:00:04] And I'm not sure that's the correct Java. :) [23:00:14] So, what I would do is eliminate the dependance on config, and make that a "glue code" situation, splinter it away from PEAK, and you're pretty code to home... [23:00:23] pje: Hmmmmm.. sorry for being dense. Does the ZConfig machinery automatically search for a file called "component.xml" in any import package directive, and then automagically load it? [23:00:27] s/code/close/ [23:00:42] Q: you can't eliminate that dependency... unless you want to invent a configuration system just for peak.naming. [23:01:06] If you read up the JNDI specs, you'll see just how complex the required configuration system is. [23:01:39] And peak.config isn't really much more powerful than what you'd need to implement a JNDI configuration system! [23:01:51] <_jpl_> Q: As I keep suggesting, get to know the code before assuming they can be so easily split apart... [23:02:05] pje: No, you create a general Naming object that is loaded through a standard API. Then write a chunk of glue code to load it from one of your config's. Abstract. [23:02:12] Jolby: yes. [23:02:29] _jpl_: Whether it's easy or not is completely unrelated tho the conversation. [23:02:36] Q: you don't get it. I think you'll need to look at the JNDI specs. [23:03:17] OK, good enough. Once again, I'll shut up. [23:03:56] When I first began the port, I had a monolithic (i.e. self-contained in naming) configuration system. [23:04:09] I then factored that *out* to become peak.config. [23:04:26] But not far enough that it became PyConfig. [23:04:46] Hm. [23:04:52] Almost... [23:04:53] No... [23:05:05] A lot of config pieces need binding. [23:05:26] pje: Ok thanks-- getting closer and closer. (I think) [23:05:26] Or to be precise, they're a lot easier to write using binding. [23:05:35] I don't think they need it, but you want them to have it - that's acceptable. [23:06:09] Binding has a way of growing on you... until writing code without bindings is like fingernails on a chalkboard. [23:06:53] There were times, writing PyProtocols when I almost gave in to my addiction. :) [23:07:05] If Guido and the PEP crew implemented it as core python, I'd agree with you. UNtil then, your dependance on it is a barrier to adoption, not a boon to it. [23:07:22] * pje shrugs. [23:07:33] Unless, of course, other people become addicted to them. :) [23:07:37] Geeze, /kick me already. :) [23:08:01] Q: more likely, I'll have to leave first. [23:08:47] <_jpl_> I've become addicted to binding, too. Don't know how I got along without it. [23:09:23] Yep, technically speaking, nobody *needs* binding, any more than they need Python instead of just using assembly. :) [23:09:48] I want more "Enterprise ready" tools in python. I just require them to be pythonic. Peak doesn't feel that way to me, and I think it's because it's so well suited to what you need, and well-tuned to how you write code. [23:10:33] <_jpl_> Q, regarding the "ease" question, it's not just about the ease of the work, but whether you'll be able to retain the same level of functionality as well. My point is that you might want to get to know PEAK a bit better to first understand why pje has done it the way he's done it, after which you'll be in a better position to make suggestions. [23:10:57] * pje shrugs. [23:11:03] I don't know if you're right or wrong. [23:11:14] _jpl_: Yes, I agree completely. It would have to be refactored in a way that maintained the existing functionality, with stubs and proxies if necessary. [23:11:20] <_jpl_> PEAK doesn't seem "pythonic" to you? [23:11:27] _jpl_: No. [23:11:34] I think that PEAK is Pythonic for the things it wants to do. But those things are not necessarily what everybody wants to do. [23:11:55] Interesting. [23:12:09] I'd be very interested in comparing JPL's reasons why, and Q's reasons why not. :) [23:12:29] I imagine both will differ from my own thoughts on this point, and thus be educational. [23:13:04] (Btw, I hope you don't mind me abbreviating you as Q... a bit late to ask, I suppose :) [23:13:16] pje: And that's the primary value of building tools for a wider audience than yourself. The education you recieve is amazing. [23:13:22] Nope. [23:13:29] <_jpl_> Well, it's using metaclasses to do all kinds of interesting things, so I suppose the way those "things" are implemented is not what is currently thought of as "pythonic" -- but metaclasses are still a pretty new concept, and open the door for all kinds of "strange" new ways of doing things. [23:13:37] Er, Q, if I wasn't interested in that, I wouldn't have published the bloody thing. :) [23:14:14] pje: PEAK is for a wide audience. peak.* is for your (PEAK's) use. [23:14:25] JPL: you're confusing me now. I wanted to know why you thought it *was* Pythonic. Q will tell me why he thinks its not. :) [23:14:43] Q: eh? [23:14:58] That went right over my head. [23:15:42] (JPL: also note that many binding tools will work *without* special metaclasses, and even in classic classes.) [23:15:53] peak.naming was built for PEAK's use. Not for some random app developer. [23:16:13] Ah. Understood, but not entirely true. [23:16:28] The naming package predates PEAK, and was originally built without it. [23:17:07] Ok, if most random application developers wanted peak.naming to migrate in one direction, but PEAK's planned path required it to bend another way, which would win? Would there even be a vote? [23:17:12] The config package got factored out of it, and the binding package grew up alongside it. [23:17:21] <_jpl_> pje, as in calling binding.something on an object not derived from binding.Component? [23:17:26] Q: of course not. I need this for my day job. [23:17:40] Just like Zope Corp needs Zope for theirs. [23:18:25] pje: Right. And (here's the hard part...) PEAK is injured as a result. Because the group mind will come up with better usage patterns than you will, as long as they buy into your core value proposition. [23:18:28] And, just like Guido can veto anything going into Python, I might add. [23:18:35] * pje chuckles. [23:19:03] Like the "group mind" knows better than Guido? [23:19:06] <_jpl_> Personally I don't buy into the "group think is better" concept, at least not as an absolute rule. [23:19:14] Sorry, I don't buy that one. [23:19:38] Guido is great at absorbing and reflecitn the group mind. [23:19:38] Systems need designers, preferably with one chief. [23:19:50] reflecting... [23:19:57] <_jpl_> Projects *need* leaders in order to get "somewhere" before there's anything to even discuss. [23:20:00] If he "absorbed and reflected", he wouldn't disagree with them so often. :) [23:20:12] Leaders are not dictators. [23:20:17] The point is for him to know *better* than the group mind. [23:20:25] AH, neve rmind. [23:20:38] Forget I brought it up. [23:20:42] * pje shrugs. [23:20:57] Certainly I "absorb and reflect" feedback from the community that PEAK has so far. [23:21:38] But I do decide, and from a personal financial POV, I have to. The group mind doesn't pay my bills, I'm afraid. [23:21:50] Yes they do - they made python for you. [23:22:01] <_jpl_> Q, you'seem to have come to this discussion with a lot of fixed assumptions, like what sort of attitudes you're going to encounter. Why would you assume anyone here is interested in being a dictator? [23:22:18] Q: no, *individuals* like me made Python. [23:23:08] Including me, I might note, though my contribution to the core so far is mainly bugfixes. [23:23:14] _jpl_: I was actually taking about python, not PEAK. I odn;t think anyone here is a dictator, except by circumstance - if you get no feedback, you have ot make a decision. [23:23:57] I wasn't aware that dictatorship was a bad thing... Guido is known as the BDFL, after all. [23:24:15] <_jpl_> Sure, but a project leader should be free to decide whether the feedback is actually worth considering or not. [23:24:26] Ok, like I said - forget I brought it up. It's your right to make what you want to make, to support your living, and to write code the way you want to write it. I appreciate you listening to me ramble, and don't require you to agree. with me. :) [23:25:10] <_jpl_> Q, do you agree that a project needs vision, leadership, and quality standards? [23:25:16] Q: essentially, I'm saying that I've already bent over backwards to make PEAK accessible and non-monolithic... and don't know how to go any further without hampering *my* productivity. [23:26:36] _jpl_: I agree that a project should have those three, yes. But I lead by following, my vision comes from standing on the shoulders of my peers, and quality is something that comes from clean interfaces, rigorous testing, and strict isolation. [23:27:05] Q: I don't think I disagree with any of that. [23:28:01] Except maybe that I'd say that *design* should follow *requirements*, but not be dictated by them. [23:28:42] PEAK is shaped by feedback from Ty (my sometimes-collaborator on PEAK) who has a very "down-to-earth" attitude about these things and is pretty skeptical about frameworks... [23:29:09] He pretty much insists that I not make it hard to write "simple scripts" or use PEAK in the Python interpreter prompt. [23:29:30] <_jpl_> What I can't understand, Q, is why you assume things about the PEAK project and pje without knowing much about either. [23:30:33] I *like* skepticism, because it helps me make better designs. [23:30:44] (Which is why I'm still here...) [23:31:36] Unfortunately, I'm not seeing anything suggested here that's both specific enough to make a difference, that also makes a *positive* difference to the utility (as opposed to merely the popularity) of the package. [23:31:44] _jpl_: I'm here to learn. I fyou wish to shatter any of my preconcieved notions, then go ahead. [23:32:22] To be honest, it would be a *disaster* for PEAK to become very popular right now. [23:32:37] _jpl_: I live by my intuition. I've read pje's writing, I've seen a bunch of his code, I've talked to him here in IRC, and this is actually my fifth trip into here. [23:32:58] I think I've drawn a pretty clean sumamry. [23:32:58] <_jpl_> PEAK != Zope, and PEAK != Twisted, and from what I can tell the attitudes you may be associating with those groups/projects do not seem to be applicable to PEAK or pje. [23:33:01] You've talked to me before? [23:33:27] pje: Yes, but only about technical issues. I've bene folowing PEAK for ... I guess 8 months, I think. [23:33:33] Since the name change, anyway. [23:33:44] * pje nods. [23:33:59] That's almost a year actually. And it wasn't a name change, so much as a re-purposing. [23:34:08] * queuetue nods back. [23:34:21] That's when it got interesting. :) [23:34:32] <_jpl_> (You've been following PEAK for 8 months but you don't understand how naming and binding work?) [23:36:38] pje: Ok, I'd like to try to execute the Zconfig script you sent in the maillist email. I now have an EchoSchema.xml inside the dir src/twistedpeak. The EventDriven example uses a shortcut, but I don't have a shortcut defined for EchoSchema.xml-- how do I invoke it like you do for EventDriven:#!invoke peak EventDriven [23:36:59] #!invoke peak zconfig.schema:path/to/EchoSchema.xml [23:37:24] Or, in your sitewide PEAK_CONFIG, set up a peak.running.shortcuts entry, similar to the one for EventDriven in peak.ini. [23:38:25] Q: you never did mention (to my understanding, at least) why you thought PEAK wasn't Pythonic. [23:38:56] <_jpl_> I'm sorry if that sounded rude. I'm just surprised... from your questions it seems that you've just recently come across the PEAK web page. [23:39:45] <_jpl_> I was guessing that it could be because of custom metaclass-based things... [23:40:08] <_jpl_> Jolby, how did you solve the issue? [23:40:29] * pje yawns. [23:40:29] <_jpl_> Or was there an issue? [23:40:48] I've got to get some food, and head home. [23:41:58] pje: Thanks for all the help-- If I get it working I'll post to the list. If not, I'll still post to the list :-) [23:42:03] <_jpl_> Glad you dropped in, you've helped a lot today. [23:42:35] _jpl_: I don't think I have solved the import src="" issue yet... :-( [23:43:06] I'm a bit curious, too, as to why he's never raised any of these issues on the mailing list, e.g. when peak.config was split out from peak.naming. [23:43:09] Ah well. [23:43:16] I'm still pretty confused by the ZConfig machinery. [23:43:19] See y'all later. [23:43:20] <_jpl_> So you're trying to import EventDriven.xml from your own schema? [23:43:38] <_jpl_> See you later [23:44:52] <_jpl_> Do you have a copy of the ZConfig manual? [23:45:39] Hmmm-- Is it in the Zope3 src? If so, I do. I looked it over a few months ago, I suppose I should do it again... [23:46:02] <_jpl_> (not that it's a huge help :) [23:46:45] Its kind of frustrating, because what I want to do is so damn simple -- pass a param to the EchoProtocol component (Port=8080) [23:47:11] RE: ZConfig manual-- yeah, I found it to be close to useless when I looked at it last time... [23:47:15] <_jpl_> Could I see what you have so far? I'm about to try to do the same thing. [23:47:39] _jpl_: you bet, how should I send it? [23:50:01] <_jpl_> I think you could DCC it to me. Though I haven't used IRC much in the last 8 years or so, so I don't remember how that works. :) [23:50:12] I think its actually running-- but when I try to telnet to port 8080- I can't connect, so something is definitely wrong-- and no output is coming [23:51:02] _jpl_: I'm a total IRC newb (but I like using it), so I don't know how to DCC-- I'll play around w/ this chatzilla interface to see what I can do. [23:51:06] <_jpl_> Ah yes, DCC send... that should do it. [23:52:48] --> queuetue has joined #PEAK [23:54:02] _jpl_: Hmmm-- I don't see any menu item or context popup for DCC when I click on your roster item-- do I just type in DCC:/path to file in the textbox? [23:54:47] _jpl_: or maybe I should just send it to the maillist. [23:58:10] <_jpl_> Just e-mailed you my address. You could e-mail me a tarball. [23:58:38] _jpl_: sounds good.