[03:28:47] ** jack-e|away has joined us [03:28:47] ** _Maniac has joined us [07:21:53] jack-e|away is now known as jack-e [07:32:27] jack-e is now known as jack-e|working [07:39:12] ** vlado_ has joined us [08:32:07] ** vlado__ has joined us [09:24:42] jack-e|working is now known as jack-e [11:20:48] ** lex_ has joined us [15:44:26] ** _jpl_ has joined us [15:58:43] hey john :) [17:03:59] <_jpl_> Hi Ulrich [17:04:09] how are you ? [17:05:08] <_jpl_> Pretty good. Working too much, though. [17:05:19] :) same for me .. [17:05:42] i've read in the logs, that you've refactored your agent-system to use twisted.pb .. is that right ? [17:06:17] <_jpl_> That's right. [17:06:53] <_jpl_> I'm hoping to release open source the PEAK+Twisted layer I've written on top of pb. [17:07:24] that's what i wanted to ask .. i'm very interested and willing to help. [17:07:34] i think we have fairly similar requirements for that [17:07:57] i have played around with openSLP (Service Location Procotol) and implemented a Naming Context for peak and slp [17:08:23] <_jpl_> Great! I have preliminary approval to publish "bits" of code like this, so when I get a few minutes to breathe I'll clean things up a bit and put it on the wiki. [17:08:31] with this, the service configuration can be stored in a central (or distributed) database and looked up easily [17:08:37] <_jpl_> Nice! I was about to do exactly the same thing. [17:08:42] :) [17:08:48] <_jpl_> Or rather, I've been wanting to PEAKify SLP. [17:09:32] <_jpl_> I've been using SLP for a few months now, but have left it out of the new PB-based framework, mainly just due to time constraints. [17:10:02] http://cvs.net-labs.de/cgi-bin/viewcvs.cgi/libs/nll/src/nll/net/slp_ctx.py?rev=1.3&content-type=text/vnd.viewcvs-markup [17:10:34] it basically should not have any dependencies within nll .. so you might just be able to configure it and use it [17:10:57] http://cvs.net-labs.de/cgi-bin/viewcvs.cgi/libs/nll/src/nll/net/tests/demo_slp_ctx.py?rev=1.1&content-type=text/vnd.viewcvs-markup [17:11:03] shows how one can use it [17:12:55] i have no experience with slp .. so i was not sure at some places how to implement it best [17:13:21] <_jpl_> Excellent. I'll take a look at this when I go to put SLP back into the framework. Which might be soon now that we're about to do a small scale deployment, and all along I've wanted clients to dynamically discover their local server. [17:13:50] yes .. same for me [17:14:45] <_jpl_> It would be useful for service objects to be able to register/deregister themselves with slpd on the fly. Does your code allow for that? [17:15:01] yes .. you can e.g. [17:15:07] si1 = ServiceInfo('workflow.netlabs', '','10.0.0.1:6372', {'nllservice': 'True'}) [17:15:13] x = naming.lookup(ctx, 'service:') [17:15:22] x['service:workflow.netlabs:'] = si1 [17:15:30] and your service is registered with your server [17:16:26] <_jpl_> hmm, that looks a little awkward. [17:17:01] like i said .. i was not 100% shure how to make it fit into the peak.naming scheme properly [17:17:22] it mainly was about implementing the NamingContext [17:17:48] in JNDI you would [17:17:55] <_jpl_> Maybe using it with naming ought to be for client lookups rather than service advertisements? [17:18:02] x.bind('name', Reference) [17:18:08] i think [17:18:24] <_jpl_> Oh, that's interesting. [17:18:40] ? [17:18:41] <_jpl_> So in essence a service binds itself to a URL? [17:19:20] as far as i understood SLP-RFC the uri of a service is it's unique "key" [17:20:07] it has serviceType.concreteType as criteria as well as attributes [17:20:35] you can query the slp-server for services of "serviceType" or "(attrName=val)" queries [17:21:10] the lookup('service:') gets you to a NamingContext (WritableNamingContext) [17:21:28] <_jpl_> I was asking more about how you're using naming -- it looks like the service is telling the naming system that it exists at a particular URI which could then be used by clients to discover the service. [17:21:45] right [17:22:16] <_jpl_> That's quite nice. [17:22:25] <_jpl_> There ought to be a simpler way to do it in PEAK, though. [17:22:43] i look for "workflow.netlabs" services and get back a list of service-info objects that contain uri's (e.g. pb host/port) to contact [17:22:59] how could it be simpler than: [17:23:25] services = naming.lookup(ctx, 'service:workflow.netlabs') [17:23:45] for info in services: print info.sap [17:23:59] <_jpl_> 'ctx' is any object in the component tree? [17:23:59] to print the uri's for all services of type "workflow.netlabs" [17:24:14] some component that has acces to configuration-data [17:24:30] <_jpl_> Client-side is easy enough, it just looks a little awkward on the service side. [17:24:49] you could just rewrite it to: [17:25:00] x = naming.lookup(ctx, 'service') [17:25:02] x = naming.lookup(ctx, 'service:') [17:25:09] x.bind(ServiceInfo(...)) [17:25:33] that would be trivial [17:25:37] <_jpl_> Could you also do this on the client side: for s in self.lookupComponent('service:workflow.netlabs', default=[]): print s [17:25:52] shure [17:26:15] that's the nice thing with a naming-context :) [17:26:16] <_jpl_> That's really nice. [17:26:42] lookupComponent uses naming.lookup after it checks in local-attrs [17:27:07] <_jpl_> Maybe we could build this into the peak+twisted layer and release it as a separate oss project. [17:27:45] <_jpl_> The harcore Twisters won't like that it doesn't use TAPs, but I think most people would appreciate it. [17:27:49] yes .. i'ld be with you :) .. i urgently need some async network communication layer for my framework [17:27:54] <_jpl_> s/harcore/hardcore [17:28:03] :) [17:28:14] they don't like peak very much over there in [17:28:21] ... [17:28:43] <_jpl_> Well, some are ambivalent, but others (like dash) are outright hostile. [17:28:52] ok [17:29:11] i've converted my imap-backend to use twisted.protocols.imap4 now [17:29:20] (today) [17:29:41] it's much more comfortable that std-libs imaplib [17:31:53] <_jpl_> Once you get the hang of programming everything asynchronous, Twisted's 'deferred' approach is really powerful. [17:32:08] yeah .. i'm not yet 100% async :) [17:32:23] <_jpl_> I especially like chaining callbacks and even chaining deferreds. [17:33:01] i use it, because it makes building workflow-engines and gui-apps easier than with threads [17:33:06] <_jpl_> And deferredList is really handy if you want to send out requests to hundreds of PB clients and get back a list of responses. [17:33:53] <_jpl_> I'm not 100% async either, and can never be because many tasks will be long running (e.g. installing an operating system), but for those I can use deferToThread. [17:34:04] i implemented basically a reactor by myself about 2 years ago when i wrote the first version of my wf-engine for Transwarp [17:34:38] i lately realized that it was basically async (based on yield) [17:34:39] <_jpl_> Though at some point I want to convert the code with calls external commands to use spawnProcess (which forks). [17:35:57] <_jpl_> My co-worker wrote one of those while we were trying to do more complex async stuff with OSE. But now we've replaced OSE with PB and our "Junction" layer, so we can make full use of deferreds wherever we go. [17:36:35] i'm looking forward to see and play with your stuff [17:37:12] <_jpl_> I have a tiny little example which shows the basics. I'll e-mail you a tarball. [17:37:22] thanks :) [17:37:41] <_jpl_> It's not the code, but it shows you the basics of how to use it. [17:37:49] ok [17:38:07] <_jpl_> There's a bit of pub/sub in there, but that's currently experimental. [17:38:39] topic based event-channel with twisted.pb ? [17:38:47] or in-process [17:40:13] <_jpl_> It's both, though I don't use channels per se -- you could simulate pretty easily with dotted keys [17:40:30] <_jpl_> Wouldn't be hard to add real channels, though. [17:40:50] :)) .. i (was) working on that issue as well .. but have focused on gui/workflow lately again [17:41:27] a good network-infrastructe api would be very worthy for peak i think [17:42:25] <_jpl_> Yes, definitely. [17:42:41] <_jpl_> And since PEAK is already Twisted-friendly, PB seems like a good fit. [17:42:54] yes .. think so too [17:43:37] <_jpl_> The basic concept of Junction is that you instantiate one both on the client and server side, and you simply ask your local object things or register services with it. [17:44:44] * jack-e got your mail [17:44:44] <_jpl_> So you connect Junctions together and they talk to each other as needed. For example, to lookup a service you'd do something like: self.junction.lookupService('AService') [17:45:36] <_jpl_> Your junction will then look in its local service registry first for a match. If none is found it asks the junction its connected to. [17:46:15] <_jpl_> Currently you can only connect to one other junction, but I want to extend it to allow multiple outbound connections for star topologies. [17:46:43] <_jpl_> That'll get a little complicated, though, so I need to think it out before doing it. :) [17:46:52] hehe [17:47:15] <_jpl_> Basically I'm just trying to duplicate the features that OSE provides but with PEAK+Twisted underneath. [17:48:28] <_jpl_> If you're not familiar with it, take a look at OSE's docs (http://ose.sf.net/) to get an idea of what I'm aiming for. [17:48:56] i think i can basically follow you .. i'll take a look at the docs though [17:50:51] * jack-e needs some sleep now .. and get's out of office now ... [17:50:58] cu soon [17:51:30] your stuff looks promising :) [17:52:00] jack-e is now known as jack-e|away [17:52:07] <_jpl_> tscheuss! [18:00:27] * _Maniac needs to look a jack-e|away's twisted-imap stuff [19:20:04] ** vlado has joined us [19:40:12] * vlado is away, somewhere far beyond... (l!on)(p!off) [cRk/bx] [21:58:29] <_Maniac> for anyone that reads the logs i'd really love to see how peak and twisted integrate. Twisted has not a bad configuration and deployment system. Does peak compliment that ? [22:29:50] <_jpl_> I think it does. You can use PEAK's idea of executable config files to set things up with about as much convenience as Twisted's TAPs provide. [22:30:11] <_jpl_> Though no persistence, of course, which I wouldn't consider a shortcoming. [22:41:18] <_jpl_> PEAK's binding and config modules let you wire together the various parts of a Twisted application quite nicely. Using PEAK you get a lot more power and flexibility than you get with the standard Twisted mechanisms for registering and looking up components. [23:07:18] <_Maniac> see, i'm wanting to make an application which listens on many services [23:07:56] <_Maniac> everytime i look at the echo example on the peak wiki it just seems MORE complicated to me [23:07:58] <_Maniac> :) [23:18:47] <_jpl_> Yes, well, Joel put a little bit of everything into that demo. :) [23:26:04] <_Maniac> i'd like to see other examples [23:29:05] <_jpl_> Ok, like what? I just wrote a tiny little twisted.web resource today as part of a much larger, PB-based app. Maybe that would be a useful example. [23:30:24] <_jpl_> Could you give me some examples of what you're doing or trying to do? [23:34:30] <_Maniac> _jpl_, actually pretty simple stuff [23:34:54] <_Maniac> i have a weblog. i want to receive notifcation when a new comment is posted [23:35:18] <_Maniac> so the weblog sends an xmlrpc msg to a xmlrpc server, which in turn sends a jabber messge to me [23:35:22] <_Maniac> (that's the concept) [23:35:35] <_jpl_> The weblog uses twisted.web? [23:35:41] <_Maniac> no. [23:35:57] <_Maniac> but the xmlrpc server and jabber do [23:36:58] <_Maniac> basically it's a notifcation server. with configurable delivery options (at least that's the unimplmented concept) [23:38:42] <_jpl_> Ok, so how is it implemented now? [23:40:29] <_Maniac> very sketchily [23:40:40] <_Maniac> planning phase? [23:41:16] <_jpl_> Alright... so do you have your XMLRPC resource yet? [23:41:21] <_Maniac> basically a xmlrpcserver that echo's messages sent to it to jabber [23:41:37] <_Maniac> the receipient is hardwired (which is ok) [23:41:46] <_Maniac> _jpl_, yes [23:41:57] <_Maniac> class XRNoteReceiver(xmlrpc.XMLRPC): [23:41:58] <_Maniac> def __init__(self, service): [23:41:58] <_Maniac> self.service = service [23:41:58] <_Maniac> def xmlrpc_receive(self,note): [23:41:58] <_Maniac> print 'received', note [23:41:58] <_Maniac> self.service.sendmsg(note,"Maniac@jabber.vandorp.ca/Psi") [23:42:00] <_Maniac> return xmlrpc.Boolean(1) [23:42:35] <_Maniac> self.service a jabber client [23:45:56] <_jpl_> And where does that come from? [23:49:51] <_Maniac> which? [23:50:11] <_jpl_> self.service [23:50:30] <_jpl_> That which is passed into the constructor -- where does it come from? [23:50:48] <_jpl_> How do you instantiate XRNoteReceiver, and where do you setup your Site object? [23:50:51] <_Maniac> this is ugly :) : [23:50:52] <_Maniac> c = client.BasicAuthenticator(user='test',password='test',resource='test' \ [23:50:52] <_Maniac> , host = 'jabber.vandorp.ca') [23:50:52] <_Maniac> application=app.Application("jabber") [23:50:52] <_Maniac> svc = HempService(c, 'client', application) [23:51:16] <_Maniac> reactor.listenTCP(7080,server.Site(f)) [23:52:36] <_jpl_> Is the service necessary? [23:52:51] <_jpl_> Or maybe I should ask what it does first. [23:53:03] <_Maniac> self.service? [23:53:55] <_jpl_> No, HempService. [23:54:05] <_Maniac> oh, that's the jabber bot [23:54:37] <_Maniac> :) (can you tell i dont' realy know what i'm doing ) [23:54:40] <_jpl_> So what is 'f'? [23:54:57] <_Maniac> f = XRNoteReceiver(svc) [23:55:58] <_jpl_> Ok, first off I don't think you need the Application object. [23:56:35] <_jpl_> What does HempService (I'd rename it to JabberBot if that's all it does) do with 'c'? [23:57:35] <_Maniac> class HempService(app.ApplicationService): [23:57:35] <_Maniac> def __init__(self,c, *args, **kwargs): [23:57:35] <_Maniac> app.ApplicationService.__init__(self, *args, **kwargs) [23:57:35] <_Maniac> self.client = c [23:58:14] <_jpl_> That's all? [23:58:29] <_jpl_> So it's not implemented yet? [23:58:58] <_Maniac> def sendmsg(self,msg,recipient): [23:58:58] <_Maniac> message = dom.Element(("jabber:client", "message")) [23:58:58] <_Maniac> message["type"] = "chat" [23:58:58] <_Maniac> message["to"] = recipient [23:58:58] <_Maniac> message.addElement("body").addContent(msg) [23:58:59] <_Maniac> self.client.xmlstream.send(message.toXml()) [23:59:18] <_Maniac> stuff like that