[02:38:13] ** gpciceri has joined us [03:29:00] ** vlado_ has joined us [03:29:09] morning [04:02:53] ** gpciceri has joined us [05:10:22] ** gpciceri has joined us [05:13:01] jack-e|office is now known as jack-e [05:13:03] morning [05:13:16] ** gpciceri has left us [08:58:18] morning [09:49:52] i made my first foray into peak.storage last night jack-e ! [09:51:15] Maniac: how did you come along ? [09:51:30] i got it working, a little command line thing like bullitens [09:51:39] just for testing really but it wasn't so hard... [09:52:15] yeah .. i refactored our old backup-tapelibrary-controller script to use peak based on the bulletins app as well [10:09:17] once you do it yourself it makes it much clearer :) [10:09:28] and adding commands is simple [10:10:05] it takes a little more work to get things going, but once the infrastructure is there it's extendable [10:10:13] * jack-e is on the way to get a friend to the airport .. bbl [10:10:22] yeah .. that's what i wanted to add :) [10:10:33] jack-e is now known as jack-e|bbl [13:25:26] ** rdmurray has joined us [14:11:48] i have 281MB in swap, maybe i need more RAM... [14:14:04] rdmurray, hello [14:27:21] 'lo maniac [14:27:35] Maybe you have a memory leak :) [14:29:42] Heh. I'm not using any swap on my desktop machine. [14:31:55] on my 'desktop machine' i run UML (which runs apache and hosts my 'weblog') and VMWARE [14:31:59] plus all my 'desktop' apps [14:32:14] Ah, no wonder you use swap. [14:32:20] yup... [14:32:26] i only has 512 MB, i need more RAM [14:32:55] Since I do work for an ISP, as a nice benny I get to colo a server with them, so all my server stuff is elsewhere. [15:00:12] ah lucky duck then [15:11:18] yea [16:30:22] jack-e|bbl is now known as jack-e|away [17:07:00] bye ! [17:07:16] ** vlado has joined us [17:32:05] * vlado is away, somewhere far beyond... (l!on)(p!off) [cRk/bx] [17:56:57] Hmm. I wonder what facilities peak provides for writing command interpreters. [18:07:00] eh? [18:08:15] In other words, instead of writing 'hello to world', 'hello for vip', I want to be able to say 'hello', and then get a prompt where I issue 'to' and 'for' commands. I want to do this (in my real ap) so that I can have state saved between commands. [18:25:40] ah [18:25:48] i to that i have no idea... [18:25:58] i would *guess* not [18:26:38] ask pje on the mailing list [18:27:22] Yeah. I suspect it won't be too hard to do. In the meantime I've fallen down a different rabbit hole :) Now I'm trying to learn about unit tests the peak way. [18:30:10] just the unittest module isn't it? [18:30:25] Pretty much, I think. [18:30:44] and then a test runner of somesort [18:52:30] Rats. [18:52:44] I think I should go back and turn helloworld into a module from the get go. [18:53:01] Or maybe in chapter 2. [19:23:17] I keep getting deeper and deeper into this rabbit hole. [20:02:15] OK, I think I'm going to do this revision to put the helloworld files into a package. [20:04:21] ** exarkun has joined us [20:04:30] Hello! [20:04:47] 'lo [20:05:27] hi [20:05:33] wassup exarkun? [20:06:25] How does PEAK handle interface implementation declarations (or does that question not make any sense)? [20:07:15] um... [20:07:26] pyprotocols vs peak [20:07:34] You mean, how does a class say it or its instances implement an interface? [20:08:55] rdmurray: yea [20:09:19] protocols.advise [20:09:50] * exarkun looks it up [20:09:54] protocols.advise( [20:09:54] instancesProvide = [ICommandDispatcher] [20:09:54] ) [20:10:00] as an example (if i get the question) [20:10:12] You can also declare it right on the class, of you want to. [20:10:25] ah [20:10:27] stack introspection [20:10:36] or in a module. protocols.advise(modulesProvide = [ ]) [20:11:02] er, I mean, you can also declare it *independently* of the class if you want to. [20:11:56] http://peak.telecommunity.com/protocol_ref/node12.html [20:40:30] ** tsarna has joined us [20:41:07] hello [20:41:12] 'lo [20:42:54] so rdmurray, looks like your chapter 4 is going to embarrassme into fixing a bunch of stuff in n2 ;-) [20:43:05] heh. [20:43:20] Well, that always happens to me when I wrote docs for my own programs :) [20:43:38] I noticed the "help \foo" problem last night, actually... [20:43:56] Ah, good. I figured that was probably a bug, but I wasn't sure. [20:44:34] the "\describe foo" problem is that I hadn't figured what kind of interface there should be for introspecing a specific object in a database [20:45:09] so right now that code path is "pass # XXX" :) [20:45:20] Maybe it could give an informative message in the meantime. [20:45:28] yeah, I'm going to do that [20:45:32] coo [20:45:34] cool [20:45:54] though I think I've decided that there will just be an iterator over a sequence of cursors and/or strings [20:46:30] and "\describe foo" will just dump them out [20:47:03] that's about as specific as one can get and still support all the different databases and give useful information [20:48:33] so \describe table would get a sequence of cursors representing the definintinos of the columns of the table and print them out? [20:49:24] or indexes on the table, or the sql for the table, or... [20:49:35] k [20:49:41] well, replace or with "and/or" :) [20:49:51] :) [20:50:34] that interface will really only be useful for something that wants to show it to the user, like n2 [20:53:01] n2 is a very cool tool, by the way. [20:53:14] thanks :-) [20:53:53] oh, also \describe *should* work on an index or view or stored procedure too, depending on the database [20:54:22] it just occured to me that n2 might be the perfect example of what you were asking for earlier rdmurray [20:54:46] thus the not-very-specific "here's some data, print it however you like" interface :) [20:55:34] maniac: I check that already. I'm not sure what it does for sql, but for python it calls the python interactive interpreter class. Not quite what I wanted :) [20:55:55] the namespace part is the most generic "command shell"-ish part [20:56:03] but I'm not sure it's a good example of anything [20:56:32] n2 dates from before peak.commands, pyprotocols, etc [20:57:06] it's been updates some to take advantage of newer features, but it's not fully peak-ish [20:57:26] rdmurray, ok it was worth a try :) [20:57:32] actually, it was my "hello world" for a lot of things in peak :) [20:57:55] heh [20:59:09] it went through about 3 nearly-complete rewites before it was even commited initially [21:02:31] btw, thanks for the great tutorial! [21:02:47] You are welcome. I'm finding it a great way to learn peak. [21:03:09] I write down what I think I know, and Phillip tells me how it really works :) [21:03:16] :) [21:03:47] Writing it in the first place is fun. I'm more worried about what it will take to keep it up to date. [21:04:26] well, I hope to obsolete a bunch of stuff in chapter 4 soon, anyway :) [21:04:51] I think it will hold up well for the stuff already covered. [21:05:12] it will get more "dangerous" as you get into the web, security, etc packages [21:05:31] Well, when pje lands the Query stuff, the whole Storage discussion may become moot. [21:05:35] hey, i just started to use security a little :) [21:05:39] well, the higher-level storage stuff is that way too [21:05:58] the connections/cursors/transactions lower-level stuff should be pretty stable [21:06:40] tsarna, you use peak @ work? [21:06:57] yes.. I work w/ pje [21:08:14] we're just beginning to use peak, really. lots of other people in the community are way ahead of us! [21:08:32] huh. That seems odd :) [21:08:37] it's only very recently that we finally grafted peak into our main application [21:09:15] Can you say what kind of application that is? [21:09:20] tsarna, i knew you worked w/pje, just wasnt' sure if you guys used it yet :) [21:09:51] * Maniac nitpicks on chapter 4 [21:10:14] ideally we were kind of hoping to ditch our codebase and start over w/ peak, but now we can at least get some benefit from it [21:10:42] * Maniac decides to not nitpick as pje does it too [21:10:47] maniac: note that the second half of chapter 4 (the non n2 stuff) is in terrible shape. The code is broken, for one thing. [21:11:04] and vice versa... now that we're using it, we can invest more in peak, like the pre-forking process manager stuff, performance timers, etc [21:11:19] tsarna, cool [21:11:33] * Maniac wants performance timers [21:11:37] our main application is a request-tracking system (trouble ticketing basically ,but used both by customers and internally. [21:11:55] Also note that I'm currently editing chapter three, so don't go correcting any typos there :) [21:12:05] closed source as i understand it ? [21:12:14] yes [21:12:29] you wouldn't want to see the source anyway ;-) [21:13:06] Request tracking systems must be one of the three or four most-often-reimplemented applications. Each business seems to have enough special requirements that generics just won't do a lot of the time. [21:13:08] ha! you *definately* wouldn't want to see the source of my projects :) [21:13:27] it's not a good example of very much, except maybe "a pile of problems that taught us the lessons that led to peak" [21:13:30] rdmurray, which is why i use roundup and customise as necessary [21:14:04] ours has all sorts of interesting and unique special requirements [21:14:10] tsarna, fwiw i like peak in alot of ways [21:14:26] it is very usefull [21:14:29] actually the *idea* of the application is pretty good. lots of good reasearch went into the design of the system [21:15:07] rdmurray, does the GroupDM work? (chapter 4) [21:15:25] we want to keep and expand on the ideas. it's just the code is not in very good shape. It dates back to being a Principia (Zope, before it was called Zope) application [21:16:12] ah] [21:16:35] you could say it had a poor childhood, and this warped it for life :) [21:16:53] maniac: sort of. But it has bugs. Like save actually adds a new record, which is not what it is supposed to do :) [21:17:18] ** exarkun has left us [21:17:46] maniac: and it is inneficient. __contains__ needs to be rewritten using the get paradigm pje introduced in chapter 3. [21:18:20] rdmurray: re chapter 4, you refer to a bunch of things as "namespaces" that aren't really [21:18:52] as an example, where you mention that if there was an IN2Interactor adaptor for imap you could be off and exploring that namespace [21:19:14] but if it was actually a namespace (in the peak sense), the existing interactor would already work [21:19:45] Well, I was using namespace in the generic sense, but that really doesn't apply to IMAP, I suppose. [21:19:52] IMAP was probably a bad choice of example :) [21:20:07] the name "namespace navigator" is probably somewhat misleading in that sense [21:20:09] well, no, I take that back. IMAP has folders. That's why I picked it. [21:20:31] since post-pyprotocols, namespaces aren't "special" to it anymore, it just one of the provided interactors [21:20:33] i just noticed the __contains__ i was wondering how you were doing 'in self.Groups' [21:21:19] maniac: that's how. But doing 'in' on a DM is dangerous if you haven't implemented __contains__. It'll hang your program mysteriously. [21:22:11] OK, so what is a namespace in the peak sense? [21:22:38] something that implements at least IBasicContext [21:22:38] namespace to me is just something that contains a set of named objects, where 'object' is ill defined :) [21:23:06] well, there are property namespaces too. but if not qualified, I assume peak.naming namespaces [21:23:15] and why dont' you want to use a where clause? [21:23:29] OK, so I just need to reword that paragraph. [21:23:41] The idea of writing an Interactor for IMAP is valid, right? [21:23:46] sure [21:24:41] oh, you want to test if there's a result [21:24:41] Maniac: because I didn't bother to test to see if 'len' works on a cursor. [21:24:57] or, maybe you could use jabber or something as an example [21:24:58] I didn't realize I could test it for 'false' to see if it is emtpy. [21:25:16] i've got to use a __contains__ on my DM which is why i'm askin :) [21:25:47] you can interact with anything with n2, it needn't be "namespacey" [21:26:08] you just have to have a namespace that gets you to it in the first place :) [21:26:49] tsarna: I don't think I understand it well enough to know what that means. [21:27:47] well, a SQL connection for example isn't a namespace even in a broad sense [21:28:03] maniac: is there some reason you can't use PJE's 'get' method? It's more efficient unless you've got a big database and a good index. [21:28:21] the connection isn't, but the database is. [21:28:38] (in a broad sense) [21:28:59] right, but you don't "cd" around in the database for example :) [21:29:25] but, you can get there *through* a namespace (like a SQL url directly, or otherwise). then you can interact with it [21:30:17] part of the issue with illustrating this is that we don't have any really interesting peak.naming contexts thatw e can release right now [21:30:23] Well, you don't normally, no. But I could imagine doing so :). cd ; insert (rowdata); etc [21:31:07] So I think I have no clue what a namespace is in the peak sense. I'll have to look it up later. [21:31:30] if we had a full ldap context implementation, you could cd around, ls, etc. [21:32:13] right now the only way you can use the actual namespace interactor part of n2 (that's the part with the [n2] prompt) is to invoke n2 without an argument [21:32:28] so when you use n2 to connect to a database you are sort of jumping right past the "namespace" part of n2 and into the sql interactor? [21:32:50] that puts you in the default context, which is empty and read-only, so it's a pretty lame example and not very educational :) [21:33:15] rdmuray: exactly! [21:33:26] Hmm. I think this is going to be difficult to explain properly ;) [21:33:41] well, what you have now is pretty good [21:33:55] Yeah, but I've got to straighten out the peak temrinology. [21:34:33] maybe just some slight rewording, and then a later chapter can go in-depth on the naming system [21:35:17] you can do the "remember when we saw this before? OK, now here's how it *really* works" trick in a later chapter [21:35:25] Yeah. Although I'm not sure where that might fit in to the hello world scheme :) [21:35:42] yeah [21:36:03] well, maybe pulling some sort of data from a corporate LDAP directory or something [21:36:10] Unless I develop a namespace representing the hierarchy of things to say hello to.... [21:36:16] Oh, LDAP. Yeah, that could work. [21:36:21] but you can't use LDAP as a naming context yet anyway [21:36:34] we only did the raw connection part so far [21:36:35] :( [21:37:16] so, don't worry about explaining it until there's something to explain :) [21:37:51] :) [21:38:54] anyway, I'm off to send some "quality time" here at home :) [21:39:09] just wanted to pop in and encourage you on the documentation [21:39:09] rdmurray, as i understand it the get method is iterating over the entire contents of the text file? [21:39:27] thanks [21:39:27] especially the parts that prompt me to fix things :) [21:39:29] tsarna, nice to meet you (in the IRC sense) [21:39:37] yes, likewise [21:39:40] yes, nice to talk to you folks too [21:39:53] I'll try to drop in more often... this is my first time [21:40:12] my irc client is here 24x7 [21:40:22] maniac: in the text file example, the first get is going to load the database. In the sql example, it only loads the single object needed. [21:40:32] anyway, catch you later :) [21:41:56] I *think* that in a more complex database you could return partial state, but I'd like to check that with pje. [21:42:01] mm, it's in the sql example already /me looks [21:42:17] nope. [21:42:29] I have't fixed the example yet :) [21:42:52] Basically, the sql get will do what the _load does now, but call preloadstate. [21:43:33] And of course it will do the cache check like the text file one. [21:44:02] ahahahahahahaahaha [21:44:17] i have a get method already [21:44:21] * Maniac kicks himself [21:44:32] def get(self, oid, default = None): [21:44:33] if oid in self.cache: [21:44:33] return self.cache[oid] [21:44:35] else: [21:44:38] return self.preloadState(oid) [21:44:43] return default [21:45:23] so i just need to change my _new(ob) method to check self.get(oid) to not try and insert a duplicate and get an ugly traceback. [21:45:38] Why call preloadstate if you aren't passing it any state? [21:46:23] Given that 'get' method, it looks like all objects will exist. [21:46:28] er, all oids. [21:48:05] um [21:48:07] ? [21:50:26] Actually, I think what will happen is: if the object is in cache, return it. If the object is not in cache, create it (empty), then return the default. So a second get call on the same oid should return the (empty) object. But if the cache is emptied, objects that exist will appear to not exist.... [21:50:38] But I could be misunderstanding preloadState. [21:50:59] Assuming what you pasted is the whole of your get code. [21:53:50] oh it is [21:54:22] If your get code isn't doing a database access if the object isn't in cache, then it isn't doing anything useful :) [21:57:44] Wait, I just looked again. That last return default will never get executed. The return preloadState will return an (empty) object if the oid is not in cache. [22:01:03] (i stole it from somewhere and haven't tested it yet :) ) [22:01:40] no wait... [22:04:16] class ShowUser(BottoolsApp, AbstractCommand): [22:04:17] useage = """Usage: bottools showuser login """ [22:04:17] def _run(self): [22:04:17] if len(self.argv) < 2: [22:04:19] raise InvocationError("missing argument(s)") [22:04:21] storage.beginTransaction(self) [22:04:24] user = self.Users.get(self.argv[1]) [22:04:27] print user.login, user.password [22:04:29] user = self.Users.get(self.argv[1]) [22:04:32] print user.login, user.password [22:04:34] storage.commitTransaction(self) [22:04:51] def get(self, oid, default = None): [22:04:52] if oid in self.cache: [22:04:54] print 'returning from cache' [22:04:56] return self.cache[oid] [22:05:02] else: [22:05:04] print 'accessing database' [22:05:07] return self.preloadState(oid) [22:05:38] if i call 'get' twice the first time i get 'accessing from database' the second time i get 'accessing from cache' [22:05:46] so it seems to work to me? [22:05:56] There, 2 and 3 are changed to use a helloworld package. So I'm up out of *that* part of the rabbit hole. [22:06:17] both times i get the correct answer [22:06:21] Nope. Like I said, the first time it simply returns an empty object. The second time it returns that empty object from cache. [22:06:42] But, that get method on that class wouldn't seem to have anything to do with your data managers? [22:07:00] Are you sure you don't have similar (but different) methods in the real DM class/ [22:07:01] ? [22:07:32] Or were you just pasting from two different classes there? [22:08:13] two differnt classes first one was from my AbstractCommand the second from the DM [22:08:14] preloadState doesn't access the database. it allos you to *provide* state to the object being returned. But your call does not do so, so the object should be empty. [22:08:25] s/allos/allows/ [22:13:07] I'll work on chapter 4 and see if I can get you a working example. You are using sql for your database? [22:13:26] yes [22:13:41] my code works, but you say it shoudnt :( [22:13:49] ah wait! [22:14:13] no it works... [22:14:43] The get gets data out of the database? [22:14:48] yup [22:14:55] Hmm. Maybe that answers my question about providing partial state. [22:15:11] I bet what is happening is that when you access an attribute on that object, _load gets called. [22:15:39] But, your get is still broken, because it will return an object (*not* None) whether or not that given oid exists in your sql database. [22:15:48] i'm trying to trace that in peak source right now, it looks like __getitem__ is called which calls _retrieve() [22:16:03] yes [22:16:26] but if the get fails i want to return None and not actually insert. i await your example :) [22:18:25] ok, easy enough, i just called my command with an entry not in the database [22:19:09] it does call _load(oid, ob) [22:19:30] and i get exceptions.TooFewResults [22:39:51] Hmm. Looks like I was wrong. You can't truth test a cursor. [22:39:56] Guess I'd better check the interface. [22:41:37] Odd. There really doesn't seem to be any way to test the cursor except to iterate it. [22:44:08] i would adopt the convention that all SQL be UPPERCASE [22:44:38] Good point. [22:44:55] ( i just read that) [22:44:56] I hate doing that in my own code, but it would be good for the examples. [22:45:24] i think it makes it more readable in that someone is not going to mix up sql keywords and table names etc. [22:55:11] I've got the example working but I haven't updated the text yet. Want me to past it here? [22:56:23] Wait, I'll just update the final GroupDM example, and then I'll work on revising the text [22:56:36] sure ! [22:57:34] OK, GroupDM is updated with a working example. [22:57:52] Now lets see if I can straighten out this mess :) [22:59:36] that looks good [23:20:30] ok i changed my stuff to kinda match [23:20:40] i still dont' like getting a traceback if someone doesn't exist [23:21:08] Well, this way you shouldn't. [23:22:14] You can say, x = myDM.get(someone); if not x: [23:24:05] oh yeah, i was doing user = myDM[oid] [23:25:13] cool works fantastico! [23:27:22] Great :) [23:30:52] ha this peak stuff is way cool [23:31:03] yeah [23:32:38] now i just need to decide what to work on next :) [23:39:04] Hmm. PJE said my first example sql code was broken, but it works. I wonder why. [23:46:37] pje said when?