[00:04:39] Hmm. there's this intriging looking 'mdl_syntax' attribute, but the docstring doesn't say anything about what you assign to it. [00:08:46] Hmm. Well later for that. An undocmented parsing syntax is not something I want to tease out right now :) [00:13:33] <_jpl_> Parsing is done by peak.util.fmtparse, for what it's worth. [00:13:48] Yeah, I found that and started reading it, which is what made me say "later for that". [00:13:55] <_jpl_> Me too. :) [00:21:52] Hmm. I would expect the constructor of an Element type to call mdl_fromString if passed a string. But I can't figure out if it does. [00:22:41] <_jpl_> No, mdl_fromString is a factory object. As such it will create a new Element object. [00:24:05] <_jpl_> er, factory method. [00:24:05] So DM code would call it explicitly? [00:24:31] like DateTime.mdl_fromString(valuefromdb) to get a DateTime to return in the dict in _load? [00:24:39] <_jpl_> It could, yes, if your DM was loading from a source that stored the Element's data in a form that's parseable through the Element's mdl_syntax definition. [00:26:00] given what you say mdl_fromString is, I can't make any sense out of the one in the DateTime class in bulletins [00:29:01] <_jpl_> Hmm. It's probably just not implemented yet. [00:29:12] Probably. [01:08:04] <_jpl_> Gotta run. Hope you get somewhere with the program. [01:08:18] Well, I've written code, but I'm nowhere near being able to test. [01:08:26] 'course, I should have written the unit tests first... [01:08:28] :) [01:08:44] thanks for all your help [01:09:22] <_jpl_> Good night. [01:09:31] <_jpl_> (you're quite welcome) [01:10:00] ** _jpl_ has left us [03:20:50] jack-e|away is now known as jack-e [03:20:54] morning [04:42:23] ** vlado_ has joined us [04:42:32] morning [05:11:11] ** vlado_ has joined us [09:58:20] re [10:37:29] * Maniac pets peak-logbot [10:41:35] * jack-e has finished the first draft of a peak-web styledSkin using XSLT [10:41:50] (will place it onto peakplace when available and cleaned up a bit) [10:43:06] xslt is cool [10:43:18] * Maniac 's 'weblog' runs on xml and xsl-t [10:44:38] within the resources.ini you can specify which templates/resources should be styled .. they get then adapted to an IXMLDocument that is transformed using libxslt [10:45:41] you are working on this for fun or profit? [10:48:59] fun currently, but if i ever have to write web-apps, i wanna go this path i think [10:49:57] we want to relaunch www.fet.org someday and we're searching for a platform that is not as polluted as zope2/3 and not so primitive as postnuke [10:50:40] currently it runs with php4 and a selfmade CMS [10:50:57] oh god postnuke will make you lose your mind [11:16:57] 2 friends of mine are using it for their development ;-) [11:19:48] ** rdmurray has joined us [11:26:56] bye [11:29:24] well, given that php gives me a headache :) [11:29:30] (for anything complex) [11:30:01] * Maniac hugs python [11:38:18] sloccount PEAK/src/peak: [11:38:25] l Physical Source Lines of Code (SLOC) = 24,204 [11:38:25] Development Effort Estimate, Person-Years (Person-Months) = 5.68 (68.12) [11:38:25] (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05)) [11:38:25] Schedule Estimate, Years (Months) = 1.04 (12.43) [11:38:25] (Basic COCOMO model, Months = 2.5 * (person-months**0.38)) [11:38:28] Estimated Average Number of Developers (Effort/Schedule) = 5.48 [11:38:30] Total Estimated Cost to Develop = $ 766,872 [11:38:33] (average salary = $56,286/year, overhead = 2.40). [11:38:35] SLOCCount is Open Source Software/Free Software, licensed under the FSF GPL. [11:38:38] Please credit this data as "generated using David A. Wheeler's 'SLOCCount'." [11:40:43] jack-e, how does one checkout nll again? (/me is setting up another machine) [11:41:10] cvs -d:pserver:anoymous@cvs.net-labs.de:/data/cvs login [11:41:14] pwd: anonymous [11:41:18] cvs -d:pserver:anoymous@cvs.net-labs.de:/data/cvs co libs/nll [11:56:52] Hmm. Is the invoke command not installed by default? [11:58:36] sloccount nll/ [11:58:39] Total Physical Source Lines of Code (SLOC) = 10,323 [11:58:39] Development Effort Estimate, Person-Years (Person-Months) = 2.32 (27.84) [11:58:39] (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05)) [11:58:39] Schedule Estimate, Years (Months) = 0.74 (8.85) [11:58:39] (Basic COCOMO model, Months = 2.5 * (person-months**0.38)) [11:58:41] Estimated Average Number of Developers (Effort/Schedule) = 3.15 [11:58:44] Total Estimated Cost to Develop = $ 313,428 [11:58:46] (average salary = $56,286/year, overhead = 2.40). [11:58:49] SLOCCount is Open Source Software/Free Software, licensed under the FSF GPL. [11:58:52] Please credit this data as "generated using David A. Wheeler's 'SLOCCount'." [11:58:59] jack-e, you should be rich! [11:59:25] very high overhead calculation i think.... [12:02:08] my jabberbot weighs in at $4,000 ! [12:02:45] hehe [12:03:36] Well, you know, if you'd done it in a formal development organization and did all the specs and stuff, it probably *would* have taken $4000 worth of time :) [12:04:25] rdmurray: are you coming along ? [12:04:32] coming along? [12:04:41] yeah, you cana adjust the overhead calculation and stuff to take into account the lack of documention etc. that it appears sloccount tries to take into account [12:04:42] with your DM's and stuff [12:05:07] Oh. Yeah, I finished writing my storage.py. But I haven't tried to test it yet :) [12:05:22] Now I'm off trying to figure out the minimal peak "hello world" program. [12:05:37] as shell-command ? [12:05:40] It's not exactly obvious. [12:05:41] Yeah. [12:05:49] there are several ways [12:06:44] Well, I'd like two examples, the 'executable ini file' way, and the 'python module' way. [12:07:01] the peak XXX methods like runIni (interpret an inifile that supplys an PropertyName('peak.running.app')) [12:07:09] i like them too [12:08:02] I don't seem to have an 'invoke', but pje's examples all seem to use it. [12:08:17] Should I just call peak directly from the #!? [12:08:28] depends on your platform [12:08:36] you can build it yourself the sources are there [12:08:54] scripts/invoke.c [12:09:07] gcc -o invoke invoke.c [12:09:23] Which platforms is it needed on? [12:09:38] i don't know exactly .. but i installed it [12:09:45] Hmm. [12:09:51] you need invoke only if you dont' use the peak command right? [12:09:52] i'm on debian linux and both work for me [12:09:59] you can allways [12:10:11] peak runIni /path/to/inifile [12:10:12] for example [12:10:41] it's only for zconfig/ini-files that are executable. the handling of the #! seems to differ [12:11:34] i often create my own "peak" script as main-app that has the python-path and ini-files set to my need e.g. [12:11:59] http://cvs.net-labs.de/cgi-bin/viewcvs.cgi/apps/afsbackup/afsbackup?rev=1.1&content-type=text/vnd.viewcvs-markup [12:12:31] one could do this with environment-variable too .. but that's not convinient for tools [12:12:47] OK, so that would be my "python module" example... [12:14:11] what platform are you on ? [12:15:31] freebsd [12:17:19] you can probably use #! /path/to/python2.3 peak runIni in your config-file [12:17:54] (peak incl. path, or without python2.3) [12:19:07] Hmm. With either that or with invoke I'm getting "no such file or directory; ./helloworld" [12:19:11] That's weird. [12:19:35] It looks like shell error, but the error changes if I mess with the #! line in the helloworld file. [12:19:52] what if you do a: peak runIni helloworld [12:20:21] That does what I'd expect (gives me an error about running, since I didn't actually put anything meaningful in the ini file) [12:21:05] that's probably what invoke is for .. there was a discussion on this issue about 2-3 month ago [12:22:06] Hmm. Now I can't get the error messages to change. I must be messing something obvious up and I'm just not seeing it. [12:25:30] Huh. That's weird. Invoke doesn't seem to work unless I call it using the full path name. [12:26:05] Same for peak. [12:26:06] Weird. [12:27:04] Hmm. Maybe that's a zsh thing. [12:27:33] i don't know .. we just consolidated all our server to debian [12:32:28] What Interface does a peak.running.app have to provide? [12:33:54] Ah, I think I found it. [12:46:46] ** _jpl_ has joined us [12:47:40] Is there any ini configuration directive that will add directories to the PYTHONPATH? [12:47:52] nope [12:48:07] Hmm. That seems like an obvious oversight :) [12:48:13] most of the parts of an ini-file are also lazy-loaded (except section-parsers etc) [12:48:21] export PYTHONPATH=... [12:48:22] * _jpl_ waves [12:48:29] hey john .) [12:49:12] <_jpl_> How's it going? [12:49:42] Yeah, but if I'm writing an ap invoked through an exacutable ini file, it seems natural to want to be able to specify in the ini file where all the auxilary files for the program are. Rather than having to put them in some directory already on the PYTHONPATH, or provide a wrapper script to alter PYTHONPATH (which would seem to defeat the purpose of having executable ini files). [12:49:55] right [12:50:07] <_jpl_> rd: pje seems to have a preference for dealing with the Python path from outside of Python. [12:50:33] jpl: busy clearing my work before i head out for the weekend :) [12:51:13] <_jpl_> rd: True enough. I usually just use a shell script for reasons of that sort. [12:52:18] What does your typical shell script look like? [12:53:37] <_jpl_> Usually just a PYTHONPATH line and a call to the 'peak' script, something like: /usr/bin/peak runIni ${1:-my-config.ini} [12:53:39] Heh. I got my simple helloworld to work, at least :) [12:53:54] k [12:54:49] <_jpl_> Still nothing from tigris.org... the latest Junction code is all nicely package-ified and in tigris CVS, waiting to be poked at. [13:02:40] i found the commit messages when i searched for peak on tigris but when i tried to read them i couldn't :) [13:27:41] nice Weekend guys .. [13:27:47] jack-e is now known as jack-e|away [13:43:27] c-u [14:02:04] yay pje talks about peak.security [14:35:18] ** _jpl_ has joined us [14:42:36] ... my head hurts :) [14:44:59] You banging your head against php again? :) [14:45:33] Oh, I missed the context. If you are reading a pje message, no wonder your head hurts. [15:17:41] yes, reading very long answer to my question :) [15:17:58] i work by example, the simpler the better. [15:18:21] somewhere mixed in that 5000 word email there's a simple example buried. There just must be..... [15:18:30] although i think i get the theory now... [15:20:54] Yeah, simple examples are good :) [15:23:42] Boy, I like either of the structured text's I've worked with better than the thing this wiki uses. [15:37:02] this wiki? [15:37:11] peak's wiki [15:37:18] telecommunity [15:37:38] ah moinmoin [15:40:26] * Maniac sees rdmurray's "hello world" [15:40:39] * Maniac wishes he had somethign like that a couple months ago :) [15:40:44] Heh. [15:40:51] Me, too :) [15:41:15] http://randomthoughts.vandorp.ca/moinmoin/moin.cgi/SimplePeakRunnableScript [15:54:56] Gak, I hate proofreading wikis :) [15:55:07] all done, though. If you see any errors, let me know. [15:58:55] i read it 10 minutes or so ago [15:59:01] looked good to me [15:59:38] _jpl_ is also trying to setup peakplace.tigris.org to have kindof a repository to share peak stuff too [15:59:52] * Maniac is trying to figure out peak.security [18:08:19] ** hazmat has joined us [18:26:59] <_jpl_> Still no word from tigris.org. Junction is there and waiting to be played with. [18:27:41] <_jpl_> I supposed I could be the source tarball on the wiki in the meantime. [18:30:03] <_jpl_> Darryl, once you figure out peak.security you can help add security to Junction. :) [18:32:08] whta is junction? [18:36:43] <_jpl_> Hooray! _Enterprise Integration Patterns_ is out. Been waiting impatiently for that one for a while... [18:37:12] <_jpl_> Junction is a simple message-oriented-middleware tool I've written in PEAK and Twisted. [18:37:36] <_jpl_> Simple currently, though hopefully it'll get more powerful over time. [18:37:57] <_jpl_> It currently does basic publish/subscribe and request/respond type of operations. [18:40:06] ** pje has joined us [18:40:19] rdmurray, *beautiful* hello world page! [18:40:22] Thanks!!! [18:43:27] <_jpl_> Hi Phillip. [18:44:31] <_jpl_> Where's the hello world page? [18:45:25] DevCenter/HelloWorld [18:45:33] * pje is making some minor edits right now [18:45:47] <_jpl_> Ah, that makes sense. [18:46:07] Mainly, adding a bit more help for Windows folks, shortening to 'commands.*' instead of 'running.commands' (since yesterday you can access it as an API framework directly.) [18:46:19] I'm also clarifying that binding.Obtain is a descriptor, like 'property'. [18:47:27] <_jpl_> Hey, that *is* a nice example. [18:48:21] <_jpl_> commands.* is a welcome addition. [18:50:27] Yeah, I noticed I kept using it all over the darn place. [18:51:01] So, since we now have 'peak.core' for the minimalists, I felt free to expand 'peak.api' with it. [18:51:14] <_jpl_> Phillip, do you think it'd be even possible to get data into a binding asynchronously? That is, doing a binding.Obtain() on something that wouldn't return a value right away. [18:51:34] <_jpl_> Oh, there's a peak.core? [18:53:16] As of yesterday, there is. (peak.core) [18:53:21] As for the obtain, um, no. [18:53:30] What would it return? [18:53:50] I mean, either it has to wait for the value to come back, or it has to return something *else* as an interim value. [18:54:01] Like a Deferred or something. [18:54:28] <_jpl_> Right, the context I had in mind would be with Twisted and Deferreds. [18:55:18] <_jpl_> The idea would be to make it possible to bind to a value obtained from a server via a Twisted PB method call. [19:00:39] howdy pje [19:01:12] Hi Maniac. [19:01:19] JPL, that wouldn't really work, though. [19:01:21] i wouldnt say i 'figured out' peak.security but i got my minimal example to work (danka pje) [19:01:35] You'd need to do it the other way around: make the deferred callback set the binding. [19:01:55] Or, just use deferreds as they were intended to be used. [19:10:57] those 'howtos' are getting to be pretty helpfull :) [19:11:48] should i add "ABriefPeakSecurityExample" with how to burn down one's building? [19:12:00] (based on the mailing list postings) [19:22:15] Please do! [19:23:07] I've been worrying about writing howtos; I'm not that good at it, but I know we really need lots of them. [19:23:28] But the community has been producing great ones. [19:23:45] <_jpl_> Phillip, what do you mean by using deferreds as they were intended? I don't really want a deferred to be returned into a binding, I want the result of the deferred to end up in the binding somehow, possibly through a custom URL of some sort. [19:24:19] JPL: I mean, a deferred is designed to execute something when it's done. [19:24:41] So, if you want the deferred to put a value in an attribute, set a callback to set the attribute. [19:25:10] You can't make something asynchronous into something synchronous, without using a thread. [19:25:30] Or the Flow package. [19:25:47] <_jpl_> Ah yes, this could be a job for Flow. [19:25:54] But not in a binding. [19:26:10] The binding can't be a generator. [19:27:23] <_jpl_> I know, I was just trying to imagine a way to obtain binding values from a remote source using the same technique used for getting local data. [19:27:42] Oh. [19:27:44] <_jpl_> Sort of like distributed properties. [19:28:16] Thing is, you can't program an asynchronous model asynchronously. [19:28:23] Er, synchronously, I mean. [19:28:50] The only way to do it is with threads or some sort of simulation thereof (such as a state machine). [19:29:22] Thus, callbacks and flows. [19:29:41] By the way, there was some question yesterday about setting _p_jar on Elements... [19:30:10] You don't need to do that. You use EntityDM.newItem(AnElementClass) to create a new instance, so it knows its jar that way. [19:30:53] IOW, you should never instantiate an element class directly, but always go through the DM that will be expected to manage it. [19:31:40] Technically, you can get away with not doing it if the object is referenced from another object for that DM; it'll effectively get detected and grabbed up when you write stuff out. [19:31:50] But it's better to be explicit from the beginning. [19:32:11] (So that the new object will have a context/parent component from the beginning.) [19:32:53] Anyway, I think that was the only question that hadn't already been answered by you or jack-e. [19:33:01] If not, I'm sure somebody can correct me. :) [19:33:16] <_jpl_> There are always lots of unanswered questions. :) [19:37:30] <_jpl_> But none that come to mind at the moment. I'm sure I'll think of a few as soon as you sign off. [19:37:53] * pje laughs [19:37:55] Of course. [19:39:42] <_jpl_> A peak.web howto would be pretty handy... [19:42:08] Maybe jack-e or Radek will write one. :) [19:42:24] Once they finish telling me all the bugs, of course. :) [19:42:52] Seriously, there's still a pretty big chunk of TODO items for peak.web in 0.5a3, let alone a4. [19:43:06] <_jpl_> Is it possible to combine peak.web with Twisted, or would you need to use something like FastCGI? [19:43:10] I hope to release a3 this month though. [19:43:40] JPL: peak.web implements IRerunnableCGI. If you can create a Twisted container that invokes that interface, you're set to go. [19:43:50] <_jpl_> Have you considered just releasing PEAK's "core" (binding/naming/config, maybe running) as 0.5? [19:44:15] And, if you'd rather use Webware or Zope or something with one of the PEAK runners, you can always write an IRerunnableCGI adapter for them. :) [19:44:30] Really, IRerunnableCGI is potentially a "lingua franca" for Python web frameworks. [19:44:56] JPL: there's not enough documentation, even for the core frameworks for a final anything. [19:45:25] See how many questions we get, and publicity even now (e.g. Daily Python-URL mention)? [19:45:49] No, documentation is a must before I'll give a version label that has even 'beta' in it. [19:46:34] Anyway, on the Twisted thing, you'll probably need to use threads, since IRerunnableCGI is a synchronous interface. [19:46:51] In principle, it should be doable, since Zope does pretty much the same thing in their asynchronous web server. [19:46:56] <_jpl_> Sure, but then writing docs for the core would then become more the focus for 0.5 than adding lots more features. [19:47:03] * pje grins [19:47:17] Well, but in lots of cases I need the features for my job. [19:47:53] darn jobs ! [19:48:12] And sure, the documentation is important too, but even in my work people say docs are important, but that doesn't mean they give you time to write any. :) [19:48:48] Worst of all, my estimation is that docs take me many times longer to write than the code that the docs are documenting. [19:48:54] :( [19:48:57] <_jpl_> There's no doubt they new features will be useful and important, but it sure would be nice to have a release of all the really great features that are already in PEAK. [19:48:58] * Maniac checks the TODO for a3 [19:49:23] <_jpl_> s/they new/the new/ [19:49:49] I know, I know. [19:50:04] But meanwhile, the demand for features from the hungry mob has increased too. :) [19:50:15] oo, 0.5 Alpha4 option parsing for peak.running [19:50:22] See? :) [19:50:23] <_jpl_> My only concern is that there are lots of loose ends at the moment, and with you being almost the only PEAK developer it could be another year before we see 0.5. :) [19:50:37] True. [19:50:55] But the best thing that people can do to help right now is writing how-tos. [19:51:08] I can edit and correct how-tos *much* faster than I can write them. [19:51:19] <_jpl_> PEAK is such a great framework and would be so useful to the Python community that it seems a shame to keep it all to ourselves. :) [19:51:20] And if lots of people contribute them, my productivity is thus multiplied. [19:51:41] <_jpl_> This is true. I need to get off my butt and finish the database example. [19:52:06] Also, if people write docs, then I have something to brag on at work about the value of doing all this stuff open source. :) [19:52:45] <_jpl_> I think the Junction tool I've written will be a great example of something really useful you can do with PEAK and Twisted, and over time could become a really powerful message-oriented-middleware tool for Python apps. [19:53:29] One concern I do have about releasing 0.5 final is that I really really want a stable core. [19:53:37] <_jpl_> Even some of the Twisted folk seemed interested in Junction, and you know how hard it is to get them interested in something. :) [19:54:00] * pje laughs [19:54:13] <_jpl_> There's a lot of skepticism in that camp about PEAK, and this could show how nicely the two can go together. [19:54:27] Anyway, 'model' is in the core, but it and 'storage' are pending some big changes when peak.query comes out. [19:54:32] <_jpl_> I think the core is pretty stable, though. It seems to be, anyway. [19:54:37] * pje smiles. [19:54:54] That's only because you don't see how incomplete it is, from my perspective. :) [19:55:08] Seriously, I think binding is nearly everything I could ask for. [19:55:29] <_jpl_> I also think of binding/naming/config as more of the core core. :) [19:55:35] config and naming still have a ways to go. I haven't used them past the extent of their current capacity, but I will need to. [19:55:44] <_jpl_> With model and storage and the next layer of the onion. [19:55:49] naming is based on model, to some extent. [19:56:00] You could look at it that way. [19:56:13] <_jpl_> It's ok to add more features after an initial release. :) [19:56:27] But to me, it's difficult to have much of an application without a domain model. [19:56:42] JPL: it's backward compatibility and *docs* that worry me. [19:56:56] Keeping docs in sync is PITA. [19:56:57] <_jpl_> It depends on the size of the application, though. You can write any small application without a domain model. [19:57:01] Think about what the binding API change would have meant if we had had better docs... [19:57:30] I think model and storage are in for some big refactorings when query comes out though. [19:57:46] So, I'm hesitant to do much with docs for them right now. [19:57:47] <_jpl_> In fact, as Fowler discusses in PEAA going with a domain model adds a lot of complexity that's not really warranted in smaller apps. [19:57:51] pje: thanks for the compliment on HelloWorld. [19:58:16] JPL: that's because he doesn't have PEAK to manage the model for him. :) [19:58:32] <_jpl_> Ha, well even with PEAK it's not always non-trivial. [19:58:37] In PEAK, even small apps can benefit from domain models, even for something simple like parsing. [19:58:58] <_jpl_> True. [19:59:02] Think about all the naming URL parsers, for example. That's a very potent example of the use of even a single model class in a small usage. [19:59:17] <_jpl_> Though no one but you and Ulrich know how to use fmtparse at the moment. :) [19:59:40] Don't forget alexander, too. Or maybe it was Radek. Maybe both of them, I don't remember. :) [19:59:53] And Ty has used it, too, don't forget. :) [20:00:16] <_jpl_> I've been wanting to add mdl_syntax to my domain classes, but just haven't had the time to get familiar with fmtparse. [20:00:54] The basics are 'Sequence(rule, rule, rule...)' -> matches the items in sequence. [20:01:08] Alternatives(rule,rule,...) matches any one of the rules. [20:01:32] Optional(rule, rule, rule...) is a Sequence that is allowed to not occur (like ? in a regex) [20:01:46] Indeed, Alternatives is like rule|rule|rule in a regex [20:01:51] * _jpl_ nods [20:02:13] If you want to match something that goes in a field of the object, just use the field. [20:02:15] <_jpl_> I've seen where you use it in various places, but have just never had the time to try to use it. [20:02:27] Like Sequence('//', host, ':', port) [20:02:32] Matches '//' [20:02:34] <_jpl_> Mainly I'd need to think of a sensible syntax for my domain objects to start with. [20:02:54] followed by the host, a ':', and the port. The syntax for host and port then comes from the type of those features. [20:04:19] <_jpl_> Is it possible to have alternate syntax models for use in different contexts? Like a simple syntax for some purposes and maybe an XML syntax for more detailed descriptions of the data. [20:04:43] No. [20:05:20] Well, actually, yes. [20:05:24] <_jpl_> We've run into the problem of needing to convert model objects into a format that can be sent over a PB method call, then turned back into the right kind of object on the other end. [20:05:32] You just can't do it with the built-in mdl_syntax stuff. [20:05:55] Just define an adapter to an interface that does "render to PB". [20:06:13] And have a factory on the other end to "unrender PB". [20:06:52] The adapter can use the metadata like mdl_features to figure out what to convert. [20:07:06] <_jpl_> I'm not sure if an adapter would work, since PB will complain about anything that's not a simple data type (or a child of Copyable). [20:07:48] You misunderstand. I mean, adapt to IPB_Renderer, which would have a 'renderPB()' method that returns the appropriate converted data. [20:08:11] Or, you can do it directly, but you would register an adapter *function* rather than an adapter *class*. [20:08:12] <_jpl_> Ah, ok. [20:08:27] <_jpl_> adapter function, hmm... [20:08:36] <_jpl_> That sounds interesting. [20:09:07] Adapter factories in PyProtocols are just two-argument callables. It doesn't care if they're types or functions or methods. [20:09:24] <_jpl_> We've come up with a way to build what Fowler calls data transfer objects which should probably be redone with adapters. [20:13:11] <_jpl_> The main problem to solve there is collapsing objects, which may contain lots of other objects, into something simple like a dictionary. [20:13:36] <_jpl_> Or at least something simple enough that we can use Copyable, etc. [20:14:47] <_jpl_> Hmm, if we improve the way we do this, and make it more PEAKish, this might be a standard "service" available with Junction. [20:16:21] Well, of course you recursively invoke the adaptation (and possibly method call) on every item you're encoding in the parent object. [20:16:45] <_jpl_> ...which would provide a generic way to shuttle model objects between clients and servers, where the server would be handling actual storage via DMs. [20:17:32] <_jpl_> I think it would be difficult to use a DM for distributing the objects, since DMs are oriented to a single entity (afaict). [20:17:56] Interestingly enough, there's a tie-in to peak.query even there. [20:18:22] peak.query will be dealing with "tuples" in the relational or (Linda/JavaSpaces) sense. [20:18:33] <_jpl_> I'd *love* to be able to use peak.query... [20:18:42] And PEAK's messaging facilities, as well as storage, will be built on those tuples. [20:19:00] <_jpl_> PEAK's messaging facilities? [20:19:02] And "events" in both the GUI sense and system monitoring and ohter senses, will also be tuples. [20:19:08] Messaging like JMS. [20:19:16] Application messaging. [20:19:17] <_jpl_> Didn't know that was in the works. [20:19:26] <_jpl_> That's essentially what Junction is about. [20:19:35] Well, to me it's a subset of storage+query. [20:19:39] Messaging, I mean. [20:19:52] Messaging is just a DM, in a sense. [20:20:13] <_jpl_> How's that? [20:20:17] Except that instead of using getitem, it's more like you have an iterator, and the ability to add items. [20:20:32] Well, a DM maps domain objects to an underlying persistence basis. [20:20:51] And, a tuplespace is a persistence basis that's structurally equivalent to a messaging system. [20:21:06] That is, shared storage means you have a messaging queue. [20:21:35] However, a messaging API like Spread or PB is equally usable as a DM backend. [20:21:49] Consider that SQL itself is just another messaging protocol. :) [20:22:02] You just have a mediating server that makes the effects of your messages persistent. :) [20:22:14] And transactional messaging fits right in with everything else. [20:22:15] <_jpl_> Sure, but Spread and PB are asynchronous... how would you implement an asynch DM? [20:22:35] <_jpl_> You make it all sound so simple. :) [20:22:40] * pje grins [20:22:48] It is, at least conceptually. [20:22:55] Anyway, you can't make an asynch DM. [20:23:04] But in principle even SQL protocols are asynch. [20:23:13] So it's obviously a solvable problem. ;) [20:23:16] <_jpl_> Tell you what, you write up a design and Ulrich and I will try to implement it. We're both *really* interested in realizing this sort of thing. [20:23:35] <_jpl_> Junction is my first stab at it (based mainly on the design of the OSE toolkit), but it's not particularly PEAKish. [20:23:43] I see the proper use of Twisted as "half async" - the I/O is async, but the app is not. [20:24:04] It's a lot easier to let the app run in a separate thread from the async portion, and call back into the reactor thread. [20:24:21] So, you could certainly take that approach for a messaging DM built on an async protocol. [20:24:40] <_jpl_> True, that would be very convenient from an appdev point of view. [20:24:54] i.e., simply arrange for your DM to place items into (and receive them out of) a pair of threading.Queue objects. [20:25:13] <_jpl_> Where writing an entire application to work asynchronously is a pretty big paradigm shift for most programmers. [20:25:18] Yep. [20:25:23] And entirely unnecessary. [20:25:52] <_jpl_> Not to mention very error prone. [20:25:57] Yep. [20:26:03] <_jpl_> ...harder to test, etc. [20:26:10] So, that probably puts me on the Twisted folks' blacklist. :) [20:26:11] <_jpl_> Harder to understand... [20:26:45] After all, rather than do the proper thing and refactor everything to be async, I wrote the 'supervisor' component to fork multiple processes to serve FastCGI. [20:26:49] <_jpl_> Yes, don't you know that asynchronous programming is far superior to any other method? Isn't that painfully obvious? [20:26:51] Heresy, I tell you, heresy! [20:27:12] Painfully is definitely the right word. [20:27:25] <_jpl_> Don't you know that everyone will laugh at you if you suggest otherwise? etc. :) [20:27:54] Anyway, a DM-as-messaging-queue only needs a few extra methods... [20:28:04] Basically put() and take(), if you think about it. [20:28:22] put(anElement) or anElement=dm.take() [20:28:38] And, depending on the application, maybe take() would accept parameters. [20:28:46] (to filter applicable messages) [20:29:36] Upon transaction commit, you'd remove any 'take()'n objects from the underlying queue, or tell other participants the object has been removed, or whatever. [20:30:07] And that is more or less a Linda or JavaSpaces implementation, right there. [20:30:37] But, this is somewhat of a "hard way" to do it. [20:31:00] <_jpl_> Oh? [20:31:19] When peak.query comes to town, there'll be an easier way. There will be a framework for pumping tuples around in an app, such that you can define filtering or joining operations to compose new tuple sources or tuple sinks. [20:31:41] And, you'll be able to use simple components to convert incoming or outgoing tuples to messaging formats or databases or whatever. [20:31:53] And that will be *without* using the DM part of the application stack. [20:32:01] IOW, it will just be "raw data", no objects. [20:32:14] DM's will then be able to map over a tuple source. [20:32:26] * _jpl_ boggles. [20:32:31] So, instead of making app-specific DM's, you'll use generic DM's. [20:32:52] And instead of app-specific DB or messaging code, you'll use generic adapters to tuple streams. [20:32:55] <_jpl_> Which map to a certain backend? [20:33:00] (or from, as the case may be) [20:33:01] Yep. [20:33:16] And I'm not finished describing it yet, so don't boggle so soon. :) [20:33:39] <_jpl_> I can always double or triple boggle. :) [20:33:55] <_jpl_> I think I also need to read the JavaSpaces spec. [20:33:58] There will be a conceptual mapping system in peak.query for mapping from a domain-object schema to a tuple-based relational schema. [20:34:10] So, the DM's won't even have to do that much. :) [20:34:47] And all of this stuff will be usable for queries against underlying data sources, in terms of the application's object model. [20:34:52] <_jpl_> So you think we'll have that next week sometime? :) [20:35:23] Of course, you could also have different domain-object schemas over the same tuple schema, or vice versa. [20:35:25] No, I don't. [20:35:39] You'll be lucky to get it in 2004Q1. :) [20:35:50] And thankful, too. :) [20:35:52] * pje grins [20:36:15] * _jpl_ certainly would be. [20:36:57] Heh in the font I'm using that looked like "200,401" :) [20:37:09] which would be a long time to wait. [20:37:31] <_jpl_> For now I might think about implementing your threading/half-async idea in Junction in order to remove the complexity for the user. [20:38:15] <_jpl_> Currently I'm doing that with an object proxy which deals with whether the "service" you've looked up is in the local process or at the other end of a PB connection. [20:39:00] I'm pretty sure Twisted has some facilities for half-async, since there are various reactor methods dealing with threads. I just haven't needed to look at them yet. [20:39:41] <_jpl_> I haven't seen any, but I'm no Twisted expert. [20:40:09] A subset of peak.query that will almost certainly land in 2004Q1 is the basic data layer, which will allow composition of queriable ("pull") tuple sources like relational DBs. [20:40:26] There's a mailing list post about that, something about "layers" of O-R mapping. [20:40:58] Ah yes... http://www.eby-sarna.com/pipermail/peak/2003-November/000851.html [20:41:07] The "physical layer" referred to there. [20:41:40] That post also talks about mappings and conceptual queries, but doesn't address "push" tuple sources (events, messaging, that sort of thing). [20:42:29] The post is also focused exclusively on normal database-ish applications of even "pull" tuple sources. [20:43:07] But even just being able to do that, means we'll be able to have a "generic DM" as soon as the mapping layer is in place. [20:49:38] Wow, it's late. [20:49:52] I'd better head home. [20:50:19] See y'all later. [20:50:21] <_jpl_> You're on the East Coast? [20:50:29] Yes. [20:50:41] <_jpl_> Whereabouts? [20:50:46] Florida. [20:50:57] * pje has gotta go [20:51:00] <_jpl_> San Francisco here.. [20:51:04] <_jpl_> See you later. [20:51:05] Later, all... [20:51:11] ** pje has left us [21:01:42] <_jpl_> Gotta go, too. See you all later. [21:01:47] ** _jpl_ has left us [21:06:13] BYE ! [21:06:34] * Maniac notices they're both gone [21:07:26] Do you have any idea how you take keycaps off a standard pc keyboard? [21:08:24] nevermind, I igured it out. [21:10:26] hammer? [21:15:35] knife. [21:16:15] Which I can say again, now that my f key is no longer sticking. [22:17:04] ** vlado has joined us [22:30:03] ** rdmurray has joined us [22:31:07] Hmm. Does peak.running automatically start a transaction? [22:31:40] AFAIK no [22:34:02] from peak.storage.autocommit import * [22:34:03] [22:34:03] class QueuedMessageSender(AutoCommitter): [22:34:03] [22:34:03] def send(self,message): [22:34:03] # ... [22:34:05] [22:34:07] send = autocommitted(send) [22:35:07] use something like that [22:44:48] ** victorng has joined us [22:55:20] * vlado is away, somewhere far beyond... (l!on)(p!off) [cRk/bx] [22:58:05] ** rdmurray has joined us [23:16:16] i have no menal energy lef [23:16:19] t [23:16:22] er mental [23:16:36] You just have no 't's left. [23:21:42] i want to do something productive but i just aimlessless wonder the intarweb and my son is on the playstation.... [23:25:18] Write some documentation for PEAK :)