[TransWarp] Tips on using model, storage, config, and naming (was Re: First attempt to use the storage package in PEAK)

[Phillip J. Eby]

At 04:48 PM 12/29/02 +0200, Roché Compaan wrote:
>But at the moment I really value these small misunderstandings because
>at the current rate you'll end up documenting most of PEAK before 2003
>;-)

Not really; just handwaving in the general direction of some of what's 
already there or planned.  This is far from exhaustive documentation on 
these features, or even a broad sampling of the features that are there.

But I will admit that this is nicely filling out common use cases and 
motivating examples for many of PEAK's key feature areas.


>In the rule foo.* = # somecode, I access the value of * with
>propertyName.  Correct?

Yes, but it's not the value of '*'; it's the full property name.  So you 
have to strip off the prefix.  But, you can access the rule's name with 
'ruleName'.  So you could do something like:

[foo]
* = propertyName[len(ruleName)-1:]

To make the value of 'foo.bar' equal to 'bar'.  It's better to do it this 
way than to hardcode the length of the rule name, because you never know if 
somebody might want to include that configuration file in another, with a 
prefix...  For example:

[Load settings from]
file="something.ini", prefix="spam"

If the previous "[foo]" example was in "something.ini", its 'ruleName' 
would be "spam.foo.*" if you looked up 'spam.foo.bar'.



> > The config package gives you all the hooks you need to deal with this sort
> > of thing, but of course you have to implement your basic application or
> > framework in a suitable way.  This could be as simple as using:
> >
> > config.setupModule()
>
>Aah, I forgot about that. This is AOP in action is it not?

Really, it's more SOP (Subject Oriented Programming), but even that is 
stretching it for this usage.  In this context, I just think of it as 
module inheritance and/or patching.  Sort of like package inheritance in 
UML, really.


> > >I've attached the mysql driver if you want to include it - it was
> > >fairly simple to write since MySQLdb implements the DB API.
> >
> > I'm afraid MySQL gives Ty and I the creeps, and we don't feel comfortable
> > including a driver for it with PEAK for a number of reasons.  Because 
> we do
> > try to "think of everything", we are concerned about its locking and
> > transactional semantics and how they would interact with PEAK's caching,
> > locking, and transactional patterns.  We couldn't say whether or not it
> > will work correctly, and so are totally uncomfortable including it in an
> > "enterprise application kit".
>
>Given the amount of effort that went into its transactional capabilities
>that was recently added, I think you owe it at least some benchmarks.

It's the recency of the adding that bothers me.  Transactions are hard to 
do correctly.  Repeat after me: transactions are hard to do correctly.

Now, if the developers of MySQL had said, "transactions are hard; we plan 
to do them in a later release once we've gotten things the way we need them 
to be as a basis for that", then that would be one thing.  That's not what 
they said.  They said, "transactions are overrated, you don't really need 
them, they slow things down, etc."  Bluntly, I don't trust anybody who says 
something like that with my or anyone else's data.  The impression I got is 
that transactions were a grafted-on afterthought that were added for 
marketing reasons.  That is, so that supporters of MySQL wouldn't keep 
getting kicked in the teeth by people who insist on ACID properties.

But it's not just in the area of transactions that the MySQL designers 
chose to pass over standard database practices: locking, relational 
integrity, subqueries...  you name it, they left it out and badmouthed it 
as "not really necessary".  No, not if you're making an ISAM database.  I'm 
sure MySQL is a fine ISAM package and it's nice that it supports a rather 
limited subset of SQL.  But for crying out loud, *Gadfly* has better 
transactional capabilities and supports a *much* larger subset of SQL.

The thing is, people see MySQL and think that that's all there is to a 
relational database: ISAM and SQL, and gosh, they don't have to even think 
about things like relational integrity.  If you look at MySQL manual 
comments or discussions of MySQL techniques, you see lots and lots of 
people doing Very Bad Things in terms of database and application 
design.  That's why more people use it, because there's less to learn.  But 
some things, you really need to do the learning, or else you need to use 
something that is going to take care of you.  Alas, MySQL needs someone to 
take very good care of *it*, not the other way around.


> > In the environment where I work, *PostgreSQL* is considered a lightweight
> > database for prototyping purposes, and Sybase or Oracle are what you use
> > for production applications.  One of our apps has a 30GB database on a 
> RAID
> > array, and its dedicated database server hardware is a 64-bit,
> > four-processor machine, with an identical warm standby server continuously
> > replicated from the first.  PEAK really is intended to work at this 
> sort of
> > "enterprise" scale, and using MySQL for production in such an environment
> > would be quite insane, IMO.
>
>Huh? Sure it's supposed to work at that scale but you will certainly not
>make an environment like this a requirement in the readme.

I won't?  Darn.  ;)  Just kidding.


>As far as I
>can see, PEAK aims to provide way more than just robustness and
>scalability.

Yes.  It also aims at saving developers from themselves, within 
reason.  Failing that, I *strongly* prefer that a developer avoid or 
abandon PEAK because it's too "hard", than that they "easily" shoot 
themselves in the foot.  I "easily" shot myself in the foot at times with 
both the early versions of Zope, and even with my own framework 
ZPatterns.  In both Zope and ZPatterns, design decisions were made that 
favored ease, transparency, and "do what I mean"-ishness over robust 
applications, where "robust" is defined as "easy to maintain and debug".


>For one, it provides a serious developer with a framework
>that heeds to proven software design patterns. There are even very small
>apps that can benefit from PEAK from a design point of view.

Oh, most certainly.  Even system administration scripts can use PEAK usefully.


> > So, while I'm sure that you would never use it for an application
> > environment it wasn't suited for, I'm not comfortable with making any
> > implication that PEAK supports or endorses its use in any way, because
> > PEAK's intended audience *does* include people who would abuse MySQL, and
> > in the process would give other people an excuse to trash PEAK, Python, 
> and
> > Open Source in general for their lack of attention to "enterprise-level"
> > concerns.
>
>My guess is that the type of people you describe will steer right past
>PEAK and click away in their decorated build-it-in-a-day IDE's.

You miss my point; I'm saying that "enterprise application development" is 
an extremely *political* endeavor.  It's not that the person who would hold 
that opinion might ever use PEAK, it's that they might use MySQL as a 
weapon to discredit someone who *did* want to use PEAK in that organization.

Let's just say I've had some experience of this sort of political 
infighting before, and leave it at that.  :)


>maybe you just
>think MySQL is really really creepy ;-)

Yes, I do.  The creepy part to me is that anybody would choose to use it 
when there are so many open source databases with better features in one 
area or another.  For example, SQLite is faster, PostgreSQL has better SQL 
support, O-O extensions, and language add-ins, etc.  There's even SAP DB or 
Interbase/Firebird if you need name recognition, however small.  :)