[PEAK] Packaging peak apps

[Stephen Haberman]

> That's why I asked about your test methodology.  When adding a new test,
> you *must* first check that the test *fails*, because that's the only way
> to have some assurance you're testing the right thing.

Agreed. I do so /most/ of the time.

I don't think I had that bad of a test methodology back in April. I
certainly knew academically "test for failure first" but, especially given
the evidence that the test never should have passed, there is a possibility
I did not actually do it.

> By the way, this is what I meant earlier when I requested that you "Verify
> that this reduced patch set is correct; i.e. its test case should *fail*
> under unpatched 2.3 and 2.4, and succeed under patched 2.3 or
> 2.4."  (Emphasis added.)

Right, and I did verify that it does fail with unpatched 2.3 but then passes
with patched 2.4. 

I'm pretty sure earlier today I tested unpatched 2.4 for failure, e.g.
taking out the ReloadModule changes, just to see what would happen. However,
I did not do this on any sort of regular basis - given the overhead of
rolling back the changes, I only tested for failure on a semi-regular basis
with my installed, unpatched 2.3.

I also do remember doing the test-with-installed-release-binary (just
"python ...") vs. test-with-custom-built-binary (using "..\..\PCBuild\python
...") back in April with my original patch. And it was my first time
submitting anything to python-dev, so I remember taking more time to make
sure I dotted all my I's and crossed my T's. Surely I should have done the
released binary fail vs. patched binary success check and made sure the unit
test was actually executing. Surely.

But perhaps not.

- Stephen