[PEAK] Packaging peak apps
stephenh at chase3000.com
Thu Sep 16 19:36:51 EDT 2004
> That's why I asked about your test methodology. When adding a new test,
> you *must* first check that the test *fails*, because that's the only way
> to have some assurance you're testing the right thing.
Agreed. I do so /most/ of the time.
I don't think I had that bad of a test methodology back in April. I
certainly knew academically "test for failure first" but, especially given
the evidence that the test never should have passed, there is a possibility
I did not actually do it.
> By the way, this is what I meant earlier when I requested that you "Verify
> that this reduced patch set is correct; i.e. its test case should *fail*
> under unpatched 2.3 and 2.4, and succeed under patched 2.3 or
> 2.4." (Emphasis added.)
Right, and I did verify that it does fail with unpatched 2.3 but then passes
with patched 2.4.
I'm pretty sure earlier today I tested unpatched 2.4 for failure, e.g.
taking out the ReloadModule changes, just to see what would happen. However,
I did not do this on any sort of regular basis - given the overhead of
rolling back the changes, I only tested for failure on a semi-regular basis
with my installed, unpatched 2.3.
I also do remember doing the test-with-installed-release-binary (just
"python ...") vs. test-with-custom-built-binary (using "..\..\PCBuild\python
...") back in April with my original patch. And it was my first time
submitting anything to python-dev, so I remember taking more time to make
sure I dotted all my I's and crossed my T's. Surely I should have done the
released binary fail vs. patched binary success check and made sure the unit
test was actually executing. Surely.
But perhaps not.
More information about the PEAK