Garfunkel's Syndrome, Part II

Lest ye believe that the paper I introduced in my previous post was authored by some kind of anti-research nutter, let me share this from the conclusion:

I wish to be clear. I recognize that Faffufnik [i.e., any mathematics education researcher] has done important research. I recognize that Chaim Yankel's [i.e., any statistician's] protocols can help quantify our results. We have to learn from the past and theoretical frameworks are important for future work. But we also have to recognize that quoting Faffufnik and Chaim Yankel is not a substitute for imagination, creativity, and the application of common sense.

This is, I think, a conclusion that one can easily agree with, mostly because it is a weaker version of the perspective that holds throughout the balance of the paper--that judgments about the quality of mathematics education projects should not be anchored in any way to research.

I agree with the weak version of this conclusion and, in part, with the stronger perspective. But I disagree somewhat with how the author gets there. So, for the remainder of this post, I will focus on those disagreements--or rather, one disagreement.

From page 3 of the paper:

It is simply not possible to prove that an approach to teaching and learning will be effective before the fact. . . . It is in the nature of the enterprise that we cannot discover what works before we create the what.

What a relief! Now I can dust off that shin-kicking proposal that's been gathering dust on my bookshelf. To be brief, it's a proposal--chock-full of "imagination" and "creativity"--to teach mathematics to second and third graders by kicking them each in their shinbones every ten minutes during math time.

I know what you're thinking, and you can laugh at my idea all you want, but you really can't possibly know if this method will work until it's tested. Right? Well, of course you can. And, yes, I realize that that's not what Mr. Garfunkel is suggesting. He almost certainly has a limited set of "approaches" in mind.

But, see. That's just it. If we can indeed restrict ourselves to considering a limited set of approaches to teaching mathematics--an idea that contradicts the two sentences quoted above--then, for crying out loud, what restrictions do we use? The following is, in part, Mr. Garfunkel's answer to that question:

Curriculum development, in particular, is best related to an engineering paradigm. In order to test the efficacy of an approach, we must analyze needs, examine existing programs, build an improved model program, and test it--in the same way we build scale models to design a better bridge or building.

So, according to Garfunkel, in order to judge the quality of a mathematics education project, one should not have to tie it to the research literature, but rather one should prove that it's (a) connected to something that's worked before (and/or an improvement on something that hasn't worked before) and (b) something that meets "needs."

He contrasts this approach with the one that produces the Faffufnik-Chaim Yankel Effect:

There appears to be an underlying assumption here that mathematics education projects must proceed in the following way. First, they must be based upon research. Therefore, we heavily quote the results of prior research (See the papers of Faffufnik). Then based upon that research we make a new research hypothesis and test it with a small number of students. If at all possible we make this experiment as close to a 'gold standard' double-blind medical approach as possible. Then using certain statistical protocols (See the work of Chaim Yankel) we conclude that there is some measurable effect and write a new proposal to test this effect on a larger population. This process is then iterated. This is now a necessary condition for funding--independent of the content and strength of the ideas being considered.

In other words, quoting research and using statistical protocols to determine effects: bad; quoting predecessor projects and using the criterion "meets needs" to determine effects: good.

The truth is that both Garfunkel's method and the current research method are seriously flawed.

Next time.