Garfunkel's Syndrome, Part IV
Okay, so back to soup. Did you read the recipe I included in my previous post? If so, what did you think about it?
For point-making purposes, it is my sincere hope that your answers to those questions are something close to no and nothing. But, regardless, my assumption (which I think is a safe one) is that, of the percent of people who might bother to read every word of the recipe, a relative few, not including myself, would have any kind of comment on it, per se--especially given such an open-ended question as the one above.
Let's start by very briefly contrasting the original recipe (the "original" that I followed for cooking anyway) with the recipe that you read, which was a modification of the original. (We'll pursue this analysis in more detail later.) I made only one substantive change to the original recipe. By taking the ingredients (and preparation suggestions [e.g., "diced," "sliced"])--which were tidily chunked together under one section in the original--and spreading these out among the cooking instructions, I changed the recipe's coherence, or, more specifically, the "packaging" of the content--how related material "hangs" together (and, of course, how unrelated material doesn't):
It is important to note here that I did not destroy the original recipe's coherence, nor did I create a coherence where none existed previously. I changed how the content was packaged, giving the rewritten recipe a different kind of coherence. The original recipe contains two larger bundles in which the content objects are related by a certain characteristic--ingredients/preparation or cooking instructions. The rewritten recipe has seven smaller bundles in which the content objects are related by point of use. One can make various arguments in support of either type of coherence, drawing on the other standards (order, clarity, and precision) or on relevant ideas outside of the principles.
It is also important to note that although the recipes above are written down, each can represent both spoken and written instructional content. That is, each recipe can be a representation of a lesson that is taught to students.
Then I tried a favorite of mine, ScienceDirect. I came up with 822 results for the search term "teacher salaries." For "teaching fractions," 34.
Of course, this is a bit of cherry picking to try to illustrate a point. One might likely easily find examples of searches that would suggest a very different conclusion. Still, the relative lack of attention given to issues "inside education" on the part of the mathematics education research community is real and has not gone unnoticed even by members of that very community:
Suppose, for example, that you study the produce departments of two different grocery stores in the same city. You rate both departments' cleanliness, the friendliness of their employees, the variety they each offer, and how much (in dollars) each store sells in produce. After your experiment ends and you analyze these data, you see that Store A sold much more produce than Store B did. You also observe that Store A scored significantly higher in cleanliness than did Store B, but that each store had about the same rating for friendliness and variety. You conclude that the cleanliness of a store's produce department matters more than friendliness and variety in its ability to make money in the short term.
Now suppose that you collected data from both stores for exactly four hours on the same (non-holiday) Friday, but that for Store A, data were collected between 2:00 p.m. and 6:00 p.m. and, for Store B, data were collected from 7:00 p.m. to 11:00 p.m. In that case, time of day is certainly a confound, because it could explain not only the superior sales of Store A (most grocery stores are much more likely to be busier on a [non-holiday] Friday between 2:00 p.m. and 6:00 p.m. than they are between 7:00 p.m. and 11:00 p.m.), but it could also explain the difference in cleanliness between the two stores. Just considering the possibility of a confound is enough to make your result go down in flames.
How about our recipes? Suppose we want to know whether or not it makes a significant difference in learning to have a teacher with 8-10 years versus 1-4 years of cooking instruction under her/his belt. We scour the literature on the subject and conclude, perhaps, that it doesn't matter. We look at 32 studies (all focused on the issue of teacher competence [operationalized as 'years experience']), and eighteen of those studies tell us that it does matter one way or the other and the other 14 say the opposite; it's a veritable tie.
But what if none of those studies looked at the specific recipes used? If the type of lesson used might likely sway the results, then ignoring it compromises those results.
Stay tuned.
For point-making purposes, it is my sincere hope that your answers to those questions are something close to no and nothing. But, regardless, my assumption (which I think is a safe one) is that, of the percent of people who might bother to read every word of the recipe, a relative few, not including myself, would have any kind of comment on it, per se--especially given such an open-ended question as the one above.
The Setup
However, and again for point-making purposes, it's worth examining this "content" more closely, especially in light of some principles that will be familiar to readers of this blog:Let's start by very briefly contrasting the original recipe (the "original" that I followed for cooking anyway) with the recipe that you read, which was a modification of the original. (We'll pursue this analysis in more detail later.) I made only one substantive change to the original recipe. By taking the ingredients (and preparation suggestions [e.g., "diced," "sliced"])--which were tidily chunked together under one section in the original--and spreading these out among the cooking instructions, I changed the recipe's coherence, or, more specifically, the "packaging" of the content--how related material "hangs" together (and, of course, how unrelated material doesn't):
It is important to note here that I did not destroy the original recipe's coherence, nor did I create a coherence where none existed previously. I changed how the content was packaged, giving the rewritten recipe a different kind of coherence. The original recipe contains two larger bundles in which the content objects are related by a certain characteristic--ingredients/preparation or cooking instructions. The rewritten recipe has seven smaller bundles in which the content objects are related by point of use. One can make various arguments in support of either type of coherence, drawing on the other standards (order, clarity, and precision) or on relevant ideas outside of the principles.
It is also important to note that although the recipes above are written down, each can represent both spoken and written instructional content. That is, each recipe can be a representation of a lesson that is taught to students.
The Question
So here's the million dollar question--and the pivot to math education: Is it reasonable to hypothesize that the difference in coherence between the two lessons would effect some kind of difference in students' learning?The "Answer"
You can guess what my answer to that question would be. But what does research in mathematics education have to say about a question like this?Flaw #1: Research in mathematics education devotes relatively little time and effort to the mechanics of school mathematics.
Then I tried a favorite of mine, ScienceDirect. I came up with 822 results for the search term "teacher salaries." For "teaching fractions," 34.
Of course, this is a bit of cherry picking to try to illustrate a point. One might likely easily find examples of searches that would suggest a very different conclusion. Still, the relative lack of attention given to issues "inside education" on the part of the mathematics education research community is real and has not gone unnoticed even by members of that very community:
The panorama of work represented at professional education meetings or in publications is vast and not highly defined. . . Research that is ostensibly "in education" frequently focuses not inside the dynamics of education but on phenomena related to education—racial identity, for example, young children's conceptions of fairness, or the history of the rise of secondary schools [Josh: Or "teacher salaries"]. These topics and others like them are important. Research that focuses on them, however, often does not probe inside the educational process. (emphasis mine)There can be legitimate disagreement as to where exactly "inside the educational process" is (or what "pedagogical mechanics" are)--and, of course, there will almost certainly be a sometimes dissonant, sometimes harmonious overlap of various personal interests within the study of "the educational process." (My own [and others'] area of interest, for example, is content--the "text" of teaching--from whence my cooking example comes.) But it should be clear that, for example, investigating different ways of teaching long division is of far more direct utility to education than is investigating, say, how children "reason"--especially when those who purport to study "reasoning" refer to it as "something one feels interacting with people" (like nausea?).
The Consequence
Flaw #2: When mechanics are not considered in education research, they can become confounds.
Suppose, for example, that you study the produce departments of two different grocery stores in the same city. You rate both departments' cleanliness, the friendliness of their employees, the variety they each offer, and how much (in dollars) each store sells in produce. After your experiment ends and you analyze these data, you see that Store A sold much more produce than Store B did. You also observe that Store A scored significantly higher in cleanliness than did Store B, but that each store had about the same rating for friendliness and variety. You conclude that the cleanliness of a store's produce department matters more than friendliness and variety in its ability to make money in the short term.
Now suppose that you collected data from both stores for exactly four hours on the same (non-holiday) Friday, but that for Store A, data were collected between 2:00 p.m. and 6:00 p.m. and, for Store B, data were collected from 7:00 p.m. to 11:00 p.m. In that case, time of day is certainly a confound, because it could explain not only the superior sales of Store A (most grocery stores are much more likely to be busier on a [non-holiday] Friday between 2:00 p.m. and 6:00 p.m. than they are between 7:00 p.m. and 11:00 p.m.), but it could also explain the difference in cleanliness between the two stores. Just considering the possibility of a confound is enough to make your result go down in flames.
How about our recipes? Suppose we want to know whether or not it makes a significant difference in learning to have a teacher with 8-10 years versus 1-4 years of cooking instruction under her/his belt. We scour the literature on the subject and conclude, perhaps, that it doesn't matter. We look at 32 studies (all focused on the issue of teacher competence [operationalized as 'years experience']), and eighteen of those studies tell us that it does matter one way or the other and the other 14 say the opposite; it's a veritable tie.
But what if none of those studies looked at the specific recipes used? If the type of lesson used might likely sway the results, then ignoring it compromises those results.
Stay tuned.