THE MISMEASURE OF HOME SCHOOLING EFFECTIVENESS (A Commentary)

Gregory J. Cizek
Assistant Professor of Educational Research and Measurement
University of Toledo
Toledo, OH 43606-3390

Keywords: Homeschooling, home schooling, home education, effectiveness

Try to imagine these scenarios. A prospective sports car purchaser asks the dealer several questions about the car’s exhaust system and fuel efficiency, but gathers no information on the car’s acceleration or handling. An office manager interviews a candidate for secretary; the manager inquires about the candidate’s favorite foods and community involvement, but never finds out about how well the candidate can type, spell, or interact with the public. Qualifying events are conducted in order to select Olympic track and field athletes in which the contestants are carefully evaluated for their height, weight, and physical attractiveness, but their speed, agility, or strength are not assessed.
What do these scenarios have in common? It is probably obvious that, in general, the primary consideration for someone who is interested in buying a sports car is how well the car drives. The most relevant information to gather about potential secretaries is their ability to type, etcetera. And no country would have much success in the Olympics if its athletes were selected based on their looks. The commonality in the three scenarios is that the most relevant information that could have been gathered was neglected, given the purpose of gathering information in the first place.
What does this have to do with home education research? Boldly stated, my assertion is this: There is little or no difference between the examples above and the current focus of much home education research. The balance of this commentary will attempt to substantiate that assertion and to provide suggestions for reconciling the objectives of home education with the goals of home education research.

A Mismatch

For the sake of illustration, consider the example described above regarding the Olympic track and field qualifications. The primary objectives that should be assessed in qualifying competitions seem obvious. The reason they seem obvious is only because people agree on the goals that bear on Olympic competition: to run faster, jump higher, and so on. The objectives derive naturally from the goals once the goals have been explicated and agreed upon. Stated another way, if it is agreed that the athletic competitions exist in order to determine who is the fastest, strongest, etcetera, then assessing the speed, strength, etcetera of competitors is reasonable, obvious, andCindeedCnecessary.
In assessing home education research, it seems reasonable to follow a similar approach: First identify the goals of home education, then assess the extent to which progress toward the goals is evident. To date, research on home education has permitted confident conclusions about some aspects of home schooling. These conclusions reflect, to a degree, on the goals of home education, although they bear most directly on the motivations of home schoolers.
The extant body of research illustrating the motivations of home educators reveals that the goals are not mysterious or hidden. Repeatedly, consistently, unequivocally, the finding is reported that the primary motivation for most home educators reflects a moral, spiritual, or religious component (see, e.g., Gustafson, 1988; Mayberry, 1991; Van Galen, 1988). Researcher Patricia Lines acknowledges that there are many reasons for home schooling, but observes that “religion is likely to be the most important motivation for home schooling” (1991, p. 14). Similarly, Ray (1989, p. 3) has concluded that “it now appears that home schools are predominantly filled with those primarily driven by ideological and moral motives.”
Reporting on her research conducted with home schooling families in the western United States, Mayberry (1988) has detailed the religious motivation:

The largest category of home school parents are those motivated by religious beliefs (65%)….They believe that it is their duty to instill particular religious beliefs and values in their children….Religious home school parents advocate an education for their children that is organized and controlled by parents and that focuses on Biblical training and teaching religious history. (p. 37)

Thus, although not the only motivation, it is well documented that the primary motivation for parents choosing home education is their desire to address the perceived spiritual, moral, or religious needs of their children. If this is the case, then by what measure should the effectiveness of home education be assessed? If a primary goal of home education have been so clearly identified, thenClike the Olympic trialsCit seems obvious that assessment should focus on the extent to which progress toward that goal is evident.

What’s Really Happening

Like the prospective sports car purchaser, home education research has focussed on nearly everything except the moral and spiritual outcomes that are the reasons that the majority of beginning home educators indicate they value most highly. The literature on home education abounds with demographic information on home schooling families, documentation of the academic achievement of home schooled students, the students’ socialization experiences, and their self-concept. On the one hand, it is laudable that so much progress has been made and that these outcomes have received critical attention. Home education researchers are few and research on these outcomes provides important contributions to a limited literature base. Nevertheless, one gets a sense from the literature that home education research has been driven more by an unconscious desire to assess home schooling from a traditional, institutional education frameworkCby standards of evidence and rationales that are external to the stated goals of the movementCthan by an alternative, radical approach that would be suggested by the counter-institutional, individualistic nature of the movement. It is precisely as if we have never bothered to find out how fast it is or how agile the sports car handles, although a wealth of information about its fuel economy has been gathered.
This point has been made previously, though it has gone unheeded. For example, a 1988 article subtitled “Is Home Schooling Making the Grade?” describes research findings related to cognitive outcomes, test scores, and socialization outcomes of home schoolers. Among the conclusions presented in the article, the author wonders whether:

all of the measuring and evaluating of home-school achievement scores, self-concept ratings, and social adjustment categorizations is a moot exercise in terms of defending and promoting this mode of educating children. Perhaps what is needed, in addition to the other comparisons, is for researchers to question home-school parents carefully to find out more precisely what their objectives are for their children. (Ray, 1988, p. 26, emphasis in original)

It is precisely this kind of research that is still necessary. In fact, Ray may have been too kind if it was his intention to imply that the lack of relevant research is attributable to the fact that researchers are unaware of home schoolers’ objectives. Indeed, as noted above, research has provided a consistent and longstanding portrait of the motivations and objectives of home educators. The problem is this: Nothing has been done to assess any outcomes related to the movement’s primary, self-selected objective.

What Might Lie Ahead?

Interesting, though challenging, lines of inquiry may lie ahead. Assessing cognitive outcomes is often difficult, though usually manageable. Assessing affective outcomes is somewhat more subjective and difficult, though this too can be successfully accomplished. Admittedly, the specter of attempting to assess spiritual or religious outcomes is fraught with difficulties.
The question, however, is not “Should these outcomes be assessed?” but rather, “How can they be assessed?” In order to evaluate the hopes of home education, the claims of home educatorsCindeed, the very rationale for the existence of most home school programsCit is essential to gather information about the success of these programs vis-a-vis their stated objectives.
A first, necessary step would be to conduct investigations of the quantity and content of religious or moral education that is experienced by home educated students. However, a second phase of research would also be necessaryCa phase in which the moral, spiritual, and religious outcomes or results of home schooling are examined.
What might some of these outcome measures look like? As with any construct, the religious dimensions of home education might only be dimly represented by “proxy” measures. Clearly this means that theory development in the area of home education and religious outcomes must occur, in which the expected relationships between reasons for home education and observable variables are specified. Further, reasonable means of operationalizing the outcome variables are needed. For example, it seems reasonable, given the aims of home education, that students in that setting would exhibit a more extensive understanding of religious concepts and principles than “control” groups. Or, if their religious and moral training were effective, home educated students would be expected to engage less frequently in behaviors proscribed by their faiths (e.g., out-of-wedlock pregnancy, alcoholism, crime, etc.) and to engage more frequently in behaviors encouraged by their religious traditions (e.g., dependability as an employee, church attendance, volunteer or community service activities, attitudes toward fellow human beings, etc.). Of course, these variables are imperfect measures of an unobservable, nonphysical characteristic (and, the listed manifestations of that characteristic are certainly weak, atheoretical examples provided for illustration only).

Conclusion

We know that the decision to begin educating a student at home is most often and primarily motivated by the desire to instill certain religious or moral beliefs, attitudes, or behaviors. However, secondary motivations are also apparent; for example, parents’ desires to expose their children to a high-quality, academic curriculum.  Much evidence has accumulated to attest to the effectiveness of home-based education in achieving these secondary objectives; little if any attempt has been made to assess the primary goal.
In theory and in practice, however, if home education is effective in accomplishing its generally acknowledged primary goal, its effectiveness should be demonstrable in terms of differences in valued spiritual, moral, or religious outcomes compared to non-home schooled students. A call for a different, more relevant, line of inquiry in the area of home education is in order. It is concluded that future research should begin to fully describe the substance and degree of religious and moral training afforded to home school students, and proceed to examine the efficacy and outcomes of that training.

                             References

Gustafson, S. K. (1988). A study of home schooling: Parental motivation and goals. Home School Researcher, 4(2) 4-12.
Lines, Patricia M. (1991). Home instruction: Characteristics, size, and growth. In J. Van Galen & M. A. Pitman (Eds), Home schooling: Political, historical, and pedagogical perspectives. Norwood, NJ: Ablex.
Mayberry, Maralee (1988). Characteristics and attitudes of families who home school. Education and Urban Society, 21(1), 32-41.
Mayberry, Maralee (1991). Political and religious beliefs of home school parents: Results of an ongoing study in four western states. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.
Ray, Brian D. (1988). The kitchen classroom: Is home schooling making the grade? Christianity Today, 32(11), 23-26.
Ray, Brian D. (1989, March). An overview of home schooling in the United States: Its growth and development and future challenges. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
Van Galen, Jane A. (1988). Ideology, curriculum, and pedagogy in home education. Education and Urban Society, 21(1), 52-68.