NORMS, STANDARDS, AND NONSENSE: TESTING POLICY GONE BAD

Consternation over the level of educational quality in the United States may now have reached its highest level.  On the U.S. Department of Education’s “wall charts,”  in state legislatures, in local newspapers, in national magazines, on television, and at the school board meeting, a major concern is educational quality.  The phenomenon is not new, of course.  What is new are the terms in which debates over educational quality are framed.  A new definition of educational quality has taken hold.  The new definition has radical implications for the assessment of home-based educational programs.
            This article has two purposes.  First, a brief analysis of the changing meaning of educational quality is provided.  Second, several ideas related to the measurement of educational quality will be presented to illustrate the effect of the revised definition of educational quality on home school families, home school policy, and the law.
 
             Defining Educational Quality
 
            Until roughly the 1960s, educational quality was traditionally defined in terms of input.  In fact, input was the major concern in assessing not just educational quality, but also in judging the equitable allocation of educational resources.  Input refers to the resources allocated to the educational process: salaries, textbooks, teachers, number of school days, materials, capital expenditures, etc.  Until recently, more was considered better:  More money spent on education was thought to result in a higher quality education.  And, if equal resources were expended on, for example, students of differing genders, or social or racial backgrounds, the expenditures were said to be “equitable.”
            Recently, however, the definition of educational quality has  fundamentally changed.  Analyses of educational quality now center on outcomes (Levin, 1978).  Outcomes are things such as grade point averages (GPAs), test scores, dropout rate, number of scholarships received, etc.  Educational quality could be considered poor if outcomes are less than desired, regardless of the amount of input.
            The change in how educational quality is defined is so dramatic that some have termed it a “paradigm shift.”  According to Chester E. Finn, Jr., formerly of the U.S. Department of Education’s Office of Educational Research and Improvement, the future of American education may be one in which,
            the enterprise of education will be defined entirely by actual learning accomplished and accounted for.  Indeed, no `education’ will have taken place unless there is evidence learning occurred.  (Rothman, 1989, p.12).
Concurrently with the arrival of a new definition of educational quality, a burgeoning interest in home-based educational programs has developed (Adams, 1984; Feinstein, 1988; Kohn, 1988; Klicka, 1988a).  The reasons for this re-interest in home schooling are many, varied, and have been ably described elsewhere (Pitman, 1986; Lines, 1987; Mayberry, 1988; Van Galen, 1988; Ray, 1989a).  The trend is clearly continuing, with increasing numbers of parents choosing the home education option.  Though no precise figures exist, estimates of the number of students engaging in home study range from 10,000 (Lines, 1984) to 1,000,000 (Naisbitt, 1984).  As some state legislatures and boards of education have moved to loosen requirements concerning home education partially in reaction to the trend (Ranbom, 1985), the number of families choosing the alternative has sometimes increased in response to the deregulation (Woltman, 1989).
            However, because the growth of the home education movement has been so rapid, it may have caught some policymakers and state-level lawmakers off guard (Cibulka, 1989; Goldberg, 1988).  The reaction of many governmental entities has been to apply–or, misapply–the new definition of educational quality to the “problem” of home education.  Unfortunately, willy-nilly applications of the output-based assessment model may prove to be devastating to home educators.
 
         Measuring Educational Quality
 
            One weighty responsibility of state-level lawmakers and boards of education is the assessment of educational quality.  Those in such positions are concerned–and charged by law–with assuring that each student in the state receives the best possible education.  However, what exactly should be measured depends significantly on the definition of educational quality being used.  Those in education charged with the monitoring and improving educational quality, and those who are affected by the resulting laws and regulations must gain a keen awareness of how the new definition of educational quality will affect–possibly adversely–the practice of home-based education.
            As described above, the definition of educational quality now in vogue focusses on outputs.  Most states have chosen to examine output though increasing reliance on test scores (Klicka, 1988b).  Although there are many who now criticize the expanding role of testing in education, tests do serve many positive functions; for example, helping to diagnose learning problems, identifying areas of weakness, and documenting student progress.  Unfortunately, little thought has been directed toward how test scores can properly be used as a measure of educational quality especially in the context of home education.  Legislatures and state boards have often hastily adopted the “customary” solutions–enacting statutes and regulations that require a specific level of student performance on a standardized test (Cizek, 1988).  These requirements are often counter to what even experts in the testing profession consider to be sound practice.
            The fundamental error in the new wave of laws and regulations is that norms are (mis)used as standards.  For example, in Virginia and West Virginia, students in home-based educational programs are required to achieve at least the 40th percentile on a standardized achievement test; Minnesota requires achievement at or above the 30th percentile; Oregon demands achievement at only the 15th percentile; Colorado, only at the 13th.  Different standards are upheld in Arkansas where students must attain scores corresponding to no lower than eight months below grade level, and in Tennessee where home educated children must not fall more that three months below grade level (Klicka, 1988b).
            The basic flaw in the regulations mentioned above is clear:  Any standard that is based on norms (e.g., percentile scores, grade equivalent scores, etc.) causes the student in a home-based educational program to be judged not on his or her performance alone, but on his or her achievement in relationship to a group.  It is especially troubling that many states have tied individual student test performance to approval of the home-based alternative. This serious flaw can translate into several perplexing situations for home educators.
            The most serious result of using any norms-based standard is that a given percentage of students will fail to meet the standard automatically, regardless of their ability and no matter at what level the standard is set.  For example, if the standard of performance for approval of the home-based programs is set at the 40th percentile, 40% of the students in the total group are likely not to meet the standard–a result that is guaranteed because a norm is (mis)used as the standard.
            It should be noted that much research has shown that home education students tend to perform above average on various measures of achievement (Ray, 1989b; Wartes, 1988).  One analyst has noted that the “evidence that [home schooling] results are at least equal to and in many cases superior to that produced by the public schools is quite abundant ” (Cibulka, 1989, p.7).  Therefore, compared to the total population of grade-level peers, a subpopulation of home education students would be likely to obtain a mean score on an achievement test that exceeds the total population mean.  Accordingly, it is likely that less than 40% of the home school subpopulation would actually fail to meet a standard set at the national 40th percentile.  However, this result does not excuse bad policy.
            The policy of using norms as standards is in strict contrast to guidelines for sound measurement practice (AERA/APA/NCME, 1985).  Nationally-prominent testing experts William Mehrens and Irvin Lehmann state the matter succinctly in their textbook on testing:  “Perhaps the greatest mistake is to interpret norms as standards.” (Mehrens & Lehmann, 1984, p.316).  The Iowa Tests of Basic Skills Manual for Administrators is equally clear in its denunciation of the practice, stating:
            In the use of test results for instructional appraisal, it is important to recognize that a norm is only a description of average achievement.  It should not be considered as a standard or as an indication of what constitutes `satisfactory’ achievement.  (Hieronymous & Hoover, 1986, p. 34).
 
            A second problem with regulations that promote the use of norms as standards is that they fail to take into account a student’s progress.  The following scenario is illustrative:
            A student who has attended the local public school and test results that show a history of performance at the 15th percentile.  The student’s parents decide to remove the student from the local school and begin a home-based educational program.  The student’s performance, as a possible result of the home-based education, begins to improve, reaching, perhaps, the 20th or 25th percentile.
In this scenario, and using performance at the 40th percentile inappropriately as a standard, the student fails to meet the mandated standard of performance.  What is especially disturbing about this scenario is that the student fell short of the standard despite what, by most education professionals, would be called a significant gain.
            A commentator on proposed changes in one state’s home education law reasoned similarly.  Reverend Clinton Birst noted that new regulations in South Dakota must take the circumstances of individual students and families into account, asking, “What does one do if a child had previously been in a public school and not done well there either?” (Woltman, 1989, p.15).
            Third, norms-based standards discriminate against those students who are most likely to benefit from home schooling.  Research cited previously has shown that home schoolers outperform, on average, their institutional counterparts.  Home schooling and its characteristic one-on-one, guided, instructional methods is particularly well-suited to the needs of the low-achieving student who often benefits from the individual attention (Good & Brophy, 1987).  The low-achieving student, who sometimes “falls between the cracks” in a formal school setting, often flourishes in the home school environment.  If rigidly enforced though, new norms-based regulations effectively deny home school instruction to such students.             Fourth, norms-based standards discriminate against home schooling families generally.  In the field of educational assessment, rigid percentiles are rarely, if ever, used to fix a passing standard.  Such misuse would certainly cause a public furor if applied in traditional educational settings.  (Consider the outcry that would ensue if students generally were required to achieve at the 40th percentile in order to pass from grade to grade.)  Any proposal that resulted in nearly half — or any fixed percentage — of the students failing each year would be sufficiently ridiculed that it would never receive serious consideration.  Yet similar proposals are seriously considered–and enacted–for application to the home-based educational programs.
            The discriminatory effect of norms-based standards applied to the home education realm is made obvious when the “remedies” for failure to meet the standards are considered.  For example, it is often suggested that home education students who fail to meet the standard be placed back into the local public school.  Would policymakers tolerate the reverse of this situation: i.e., Should parents whose students fail to meet the standard in the local public school be required to home school their children?
            Fifth, norms-based standards could have the effect of driving home school curricula toward greater similarity to the public school curriculum.  It has already been shown that testing does, to some extent, “drive” the curriculum (Porter, 1978).   There is reason to suggest that the driving effect would also be realized in the area of home education.  For example, a typical school would, for educationally-sound reasons, choose an achievement test to match its curriculum, although it has been noted that the actual “match” may not ever be perfect (Floden, Porter, Schmidt, & Freeman, 1978; Haney, 1985).  If a home school family were to select and follow a different curriculum than that used by the public school (as is likely), the degree of match between curriculum and test would probably be reduced even further.  Thus, the student educated at home could appear to be achieving less well, using the same test as used in the school, than if the family had followed more closely to the local school curriculum.
            In order to perform as well as possible on a state-mandated test, home educators would likely be influenced in subtle ways to align their curricula as closely as possible to the local school’s.  This prospect certainly defeats three of the primary reasons that families choose home schooling — the desire to tailor subject matter to the needs, pace, and interests of their students, the desire for increased flexibility in choice of teaching methods, and the desire to gain more direct influence over the content of their children’s education.
            This threat to the viability and identity of home education as an educational alternative is noteworthy.  Two apparent trends in American education this decade have certainly been the desire to widen the scope of educational alternatives in order to promote educational quality, and the desire to enact mandated educational assessments in order to promote educational accountability (Cibulka, 1989).  In the case of home education, the latter desire may have consumed the former.
 
Conclusion and Recommendation
 
            Despite the serious problems associated with misusing norms as standards, the trend continues.  There are many possibilities for courses of action that can alleviate some of the problems associated with the misuse.
            As the “conservative wing of the Social Science party” (Glass, 1986, p.9), measurement specialists often eschew nasty entanglements with applied policy debates.  However, it is appropriate that professionals in the area of testing seek to exert their influence to enlighten policymakers and correct misapplication of their methods. 
            Second, home educators should become knowledgeable about applicable testing requirements and policy in their states.  Information should be gathered concerning what standard is used to measure educational quality and how the standard will be applied.  In some states, home school families may be allowed to participate in the choice of which test to use with their students.  In those cases, it will be especially important for parents to become familiar with the various tests so that they might select the one that is the best match for their curriculum.
            Third, local school districts and state-level education personnel can seek to work with home educators toward the common goals of better intergroup communication and increasing educational quality.  The assessment of home education students should be marked by less parental and professional finger-pointing.  “Entrenchment, hyperbole, and lack of communication [have] clouded this issue for the past several years,” according to a recent report issued by the National Association of State Boards of Education (NASBE, 1988, p. 3).  The same report also noted that “there is a lack of communication and distrust between professional educators and home schoolers” (NASBE, 1988, p.3).  These succinct appraisals are probably quite accurate.  Calls for cooperation (Lines, 1985, 1987) are appropriate.
            For parents and professional educators, several possible outlets for expression of the needed mutual commitment to communication and education exist.  Home school parents can take steps to allay public concerns about educational quality and commitment through frequent and friendly contact with state and local educational leaders.  As one researcher on home education has noted, “if there is one avenue which has led to the broader acceptance of home schools as a reasonable way to educate children, it is the energy that home school parents express to others…” (Knowles, 1988, p.11).  Public educators also need opportunities to view firsthand the viability, feasibility, and benefits that can result from home education.
            Public education professionals can also develop “fruitful partnerships”  (Holt, 1983) with home educators–actively seeking innovative ways of helping parents to discharge the serious responsibility of teaching their children.  Such innovations might include development, publication, and distribution of materials designed to inform parents of appropriate instructional techniques, available curricular resources, and proper interpretation and use of test results.
            Governmental entities can also seek to develop relationships with home educators.  For example, the NASBE report on home schooling suggests that “State boards should review existing channels of communications and establish procedures for dialogue between public educators and home-schooling parents so that when problems arise there is a way to resolve them” (NASBE, 1988, p.2).
            Finally, research evidence indicates, overwhelmingly, that parental involvement in a child’s education “in almost any form appears to produce measurable gains in achievement” (Henderson, 1988, p.149).  While these studies have primarily examined the effect of parental involvement in the public school setting, parental desire to positively affect the educational process undoubtedly motivates parents to choose home education (Knowles & Hoefler, 1988) and affects its results (Lines, 1987).  Educators who truly seek the highest quality education for all students might do well to begin thinking about how to tap the wealth of parental concern represented in home educators.
 
        References
 
Adams, J. (1984, September 15).  Home schooling: An idea whose time has returned.  Human Events, pp. 12-15.
American Educational Research Association, American Psychological Association, National Council for Measurement in Education (1985). Standards for educational and psychological testing.  Washington, DC: Author.
Cibulka, J. G. (1989, April).  State regulation of home schooling; A policy analysis.  Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
Cizek, G. J. (1988).  Applying standardized testing to home-based educational programs: Reasonable or customary?  Educational Measurement, 7(3), 12-18.
Feinstein, S. (1986, October 6).  Shunning the schools, more parents teach their kids at home.  The Wall Street Journal, pp. 1, 21.
Floden, R. E., Porter, A. C., Schmidt, W. H., & Freeman, D. J. (1978).  Don’t they all measure the same thing?: Consequences of selecting standardized tests (Research Series No. 25).  East Lansing, MI: Michigan State University, Institute for Research on Teaching.
Glass, G. V. (1986).  Testing old, testing new: Schoolboy psychology and the allocation of intellectual resources.  In B. S. Plake and J. C. Witt, The Future of Testing (pp. 9-24).  Hillsdale, NJ: Lawrence Erlbaum.
Goldberg, K. (1988, May 4).  Iowa will shelve attendance law for 1 year.  Education Week, p. 15.
Good, T. & Brophy, J. (1987).  Looking in classrooms (4th ed.).  New York: Harper & Row.
Haney, W.  (1985).  Making testing more educational.  Educational Leadership, 43, 4-13.
Henderson, A. T. (1988).  Parents are a school’s best friend.  Phi Delta Kappan, 70, 148-153.
Hieronymous, A. N. & Hoover, H. D. (1986).  Iowa test of basic skills, manual for school administrators, levels 5-14.  Chicago: Riverside.
Holt, J. (1983).  Schools and home schoolers: A fruitful partnership.  Phi Delta Kappan, 64, 391-394.
Klicka, C. J. (1988a).  Home schooling: The return of a traditional educational movement.  Religion and Public Education, 15, 88-96.
Klicka, C. J. (1988b).  Home school statute chart of the 50 states.  Great Falls, VA: Home School Legal Defense Association.
Knowles, J. G. (1988).  Introduction: The context of home schooling in the United States.  Education and Urban Society, 21, 5-15.
Knowles, J. G. & Hoefler, V. B. (1988, April).  An exploratory case study of how parents think about developing enhanced parent-child relations in home school settings.  Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
Kohn, A. (1988, April).  Home schooling.  The Atlantic, pp. 20-25.
Levin, H. M.  (1978).  Educational performance standards:  Image or substance?  Journal of Educational Measurement, 15, 309-319.
Lines, P. M. (1984).  E. C. S. Issuegram #49: Home Instruction.  Denver, CO: Education Commission of the States.
Lines, P. M. (l985, May).  States should help, not hinder parents’ home-schooling efforts.  Education Week, pp. 24, 17.
Lines, P. M. (1987).  An overview of home instruction.  Phi Delta Kappan, 68, 510-517.
Mayberry, M. (1988).  Characteristics and attitudes of families who home school.  Education and Urban Society, 21, 32-41.
Mehrens, W. A., & Lehmann, I. J. (1984).  Measurement and evaluation in education and psychology.  New York: Holt, Rinehart, and Winston.
Naisbitt, J. (1984).  Megatrends: Ten new directions transforming our lives.  New York, Warner.
National Association of State Boards of Education (1988).  Home schooling.  Washington, DC: Author.
Pitman, M. A. (1986).  Home schooling: A review of the literature.  Journal of Thought, 21(4), 10-24.
Porter, A. C. (1978).  Relationships between testing and the curriculum (Occasional Paper No. 9).  East Lansing, MI: Michigan State University, Institute for Research on Teaching.
Ranbom, S. (1985, May).  Four states, feeling pressure, loosen home-schooling rules.  Education Week, pp. 1, 14.
Ray, B. D. (1989a, March).  An overview of home schooling in the united states: Its growth and development and future challenges.  Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
Ray, B. D. (1988b).  Home schools: A synthesis of research on characteristics and learner outcomes.  Education and Urban Society, 21, 16-31.
Rothman, R. (1989, November 8).  States turn to student performance as a new measure of school quality.  Education Week, pp. 1, 12-13.
Van Galen, J. A. (1988).  Ideology, curriculum, and pedagogy in home education.  Education and Urban Society, 21, 52-68.
Wartes, J. (1988).  The Washington home school project: Quantitative measures for informing policy decisions.  Education and Urban Society, 21, 42-51.
Woltman, J. (1989, November 8).  Home schooling doubles in N. D. since deregulation.  Education Week, p. 15.