RECENT RESULTS FROM THE WASHINGTON HOMESCHOOL RESEARCH PROJECT
The Problem
Homeschooling is in a period of rapid growth. Legislators, educators, and the general public are being asked to learn more about this topic and to make decisions. A pragmatic approach to this issue would be to focus upon outcomes. If an educational method (however unconventional) works, then it deserves respect. This report describes some of those outcomes.
Other Research
Case histories abound in the homeschooling literature but there have been relatively few systematic studies reported concerning the academic achievement of groups of students. Many of the studies that do exist contain serious flaws. In one study (Alaska Department of Education, 1985) the results were positive for homeschoolers (students scored better than the national norm on the CAT). However, the response rate was only 53% — the conjecture is that parents of low scorers might have been more reluctant to send in their test results. Ray (1986) reviewed 11 reports relating to homeschool outcomes: while the overall findings were positive for homeschoolers, many of the reports were found to contain moderate to serious defects. The Washington Homeschool Research Project has produced earlier reports in this series (Wartes, 1987, 1988a, 1989). Again the general findings were positive for homeschoolers. Other tabulations of homeschooler test scores have been produced by public authorities in Oregon, Arkansas, and Tennessee (Wartes, 1988c). Because these tabulations are the least restrictive in terms of method of sampling, they are probably the most generalizable results available so far. While these results have been uniformly positive for homeschoolers (average to above average mean scores), they have not been incorporated into a formal report.
Only a few researchers have been able to correlated home school achievement with variables such as parent education level, contact with a teacher, and others. Most of the topics mentioned in Part 2 of this report were reported earlier (Wartes, 1988b) but were based upon a smaller Washington State sampling. Rakestraw (1988) found no difference in test scores according to parent education level, parent’s teacher certification status, or gender of student in a sampling of 84 Alabama home education students. A very recent national study (Ray, 1990) of 1,516 home education students who are members of the Home School Legal Defense Association also produced a negligible relationship between test scores and parent education level, and no relationship between scores and teacher certification status of the parents. It must be noted that all of these studies suffer from the self-selecting nature of the sampling process that was used.
Methodology
All of the test scores reported herein come from those homeschoolers who utilized any one of several testing services within Washington who cooperated with this research effort. Each service used the Stanford Achievement Test series, Edition 7 (Psychological Corporation, 1985), and forwarded a set of scores for each homeschooler who was tested between 1986 and 1989: these 2,911 sets of scores are the basis for the tabulation presented in Part 1 of the results. A subset of 877 homeschool students (those appearing in the 1987-1989 sampling whose parents also filled out a questionnaire) are the basis for the relationship topics described in part 2.
Definitions
Home schooling–For the purposes of this study, each student was considered a home school student if 75% or more of what the family considers to be “schooling” was provided by or conducted under the supervision of the parent(s).
Test–The test used in this project is the Stanford Achievement Test series. This series consists of the SESAT (Stanford Early School Achievement Test) for grade K, 2nd edition, the SAT (Stanford Achievement Test) for grades 1 to 8, 7th edition, and the TASK 2 (Test of Academic Skills) for grades 9-12, 2nd edition. 1986 norms were used.
Limitations
1. Achievement is measured only in terms of scores on the Stanford Achievement Test series. Other kinds of outcomes are not considered.
2. With respect to the results in part 2 of this report, there is some reason to believe that those parents who responded to the questionnaire may be somewhat different from those who did not. Thus the generalizability of these results beyond the group sampled is in question.
3. The input variables utilized in this research are based upon the responses by parents to a written questionnaire. There was no opportunity for direct contact between parent and researcher to discuss the items. Some degree of confusion or misunderstanding is possible.
4. Some of the results described in this report involve the combining of test scores from all eight levels of the Stanford Achievement Test series. In cases involving small sample size, this mixing of test levels was the only way that the data can be examined. This practice assumes that the norming group for each level of the test was similar — an assumption that has not been tested empirically.
5. All relationship described in this report are correlational in nature. Nothing can be inferred from this analysis suggesting one variable is causing another to change.
Findings
Part 1 – Test Scores
Composite scale scores for the 2,911 tests received during the 1986-1989 sampling seasons are tabulated in Table 1.
A summary of Table 1 is as follows: Of the 36 cells for each year,
The median cell is:
1986 68th percentile
1987 65th to 66th percentile
1988 65th percentile
1989 65th percentile.
With the median cell on the composite scales of the SAT series at the 65th to 68th percentile each year, it is evident that these students taught at home are not at a disadvantage compared to their conventional school counterparts.
Part 2 – The Relationship of Selected Variables to Academic Outcomes
Topic 1 — The relationship between parent education level and academic outcomes.
These homeschool parents appear to be somewhat above average in the level of their own education. Particularly underrepresented (only 2%) are those who have less than a 12th grade education. The primary parent (parent most involved in the education of the student) had, on
–This table is missing in this web page presentation; you may find the PDF format elsewhere on this web site with the table included–
Table 1. Mean scores for all home educated students in the 1986 through 1989 sampling on the six composite scales of the Stanford Achievement Test Series. (N=419 in 1986, N=783 in 1987, N=726 in 1988, N=893 in 1989)
average, 0.7 years less formal education than the other parent (t = 7.47, n = 845, p <.001). In a linear regression analysis of each composite test scale according to education level of the primary parent, other parent, and average of both parents, 55 out of the 105 tests produced a statistically significant positive coefficient of weak to moderate magnitude (significant at .05 or higher). These coefficients ranged from 1.30 to 4.50 NCE (Normal Curve Equivalent) per year of formal education attained by parent. No negative coefficients were produced. Where all test levels were combined a significant (p < .01) but weak positive relationship was shown. The equations that would best predict Total Complete Battery test scores as a function of parent education level were as follows:
For the level of education of the primary parent Y = 2.11X + 32.1
For the level of education of the other parent Y = 1.70X + 36.9
For the average level of education of both parents Y = 2.68X + 23.5
Where Y = Test Score in NCE
X = Number of years of parent education
Children in families where the primary parent (the one most actively involved in the education of the children) had 12 years of formal schooling or less, the mean Complete Battery Total score was at the 61%ile (n = 227). Where both parents had a high school education or less, the mean score was at the 59%ile (n = 106). In general, fears that children of less educated parents are at an academic disadvantage receive little support from this evidence.
Topic 2 — The relationship between contact with a certified teacher and homeschooler outcomes.
Only 15% of the parents and 7% of the students appearing in this sample had any contact with a certified teacher. Due to the small number, comparisons between those with some teacher contact and those with none were not possible. However, the 545 students who had virtually no teacher contact during the previous school year scored, on average, at the 68%ile on the Complete Battery Total scale. Students of parents who are certified teachers did tend to score better than students of non-teachers but this result lessened as the number of years of homeschooling increased and the evidence suggests that this difference may relate more to parent education level than to teacher certification as such. In general, these data do not provide support to the concept that teacher contact is important to success in home schooling.
Topic 3 — Level of structure and hours of schooling as they relate to academic outcomes.
Parents in this sampling rated their level of structure as an average 4.6 on a scale of 1 to 7 where 1=Very unstructured and 7=very structured. The number of hours of “formal schooling” tended to increase with student age and averaged 15 hours/week. Probably the most important result here is that neither hours/week of formal schooling or parent rated level of structure appear to have much relationship to academic outcomes. When “level of structure” was examined by linear regression on all levels of the SAT series, only 7/35 tests resulted in a positive coefficient and all of the seven were weak in magnitude. When all test levels were combined, 5/6 tests showed a positive relationship but of very weak magnitude. When “hours/week of formal instruction” was evaluated on all test levels, three positive and five negative coefficients (all were weak) resulted out of 35 tests. When all test levels were combined, only one composite scale in six, Total Listening, produced a relationship (it was negative but extremely weak). In general, these test data suggest little or no relationship between level of structure and academic outcomes.
Topic 4 — Is there any relationship between grade level and academic outcomes?
Analysis of variance revealed that very few mean scores earned by one grade level were significantly (alpha = .05) different from the means of the other grades. Only 10 of the possible 89 comparisons between means produced a difference that was statistically significant. Of the 10, nine favored the higher grade (the higher grade students out scored the students in lower grade(s). Caution is advised however because (a) five significant differences had their origin in one group of 69 first grade students who scored unusually low in Total Math, (b) most of the other differences were moderate in magnitude, and (c) with 89 comparisons it would be reasonable to expect some differences simply due to chance. No significant differences between grade levels were found on the Total Complete Battery scale. While an achievement test like the SAT measures only some of what is taught during the senior high years, the mean score (66%ile) of the 44 grade 9-12 students appearing here suggests that, in general terms, they are scoring well. Generally, this sampling produced little to no relationship between grade level and academic outcomes.
Topic 5 — The relationship between academic outcome and the length of time the student has been homeschooled.
For the 875 students sampled, the mean number of consecutive years homeschooled was 2.7. Of 35 linear regression tests performed on the various levels of the composite test scales, only two produced a statistically significant coefficient (.05 or higher). None were negative. When all test levels were combined, only one (Total Listening) of the six composite scales produced a positive coefficient of weak magnitude. Speculation by neither critics of homeschooling (concerned that the longer you homeschool the worse it will become) nor proponents (who have argued that it gets better) finds confirmation in this sampling. In general, test scores did not seem to change as the number of years of homeschooling increased.
Topic 6 — Does family income level have a relationship to test scores?
The 845 parents who provided information reported a mean, before tax, income in the $30,000 to $35,000 range. Less than 6% of these families described themselves as a two-income family. Where family income and scores on all levels of the composite test scales were examined by linear regression, 11 of 35 regression coefficients showed a positive relationship of weak magnitude. None showed a negative relationship. Where all test levels were combined, five out of six composite scales produced a positive coefficient of weak magnitude. The 72 students whose family income was reported to be less than $15,000/year averaged at the 62%ile on the Total Complete Battery scale. This sample produced results on the border between no relationship and a weak relationship when comparing family income to test scores. Students at the low end of the economic spectrum were scoring well. In general, these data were unable to provide any tangible basis for concern regarding academic achievement among homeschoolers based upon family income levels.
Topic 7 — Male-Female Comparisons.
Because sex identity of the student was asked only on the 1989 questionnaire, the sample size on this topic is lower than for most others. In general, the two sexes scored within the same range on the scales tested. The two exceptions were that girls out scored boys in Total Language (73%ile and 59%ile with an N of 127 and 114 respectively) and boys did better than girls in Science (86%ile and 72%ile, N of 100 and 106). It is interesting that while boys scored higher than girls in Science, the girls average was very respectable. It is also of interest is that there was no significant difference between the sexes in Total Math. It is currently of concern that girls often tend to do poorer than boys in this subject among regular school students. Potential concerns that either sex may be at a disadvantage compared to the other as a result of homeschooling are not validated by these data.
Topic 8 — The relationship of previous success in conventional schooling to current test scores as homeschoolers.
Among those homeschoolers who had previously been in a conventional school, a regression analysis revealed a positive relationship of moderate magnitude between previous parent-reported success in that conventional schooling and their test scores as homeschoolers. The general tendency is that these students remain at or near the same level when homeschooled. An interesting observation is that the overwhelming majority of these students were doing well in their previous conventional schooling. According to parent ratings, 41.7% were in the upper 20% of their class at the time of transfer to homeschooling; 88.7% were in the upper 60% of their class. A possible implication is that one reason why homeschoolers, as a group, do so well is that they were strong students to be begin with.
Topic 9 — Relationships involving religion.
The vast majority (98.7%) of this sample was affiliated with one subset or another of the Christian faith. Most common were those identifying with a nondenominational Christian group (42.3%) and those belonging to a conservative evangelical denomination (30.7%). On a five question Christian orthodoxy scale, 80.1% of the sample registered the strongest possible response (25 on a scale of 5 to 25).
When mean test scores of students were examined according to their parents’ religion, virtually all were in the average to above average range. In this sample, no differences in test scores were detected between those of different religious orientation. For those parents who rated themselves at the high extreme on the orthodoxy scale (25 out of a possible 25), student scores were quite credible (72%ile on the Total Complete Battery). In a Regression Analysis of test scores and the parent reported degree of religious content in their homeschooling, virtually no relationship emerged. Concerns by some (usually those of a more secular orientation) that homeschooled children of strongly religious families are, as a group, being under educated are not supported by these data. In general, on a secular measure (the SAT series) of educational outcomes, these homeschoolers appear to be doing quite well regardless of the kind or degree of their religious orientation.
This analysis did find that the degree of structure in the style of homeschooling is positively correlated with the degree of religious content but that the magnitude of this relationship was weak. In general, those who indicated greater religious content tended to be somewhat more structured. Exceptions to this generality were, however, not uncommon.
General Conclusions
If there is any one theme that seems common to the evidence presented here, it the is utter inability to confirm some of the concerns voiced by critics of homeschooling. As a group, these homeschoolers scored well on standardized achievement tests. The education level of parents does not appear to be a very good predictor of homeschooler success. The absence of contact with a certified teacher does not keep homeschoolers from doing well. Hours of schooling or level of structure does not seem to influence outcomes. Homeschoolers do as well in the upper grades as in the lower. Long time homeschoolers do not do worse the longer they homeschool. Family income is a poor predictor of student success. Females are not being held back relative to males. In general, those who did well in conventional schooling tend to continue doing well after changing to homeschooling. These homeschoolers did well regardless of the kind or degree of religious orientation of their families.
Due to the self-selecting nature of the sampling and difficulties with some questions on the survey, the generalizability of these results is in question. Replication by others is desired.
But these results should suggest caution to the critics who offer their thoughts without documentation. At least with this sample and on the topics mentioned here, the critics of home education have been wrong. There is a need, by those on both sides, to become more objective in the treatment of the home schooling debate.
References
Alaska Department of Education. (1985). SRA Survey of Basic Skills, Alaska Statewide Assessment, Spring of 1985. Juneau, AK: Author.
Psychological Corporation. (1985). Stanford Seven-Plus. San Antonio, TX: Author.
Rakestraw, Jennie. (1988). Home schooling in Alabama. Home School Researcher, 4(4), 1-6.
Ray, Brian D. (1986). A comparison of home schooling and conventional schooling: With a focus on learner outcomes. (Available from the National Home Education Research Institute, 25 W. Cremona St., Seattle WA 98119)
Ray, Brian D. (1990). A nationwide study of home education: Family characteristics, legal matters, and student achievement. (Available from National Home Education Research Institute, 25 West Cremona Street, Seattle WA 98119)
Wartes, Jon. (1987). Report from the 1986 homeschool testing and other descriptive information about Washington’s homeschoolers. (Available from Washington Homeschool Research Project, 16109 NE 169 Pl., Woodinville WA 98072)
Wartes, Jon. (1988a). Report from the 1987 Washington homeschool testing. (Available from Washington Homeschool Research Project, 16109 NE 169 Pl., Woodinville WA 98072)
Wartes, Jon. (1988b). The relationship of selected input variables to academic achievement among Washington’s homeschoolers, preliminary report. (Available from Washington Homeschool Research Project, 16109 NE 169 Pl, Woodinville WA 98072)
Wartes, Jon. (1988c). Correspondence regarding homeschooler outcomes from Oregon, Tennessee, and Arkansas. (Available from Washington Homeschool Research Project, 16109 NE 169 Pl., Woodinville WA 98072)
Wartes, Jon. (1989). Report from the 1988 Washington homeschool testing. (Available from Washington Homeschool Research Project, 16109 NE 169 Pl., Woodinville WA 98072)
Wartes, Jon. (1990a). Report from the 1986 through 1989 Washington homeschool testing. (Available from Washington Homeschool Research Project, 16109 NE 169 Pl, Woodinville WA 98072)
Wartes, Jon. (1990b). The relationship of selected input variables to academic achievement among Washington’s homeschoolers. (Available from Washington Homeschool Research Project, 16109 NE 169 Pl., Woodinville WA 98072
1 The authors of the two papers in this issue of Home School Researcher prefer to refer to the practice of home education with one word (e.g., homeschoolers, homeschooling). The traditional editorial practice of this journal has been to use two words (e.g., home schoolers, home schooling, home educators).
Leave a Reply
Want to join the discussion?Feel free to contribute!