Review by Douglas Pietersma, Ed.D.
NHERI Research Associate
Background
Dr. Hamlin’s (2024) review of existing research on home education highlights numerous methodological challenges. The rapidly changing landscape of home education complicates this effort. Limitations identified included “modest descriptive analyses, small nonrandom samples, and uniform operationalizations of homeschooling practice that treat homeschooled children as a homogenous group.” Hamlin notes that research is generally not “responsive to changes in the practice that have occurred in recent years,” and that many studies are poorly designed.
Methodology
The studies reviewed in this analysis were derived from the reference sections of three “comprehensive scholarly syntheses of research on home education.” Additional studies were identified through online searches of scholarly journals known to publish empirical homeschool research.
Hamlin focused exclusively on “quasi-experimental and correlational studies using statistical controls” in order to “describes methodological strategies that researchers can use to produce reliable evidence with a focus on addressing methodological limitations in existing work and being responsive to contemporary homeschool practice.”
Findings
From the review of these studies, Hamlin focused on conclusions in three areas: academic performance; social development, health, and welfare; and post-secondary and life preparation.
Academic Performance
Although Hamlin acknowledges that research on academic performance of the home educated is more positive than negative, much of the extant research does not provide strong causal conclusions. Most studies have the limitation of selection bias, and the nature of home education makes true experimental designs difficult to conduct. Naturally occurring experiments remain elusive without a way to randomly assign home educators to one group and others to a control group. Parental involvement almost certainly acts as a confounding variable, both in the case of the home educated, as well as those who are not.
Hamlin found few quantitative analyses completed since the year 2000. The most compelling study among these had a large sample size and used comprehensive control variables. It showed that home educated students generally score higher on the SAT math and reading sections. Other studies showed disadvantages to those who are home educated on other metrics, but due to smaller sample sizes, the results are less generalizable. Hamlin summarizes that “the overall quantitative literature on academic outcomes is also derived from cross-sectional data sources that capture a single point in time. Longitudinal work is altogether lacking. Moreover, this research does not account for different forms of homeschooling or even the number of years an individual child is homeschooled.”
Social Development, Health, and Welfare
Hamlin notes that social development and the wellbeing of children has been both the rallying cry of home education opponents and conversely the driving factors behind home education advocacy. The limited amount of “descriptive research” presents varying conclusion, but with flaws in the methodologies and further limited by small or unreported sample sizes. One study among 1,000 homeschool students showed that they are less likely to feed socially isolated. Another study addressed child safety and found no implications correlating to the home educated, but a limitation of this study was access to the home educated depending on state requirements. Finally, another study showed no difference in various self-reported categories of abuse, but the sample size of the study limits the generalizability of findings.
In general, Hamlin finds that “this literature generally suffers from a familiar set of consequential limitations, such as small homeschool samples, cross-sectional data sources, and no distinction of homeschooled students according to the number of years that they are homeschooled.”
Post-secondary and Life Preparation
Studies on this last area of concern were hampered by “small convenience samples.” One such study of limited generalizability showed that formerly homeschooled students scored higher in college calculus. Another small-sample study concluded no statistical difference on marriage and family metrics. Another, equally limited study, showed no difference in various social categories.
Some studies did present negative information on adults who were home educated, but the conclusions are in areas that might not seem like negative points to some home educating families. These include not attending or completing college as frequently as their public-school counterparts or a lower likelihood to continue volunteering in adulthood. As with other areas, these studies “tend to lack sufficient details needed to appraise the quality of the work, offering only vague descriptions of the research methods used.” They also do not distinguish different modalities of home education and are too small to apply control variables.
Conclusions
Hamlin’s comprehensive review of the current research led him to conclude that no current studies have generated causal estimates. He suggests that the next generation of research needs to address the limitations of existing studies while considering the ever-changing paradigms that now fall under the umbrella of home education. In line with this he makes the following suggestions.
Operationalize Homeschooling
Hamlin suggests that future studies need to consider the length of time that students spend in home education. Additionally, the methods employed in home education deserve consideration as variables rather than treating home education as a heterogeneous process. Lastly, there are copious contributing factors in the outcome of education and home education has inherent flexibility which allows a greater variety of these elements to be employed, which must be considered.
Data sources and statistical analysis
Hamlin noted that large sample sizes are needed to perform more detailed analysis on this topic, which will require concerted effort and investment. Some existing data sets might be useful for longitudinal studies, but since every state does not require testing, generalizable data would need to be extrapolated from such results. There would still be limitations that would be hard to overcome, potentially requiring independent observations to collect viable data on “social competencies, development, and other ‘soft skills.’”
Outcomes
The most obvious metric for comparing educational outcomes are test scores, but Hamlin notes that “many homeschool families have non-academic priorities,” which certainly impacts the emphasis of education efforts and subsequently on outcome. Identifying and evaluating what these families consider valuable is also critical. Cases of students with disabilities or exceptional abilities would need “different evaluative outcomes” to assess the results of home education.
Final Conclusion
Hamlin concluded that while it is unlikely that social scientists will be able to design studies that offer causal evidence on whether homeschooling is more effective than other forms of education, there are certainly opportunities to produce more rigorous work.
NHERI Comment
A key takeaway from this study is the need for the existence of such organizations as NHERI. It also helps to guide research design in order to perform increasingly refined research to better advise families, policy-makers, and current or future researchers.
References
Hamlin, D. (2024). Contemporary homeschooling, persistent debates, and the need for a new generation of research. Journal of School Choice, 18(4). pp. 697-712. https://doi.org/10.1080/15582159.2024.2422247