Portals to Learning: A Case Study of Teacher Training for Literature and Project-based Learning
Home School Researcher, Volume 38, No. 4, 2024, p. 1-12
VP of Development, Portals, Saint Paul, MN, sdikkers@portalsacademy.com
Mandy McRaith
Data And Hub Support, Portals, Saint Paul, MN, info@portalsacademy.com
Stephanie Dikkers
Curriculum Designer, Portals, Saint Paul, MN, info@portalsacademy.com
Home School Researcher, Volume 38, No. 4, 2024, p. 1-12
Abstract
This pilot case study is design-based research in the form of a pilot study of the Portals support program for training teachers and providing resources and lesson plans to teach throughout the year. Portals is a scalable set of resources for parents and teachers to administer learning outside of traditional models. The study is exploring the first iteration of tools for teachers to proctor innovative learning with minimal training with provided scope and sequence, lesson plans, project ideas, supplemental digital media, and a call-in help center. The pilot included 17 teachers/parents at 13 locations with 34 students including one microschool for one school year. Multi-modal data included feedback surveys, interviews, and observation along with standardized test scores. All 17 teachers were able to administer a year of education using the model provided with perceived success and above average age–based grade equivalent test scores.
Keywords: Experimental, Education, Innovation, Literature-Based, Project-Based, Digital Media, Learning, Homeschooling, Microschooling.
Introduction
Much effort has been made to define education needs of the modern era. The 21st century skills (Partnership for 21st Century Skills, 2008) most commonly cited by the literature today include STEM (Science, Technology, Engineering and Mathematics), cross-cultural skills, collaboration skills, critical thinking, problem-solving, creativity and innovation (Kennedy & Sundberg, 2020) and even the rising call for diversity, equity, and inclusion (Mitchell & O’Brien, 2020). Significantly, the entire push for standards, standardized testing, and teaching to the test are also institution-wide changing efforts to improve education. Each trend is largely the result of expectations in employment and anticipated needs for changing economies alongside lower performing government schools in reading and mathematics. After decades of such efforts, results remain elusive at best, detrimental at worst, (Wigfall, 2023).
Government Schools
Not only are these higher cognitive skills not being acquired but as old methods are being adjusted or abandoned even basic skills are retracting at an alarming rate. The National Center for Education Statistics (2020), shows a drop off in basic skills matching the concern that higher level workplace skills are not being developed. Locally, in Minnesota, we are seeing the lowest scores in 30 years after massive transformation in the government schools toward ‘standardization’ and efforts to ‘close achievement gaps’ (Wigfall, 2022). We also see a need for measures in digital literacy (Lankshear & Knobel, 2008) that are not captured in standardized testing of computational math and non-fiction reading comprehension. In other words, each wave of reform has not produced promised results in basic math, reading, and overall literacy by any quantifiable measure. This decline is not a recent phenomenon but part of a downward trend that “has lasted more than a decade” (Wigfall, 2023). Public government schools, regardless of reform waves, seem unable to produce improved overall results via standards, testing, or increased national oversight and direction in order to ‘close gaps’ in learning between groups.
The design of the Portals pilot avoided the government model of schooling. Existing traditional government models have an extensive and comprehensive model of a structured experience that seeks fidelity with governing agencies and standardization for all students. This follows the Prussian model for factory schools (Alexander, 1919). All students experience the same setting, curriculum, pedagogies, and even, more recently, outcomes. The aforementioned national results are showing this model fails to educate in a way that satisfies parental beliefs and values, workplace demand, and/or learner outcomes. Further, individual technologies like mobile devices and personal computers become a challenge to ‘integrate’ into a standardized model that more easily adopts standardizing technologies like testing and textbooks. If everyone is held to the same standard and given the same resources, the system itself will struggle to create exceptions – including excellence. This hyper monogamy of structured experience is not only ineffective for most, but is challenged in the prevailing academic discussion calling for differentiation, flexibility, 21st century skills, and individualized educational plans.
Teacher and Administrative Professional Development
A number of studies attempting to equip educators towards performance within a defined, or standardized, set of expectations, starting with widely accepted work around teacher evaluation and training toward a ‘best practices’ (Danielson and McGreal, 2000), (Published in part by the Educational Testing Service). Danielson’s rubrics have been used for over twenty years to help develop teachers and are foundational in administrative efforts to build professional learning communities through evaluation (Bredeson, 2013), and even creating standards for administration (Engler, 2004). Interestingly, teacher growth can also be tied to compensation strategies used by administration (Odden & Kelley, 1997). This is an effort to improve student learning by increasingly standardizing the institution from the top down. The underlying assumption of this work is that the standards are accurately representing outcomes desired.
For overall improved learning by current measures of standardized testing then becomes an issue of training, equipping, or increasing ‘teacher capacity’ (Kelley & Shaw, 2009) – toward teaching government standards. Where a number of strategies for training are developed by looking at the administrative practices of successful school settings, we can also look outside of school settings for how non-government educators are developing teacher capacity. As an institution, the government mass schooling is not necessarily achieving its own goals as aforementioned measures of standards and testing.
Homeschooling
This is not true of all sectors of education, however. In the homeschool community, which has grown exponentially in the last decade, all research done so far supports that home-educated students continually have performed at least as well if not better than their public educated peers not just in academics, but in every other measured metric as well (Ray, 2017). Giving rise to remembering cultural traditions of more effective educational times (Hirsch, 1987), less intense learning models that value a child’s play (Hirsh & Golinkoff, 2008), and homeschooling itself. This represents a community with no formal training structure, yet exceptional overall results. This leads one to ask what exactly is so effective about homeschooling as a model or framework for learning outside of government solutions. Further, promising research communities show positive results in literature, project, and digital media use – all of which are arguably being leveraged by the homeschooling community by intent or incidentally. Homeschooling is simply successful in the same endeavor that the government schools are reeling in.
Perhaps solutions lie with successful educational efforts, and not from the framework of unsuccessful efforts. To equip any design for teachers we can leverage the shared cultural heritage that homeschoolers are remembering, the tools they are using, and the functional cultural adaptations they are making – either knowingly or unknowingly. And, of course, are there practices and resources that may prove more effective than the recent fads in government schooling outlined above. Can they be used at scale? What would a nation of homeschoolers even look like? These questions are already convincingly being answered in the homeschool movement and the surrounding resources that are being built for that community, at scale, and proving effective collectively (Ray, 2017).
Design-Based Research
However, any real change in education does require actual trials, actual constructs, based on key design principles. These cannot be half measures, or things that further separate results from clear design choices. Such efforts must show viability, or proof-of-concept, that informs others in their efforts. Quest to Learn (Tekinbas, et al 2010), for example, is a whole school design effort where reporting has a balance of their design approach and what they saw working. Likewise, the study of educational digital media (Squire, 2008; Steinkuehler & Williams, 2006) examines perceived impact in learning via qualitative data of interviews, observed activity, and learning outcomes and products.
Notably, teacher training was a bottleneck for all three of these endeavors. Where the models showed promise, the ability to equip and train teachers to function within those models was restrictive of growth. This study is within this tradition of design research, but instead of targeting student experience itself, Portals design provided teacher resources to create student experiences – or build teacher capacity with no direct administrative training, compensation, or evaluation process. Teacher training designs are targeting the bottleneck found in prior research efforts that prevented scaling up successful design efforts. Portals is an initial full program, year-long, designed to equip new teachers using successful homeschool resources, methods, and to gain teacher capacity in instructional practice.
Teacher Training
The focus for this study is on the implementation and teacher reaction to Portals as a program. The authors seek to provide tools for educators to provide innovative learning based on homeschooling and academic promise – effectively using the resources themselves to support the confidence of practice directly. The Portals pilot covers all subjects K-8, designed around key homeschooling design commonalities, and providing teacher support/training that can be used at scale at a state or national level. What is lost in a standardized experience for the student, is gained in teacher autonomy, agency, and flexibility so that they can function without costly intervention and training.
Experiments in innovative approaches reveal that while students can learn a fair amount on their own, their learning potential is limited without the support of a teacher or at least a ‘More Knowledgeable Other’ (MKO) as defined by Lev Vygotsky (in McLeod, 2023). Even in Montessori practice, a core element of the educational design is a “Trained teacher” to support child development (National Center for Montessori in the Public Sector, 2023). Training then becomes the primary challenge for effective learning even at the most flexible end of the spectrum. Where flexibility for the student is relevant, some kind of structure is relevant to teaching and learning as provided by an attentive MKO figure. This may be a teacher, an aide, or a homeschooling parent – as long as they know the student and make choices to advance learning locally.
Particularly, this study explores a design-based research pilot for educators that provided embodied experience via curriculum and pedagogy using successful predominant homeschooling practices for a structured experience via 1) Literature, 2) Project-Based Learning, 3) use of Digital Literacy, and 4) the development of Beliefs and Values; all elements of a growing and successful home education movement. Portals implementation and the data gathered in the pilot helps to address questions of viability and scalability of these elements: What is the viability of support programming that equips the local MKO to deliver a year of content successfully? Can it be done and replicate some of the results of autonomous homeschooling, and/or innovative academic efforts that have documented success? In short, is Portals a viable tool and is it scalable?
Literature Review
Structured Experience via Literature
Homeschooling flexibility varies by family, of course, but we do see common threads. Even a quick review of curriculum used, Devitt shows a wide variety of approaches including classical literature, prepared curriculums, unit studies, projects, or self-paced programs, (Devitt, 2023). Most of the fifty reviews include content to provide structure with the help of curriculum that many homeschooling families can purchase and switch out if the student is not responding well to one or the other. So, both the MKO and the curriculum itself provide a designed structure within a flexible environment or a natural differentiation (Tomlinson & McTighe, 2006). This malleability of structural resources is a key advantage (over government schooling that selects resources for entire districts or states) – like Montessori, where the child’s interest and proclivities can direct the MKO’s planning and provision of resources. Students learn more in an environment designed to foster discovery than they do happenstance and teachers are key to creating these discovery rich situations (Wood, Bruner & Ross, 1976).
Content is often delivered through textbooks that structure experience, but little research has been done on the effectiveness of the textbooks as a designed artifact and what has been done does not provide convincing evidence that textbooks are better than primary resources (Klymkowsky, 2007). As per Devitt’s review above, many homeschooling curricula have shifted away from summative textbooks to primary sources, classic literature, and authored novels. Thus literature-based curriculums have been developed, particularly in the homeschool community, to deliver content to students through “living” books rather than textbooks.
Stories, particularly well-written ones that we generally label as classics, have the ability to meet many of the criteria for learning (Hargraves, 2021). These benefits give rise to literature-based curriculums that can include books from a wider range of people and perspectives than is likely to be found in limited and singularly focused textbooks. This would not be a ‘one size fits all’ solution, but a framework in which the MKO still has autonomy to make daily changes or swap out texts as fitting for the local setting. Further, they can integrate digital media easily to complement texts in the form of video, audio, game, and even mobile experiences (Squire & Dikkers, 2009).
Structured Experience via Project-Based Learning
MKOs also are vital to identifying what students can do on their own and what they can do with support within “Zone of Proximal Development” (Vygotsky, 1987) and providing that support in a timely manner that challenges without reaching a point of frustration. Accordingly, effective practice in education can occur when a teacher monitors and adjusts resources to fit the learner’s optimal learning range – not too hard to be frustrating, not too easy to be dull.
Projects, games, and interactives are exceptional in that they are student driven and allow discovery in process (Bruner, 1961), and also allow for a variety of activities for different dispositions toward learning. Howard Gardner’s work around multiple intelligences supports an idea of individualized, and differentiated approaches to learning (Gardner, 1993), which can be facilitated with multiple interactive experiences. Teachers need multiple ideas, multiple options, and freedom to determine impacts on learning.
Students can self-select the degree to which they ‘lean in’ to a project. Learning is socio-cultural and students can develop faster when they can process their discoveries with others (Resnick, 1991; Wertsch, 1998). These discussions can provide the student with language for their observations, assist in meaning making, and allow knowledge to be internalized (McLeod, 2023). The MKO’s role then is also affective in nature, as they monitor learner reactions and create discourse around a subject and the space in which it is done.
Lave and Wenger (1991) explored a similar idea of ‘situated learning’ where learning is essentially always in the context of other people within a space and with designed tools (Hutchins, 1995) within that space. Learning is not just ‘easier’ within a situation, but it is always within a situation, always utilizing tools designed by another. MKO’s can build learning with vision, flexibility, and passion for an area of learning to make learning into an adventure or a ‘quest’ (Dikkers, 2016). Learning is more effective when knowledge and skills are acquired in real-life situations and communities of practice in contrast to the decontextualized activities found in many classrooms (Papert & Harel, 1991; Squire, 2021). This implies a needed emphasis on professional discourse with experts, field trips, and video calls with practitioners within a learning area – or at least allowing time for the MKO to arrange such conversations.
Conversation around projects allows for discourse and a kind of play as a learning modality, “Play isn’t the enemy of learning, it’s learning’s partner. Play is like fertilizer for brain growth. It’s crazy not to use it” (Brown, 2009, p. 101). Play has been shown to support healthy socialization, foster adaptability, vocabulary, increase brain development, and inspire optimism and creativity (Brown, 2009) or problem-solving cognition (Lave, 1988), or situational learning within complex semiotic systems (Gee, 2004). Use of digital media can and does amplify learning with the right context (Squire & Dikkers, 2012). Games show promise of being effective in developing many 21st century skills: strategic thinking, planning, communication, application of numbers, negotiating skills, group decision-making and data-handling (McFarlane et al. 2002 cited by Shrier, 2016, p. 24). Portals then, is designed to have projects and play as a key feature in the tools designed for the MKO’s to allow for development, situated learning, and play.
Structured Experience via Digital Literacy
As we move further into the digital age, sources for content are not limited to physical books or localized repositories of knowledge (Levy, 2015). Computer technology is a cultural tool (McLeod, 2023) and requires native exposure (Prensky, 2001) for fluency. Digital technology can support access to real-life situations and communities of practice as students are not limited to the experiences and experts available in their immediate environment but can learn from people and situations around the globe (Schrier, 2016). Indeed, entire curriculums can be built using this model too (Barab, et al, 2007). To have a structured experience with digital media however, MKO’s are still needed to support students in developing the skills needed to access and utilize these digital spaces – they must have the resources to deliver such a non-typical model of learning even if they are not ‘digital natives’. As their context changes due to personal or societal changes, so will the digital needs and expectations. Therefore, digital literacy is better seen as “a condition [to be maintained] than a threshold [to crossover]” (Martin 2006 as cited by Bawden, 2008, p. 28). At best, the MKO doesn’t ‘assign’ technology, but instead uses it as a tool for learning other content (Dikkers, 2015) as it is used in the workplace.
The American Psychological Association reports more digital media use and less print media reading; (Twenge, Martin, & Sptizberg, 2019). This may be read as a net negative trend for learning. However, some studies indicate the exact opposite effect. Notably, exposure to popular games can actually lead to more reading, higher level reading, and investigation (Steinkuehler, Compton-Lilly, & King, 2010). Reading is reading, regardless of media format, and comfort with reading leads to increased willingness to read more (Gee, 2007).
Structured Experience via Beliefs and Values
It is very challenging if not totally impossible however to pass on content without also passing on beliefs and values. Homeschooling researcher Brian Ray makes the case that “all education and schooling is the teaching, training, and indoctrination of children” (Ray, 2021). Likewise, progressive education researcher Michael Apple argues that “All education is a political act” (2013). Efforts to avoid particular beliefs or conduct study outside them would be to necessarily conduct research outside of education itself. Where there is minimal research about conveyance of values, they remain of primary importance to parents in selecting educational settings and evaluating educational effect. We approached our research agreeing that the passing along of beliefs and values was an essential aspect of any intervention study. Therefore, content resources must be chosen with care and must be up for negotiation and continual change as a community works together to determine what knowledge, skills, beliefs, and values are and will be most helpful to the next generation.
All structured content, all projects, and all media formats contain embodied values and beliefs of the designers, the tools created for MKO’s will have the skills, talents, and training of the designer baked into the tool itself (Hutchins, 1995). If this is the case, the MKO must trust and agree with the designer in order to be willing to share or pass along the act of education. Any study that disregards the importance of aligned values undervalues its relevance to the parent of the learner. Whatever intervention is designed for homeschool trial study, must also align with the values and beliefs of the participants of that study for results that focus on the design itself.
Further, upon summative interviews, we gathered comments and indicators that MKO’s perceived their values were effectively being passed on to the learner and that the overall experience allowed for variation and flexibility to meet their perceived need along those lines.
Portals was designed with these four structured experiences as part of the design of educational experience. Current research suggests to us that K-12 education for today’s society should use literature, project-based approaches, digital tools, and support the beliefs and values of the MKO’s. If these elements are combined effectively, they should provide expertise embodied in the support materials for an MKO to homeschool with positive perceived and actual outcomes.
Where literature-based, project-based, and multi-modal learning models show promise in the research, what is an educator to do if they do not have the background, exposure, and/or training to deliver these approaches viably? Again, the literature points to the need for design around supporting and equipping many more educators that use successful tools and approaches to learning.
Is it then possible to combine structured experiences of literature, projects, digital media, that provides for transmission of MKO beliefs and values into a working model of educational support resources so that a parent with reservations around homeschooling can successfully complete a year with increasing perceived agency and competence. Portals is the designed instrument to explore the extension of this research and if such an informal support system can viably support teachers.
Design Model and Method
The design of the Portals program is an attempt to address the question of both viability and scalability. Method of study employs a design-based research study (Barab & Squire, 2004) with a mixed-method analysis approach. We invited participants that were interested in homeschooling and micro-schooling with designed support materials around the research questions.
Participants
Pilot participants were recruited voluntarily through advertisement in the Minnesota Association of Christian Home Educators’ Newsletter and word of mouth. We offered free curriculum planning, lessons, project ideas, and a call-in center as needed for support. Notice went to over 5,000 recipients and participants self-selected from that pool as in need of support. This is a discrete population that are already interested in homeschooling and subscribed for more information. These are MKO’s willing to try a new educational setting, but willing to have the help of designed resources for learning. There may always be those interested that are not part of the email campaign, limiting the possible participant base.
All student information was kept by the parents as we only needed to know age and MKO practices in delivery of learning. Participant names were used in conversation, but not required for data collection and documentation of interviews and comments. All data was stored in password protected files and kept in a separate secure hard drive for long term storage.
There were 13 sites total with one microschool. Participants included 17 unique families and 34 students. Site sizes ranged from 1 to 6 students, with an average and mode of 3 students per site. There were 14 girls and 20 boys. There were 4 in kindergarten, 7 in first grade, 5 in second grade, 1 in third grade, 6 in fourth grade, 4 in fifth grade, 3 in sixth grade and 4 in seventh grade. Half were new to homeschooling and half were veterans at homeschooling.
All but two of the thirteen sites had students of different ages. Within the developmental levels, the subjects of Bible, history, science, and literature were shared. Math and language arts however, necessarily had to be individualized. Several of the sites with more than one developmental level did end up cutting out or combining some of the subjects to stretch their time.
In exchange for their participation and feedback, pilot participants were given lesson guides, instructional resources, texts, and laptops. Participants were responsible for project and supplemental materials and programs. After an initial meeting, families were sent an agreement with the terms of participation that agreed to weekly feedback of the program, sharing standardized test scores (without student identifying information), and two larger interview sessions conducted in online video calls. Testing is required by Minnesota law for homeschooling families.
Resources Designed
Curriculum resources and instructional materials were selected and developed for three developmental levels: A (5 through 8 years old), D (9 through 12 years old), and G (12 through 14 years old). There were 19 students in level A, 13 students in level D and 2 students in level G. Five sites had students in only one level, while the remaining eight sites had students in two levels.
Lessons covered ages 5-14, 26 courses, for 144 days (3,744 lessons) and 36 Friday Activity pages with 4-5 ideas per week. Each lesson had a reading selection, ‘think’ prompts to start conversation around the content (11,232 prompts), and project ideas for each lesson (3,744 project ideas). These created structured activity via literature, discourse, and projects, integrating digital media and opportunities to discuss beliefs and values in relation to the content.
Design Guidelines
Projects were designed to amplify various learning styles and types of situated experiences. Planners were given a rotation of types of projects including: Active, Building, Crafting, Programming/Computer Use, Play, Create, Music, Interview, Writing, Cooking, Experiment, Field Trip, Skill Building, Service, Maps/Charts/Graphs, and Gaming. Supplemental ideas and options were also provided for MKO’s to customize to learner preferences. Field trips, interviews, and outings could also make use of mobile media tools for learning (Dikkers 2014).
Subjects included Bible, History, Literature, Writing, Science, and Math. We integrated non-core subjects into the curriculum (as noted above) including “spelling” and “vocabulary”. Our expectation was that they would improve with increased reading, not deconstructed attention outside of situated cognition.
Before the pilot program officially began on September 6, 2022, each site was sent all the instructional materials and trained in their use via in-person and/or online conference calls. Sites were sent their digital lesson guides via email. Guides laid out the literature, inquiry, and project-based assignments for each of the core subjects: Bible, History, Literature, Science, Language Arts and Math. Each lesson plan was structured as “Read, Think, Do” Lessons emphasizing literature and projects as the core pedagogical design. Digital lessons also allowed hyperlinks to digital media alongside reading selections and projects. Lessons and projects were built around Sonlight, Apologia, and MathUSee which are part of the existing homeschool curriculum.
The digital guides and physical instructional guides were set up for a four-day week allowing for MKO’s to use the fifth day for situational learning. The fifth day of each week was designed for further exploration, thematic investigation, field trips, family events, community events, and unique student interests – or flexible time for discourse experiences and/or field trips for situated learning experiences. In addition, each Friday a document was emailed to participating sites detailing the theme of the week often related to the season or upcoming holiday. The document also provided weekly updates and lists of upcoming projects and experiences that required advanced preparation.
Data Collection
Data and feedback were regularly collected from sites through weekly Google surveys and quarterly online video conference calls. Additional feedback was gathered via participant-initiated phone calls, emails, and video calls in a phone log. On-call support was available to sites from 8 a.m. to 4 p.m. Monday through Friday. Educators and students could contact the Portals Learning Facilitator during these hours via phone, text, email, or video chat.
At the end of the year, the pilot participants provided standardized testing scores and completed a comprehensive end of the year survey and interview sent out via Google forms. Initially, we provided key observations that would confirm or deny the viability of the intervention. The research team was instructed to always look for indicators of our design goals. We used the following indicators for qualitative points for observation, interview, and exit-comments:
Provision of structured learning via literature, projects, and digital media:
- Was the content age appropriate?
- Easy to complete in a reasonable amount of time?
- Worth the amount of time spent?
- Well connected to the other content?
- Generative of life-growth or spiritual dialogue? (Beliefs and Values)
Finally, overall we sought data that would indicate ongoing academic progress (despite an atypical design setting). At the end of the year, all participants took the California Achievement Test as a norm-referenced standardized measure of academic ability. We could compare the test scores only based on parent perception of growth due to varied tests used or non-existent previous year test scores. Where prior testing would be valuable, it was not consistently available. In future research, we seek to compare these scores in longitudinal study.
Findings
Use of Instructional Guides
At the beginning of the year, there were some comments from site instructors that described feeling overwhelmed or as one parent put it “analysis paralysis.” With one lesson and one project with each subject area, our initial provision amounted to more than MKO’s found sufficient. Lesson plans were then edited after week three to have less content and less projects to choose from. No further comments were made after reducing to one reading and one project per course. Length of reading varied by reading level.
After three weeks of the pilot, sites were given the option of daily or weekly lesson plans. 11 out of the 13 sites choose weekly. Many of them explained they found weekly guides easier to adjust to their own unique schedules. The two sites that chose daily had older students who were working independently a great deal of the time and for them daily guides were less overwhelming and more manageable as they were handed off to the learner.
In general, participants had little difficulty managing the digital lesson plans. One parent wrote, “My kids know exactly what to do and get right to it. The links to the extra web information are really good and I love that we’re actually using it because we don’t have to find the information.” Another parent agreed, “They know where to go for assignments and easily navigate on their own when I am helping another child.”
When it came to navigating from the digital lesson plans to the physical instructional guides, almost every site instructor cited some initial struggle. Some participants continued to report frustration throughout the year. One parent declared that they would “like to never have to go to the… binder again.” As we continued through the year, we minimized the printed guides and included as much as possible in the digital lessons themselves.
Five sites reported that a grandparent or another adult had jumped in for a week or more to help teach. Substituting a primary MKO to another was reported without noted issues or difficulties. One site regularly rotated teaching between mother, father, and grandparent without a loss of structured experience consistency.
Use of On-Call Support
MKO’s preferred email as the main mode of communication with 355 recorded exchanges and 68 hours logged. Video was the second most utilized with 137 individual video conferences and approximately 123 hours spent. In-person contact predictably occurred 14 times and was third in hours spent at 43 hours logged.
As expected, the first few weeks of September saw the most communications (96) and the longest exchanges (40 hours) as participants sought support with finding and understanding instructional materials. As the year progressed, on-call support decreased and stabilized. March saw the fewest communications (34) and the quickest communications (13 hours) as by then educators and students had routine practice.
Preparation Time
After the first month of transition, site MKO’s were asked weekly how much time they had had to spend in preparation for that week’s learning. On average they spent just under an hour preparing for the week (24 classes + Fridays), or 2-3 minutes per course. Most preparation involved bookmarking pages and tabs and assembling student binders. MKO’s reported that this preparation usually took half an hour or less to do. Bigger projects that involved ordering and assembling materials naturally took longer to prepare. The majority of projects completed by sites did not require a noticeable amount of additional preparation.
Overall, this is less than a typical traditional teacher (50min/course) ‘prep hour’ in that the provided resources did much of the prep work already. As the year went on more and more MKO’s reported that they were spending less and less time in preparation. The provided resources allowed non-trained MKO’s to use less time than state certified teachers.
Structured Experience via Literature
Survey results provided data on the perceived value of reading materials and literature provided for the learning process. The emphasis on reading was different from traditional models and students read on average 25-30 books over the course of a year. Feedback had MKO’s assessing age appropriateness, amount, value, connection, and if it was generative:
Figure 1
MKO Evaluation of Reading Content of Portals
Interview feedback was generally positive (8-11) and critiques were in relation to student preference and MKO’s switching to other readers – which supports successful flexibility built into the model. Most respondents “Always” or “Mostly” deemed the readings appropriate, timely, connected, and generative. Comments often noted time adjustments even if not rated in the survey as such, “Books for the most part were spot on for everything as far as age & time appropriate… if it wasn’t on audio, timewise with our schedule we could run out sometimes for that specific week!”
Again, showing MKO agency and willingness to adjust as needed.
Structured Experience via Project-Based Learning
Projects were provided with each lesson, or six per day, with additional ideas for Fridays. We did not anticipate parents doing all projects but selecting from options. This is reflected in the survey results.
Figure 2
MKO Evaluation of Project Content of Portals
MKO’s reported positive age level on projects, moderately on ease of preparation, varied on worth of time, positively on connection to content and moderately on generativity. These results inform future project design and capture a mixed reaction from parents. Some valued the project as a key element of learning, some didn’t see the point or skipped them due to time. For example, in the interviews they explained, “We didn’t always do the activities that were suggested. Sometimes it was just a matter of not being able to get the supplies” and “I didn’t do a whole lot of the Dos, but the ones we did – they went well with the content. And it allowed for more discussion.”
Others embraced the projects and noted, “It is great to have activities to Do! Another way to learn and have a lesson sink in” and “The kids had fun getting their hands onto materials.” “I appreciated that the ‘do’ was connected to the lesson taught that day so the kids could really get a feel for the era or get a better feel for the context of the lesson.” For those that made time for projects, the responses were positive, for those that didn’t they were mixed.
Structured Experience via Digital Literacy
Provision of digital resources to supplement the reading and project were nearly universally used and perceived as valuable. MKO’s gave “always” perceived value for age, access, time, connections, and generative of dialogue in most cases.
Figure 3
MKO Evaluation of Digital Media and Resources
Interview comments concurred with the survey data. Interestingly, these were perceived as additions to “School”, “[I] Loved having these to add to the lessons to give the kids the electronics balance with school work.” Additional materials were welcomed as a more engaging modality, “My children are so visual, and the videos were great if I needed a day for Dad to teach,”; and some saw the digital media as a way to work with multiple students or ages. “It helps at times when one or all of the kids want to work more independently.” MKO’s perceived these as uniquely valuable for learning overall.
Academic/Testing Indicators
With decreased preparation time, increased project time, decreased days per week, and increased conversational time around beliefs and values, we expected decreased standardized test scores as an acceptable trade for other design goals. We sent out a letter explaining the shift in learning model, time allocation, and use of digital media as not accounted for in the standardized tests and to expect scores to remain static or even decrease in regards to a test designed to measure a traditional model. Regardless, we wanted to see them and have MKO’s provide feedback on scores compared to the previous school year. 30 out of 34 of the students in the pilot took the California Achievement Test (CAT). 4 Kindergarteners did not take the test. 16 boys were tested and 14 girls.
Figure 4
Portals Student Stanine Scores on CAT Testing from 1st to 7th Grade
27 of the 30 students met or exceeded grade level expectations. 18 of 30 students were in the 75 percentile and above when compared with their peers.
Regarding the three that did not meet grade level expectation, one had a learning disability, and the MKO reported more than a one year gain from the previous year; another didn’t push for completion; and the last one reported “not taking the test seriously and rushing through it.” Including these outliers, all test scores were deemed as expected or above expectation by the MKO’s. The two that did not try on the test are removed from the data below. The student with a learning disability is #26 on Figure 4.
Table 1
Portals Pilot Study CAT Testing Results and Average Difference from Age-Based Placement
On average pilot students scored +2.2 grade levels above their age-based grade expectations with several students scoring multiple grades higher. On average pilot students scored highest in Language Usage and Reading Comprehension. Math and Spelling scores remained consistent with age expectations. The one student with learning disabilities performed just above grade level equivalent in Language Structure and below grade level equivalent in vocabulary. These were reported as large gains in one year from previous testing. Language Mechanics grew to four grade levels above age equivalent.
Indicators of Belief and Values Development
All our sites that responded to the end of year survey stated that their family had grown together in beliefs and values. When asked specifically if the designed resources were contributors, they perceived that Portals was a key contributor to this process. They all agreed that the books provided were morally sound, generative of conversation, and they had time to have longer conversations around the provided content. Two sites noted the provided questions led to deep conversations. Bible projects were frequently cited as the favorite projects each week on survey feedback, noting that they provided longer time to engage with, challenge, and have discourse around beliefs.
Parents reported an overall trust in the resources when it came to alignment with beliefs. When this wasn’t the case, parents used the call center to express concerns with content, “I can recall one situation where we clicked a video that was linked as a resource and the lyrics on the song that was playing were not parent-honoring or God-honoring. I did chat with Mandy about this and we agreed that it should be removed.” This open on-call channel allowed for addressing curriculum concerns immediately and at a particular level of a song, video, or text, versus distrust of the entire program or the scope and sequence of the curriculum as a whole. For any scalable system, this immediate channel for concerns allows both trust and agency on behalf of the MKO.
When interviewed, some of the students at one site stated that they thought “more students should come to their homeschool co-op,” because they felt, “God’s presence there in their relationships and opportunities to pray together.” Among the pilot sites, other examples include discussions around: ministry, service, meditation, prayer, perseverance, charity, memorization, study, fellowship, and discipleship.
Returning users with no incentives indicates perceived value and success of Portals over the year. The initial pilot study population 34 with incentives has grown to 225 signed up for the following year without incentives.
Limitations
Any designed study in education entails a limited set of findings. Changing the context of a child’s learning employs so many changes that no one can be pointed to for the cause of the change in outcomes, also child development occurs overtime, so prior education and future education all play into the outcomes. Moreover, any educational research is essentially a kind of self-reflection once the researcher enters the learning space. The writers of this research are likewise also the designers of the intervention, and the developers. The authors have attempted to report without bias, but we acknowledge that it is impossible to be completely impartial. The danger of any designed based research is an overstatement of ‘finding,’ or an underappreciated number of variables at play that create a misinformed loop of interventions without direction (Cuban, 1990).
Further, this study is only one year. The pilot program is limited by a single set of CAT scores, and needs to be documented over time. A proper longitudinal study is intended by the authors, but first needed the design-based approach to develop a system that is viable and can scale, the specific target of this study, prior to any promise around student performance.
Study participant selection was voluntary and likely includes highly motivated parents looking for strong educational options. This emphasis on 7-year-olds helps to offset the preconditional learning that had already happened in other settings, yet retain a shift in model. High performance is likely influenced by attention to the new model itself. Results over time in a longitudinal study would increase significance of results. In addition, the study was done within a faith community, and could be tested with a more diverse and/or secular context and population.
The study is a sweeping design-based model testing a new way to support and provide structured experiences for learning. Even among participants it was difficult to understand this was not simply “school at home” or “homeschooling with support.” The integration of pedagogical approach was built into the lessons themselves and provided a carrot for activity, but not instruction or demand for it. Any study that allows for such flexibility becomes challenging to document and share. The flexibility itself is a result, but only is compelling to readers that also perceive that as an inherent value versus a liability in research.
The study is unique in that it is not easily replicated. Large investment in lesson design, project ideas, and links to digital media required copious planning time and cannot be easily shared in a research paper. The design itself would have an impact on learning and perceived learning. Replication would need to be centered on equipping the MKO, and not the student themself. Overall, no design-based research is conclusive, but meant to be a contributor to a larger discussion of theory and practice. The study is limited to positive “proof of concept,” but not within another context, population, or designed elements used.
Discussion
The Portals pilot study is a support model for MKO’s to deliver structured experiences via literature selection, projects, and digital media that support learning in a way that aligns with the beliefs and values of the MKO. In addition, test scores support that comparison to age-level expectations of learning was not only not hurt by a shift in time and activity emphasis, but increased. The change to a new format, context, MKO’s support did not have observable adverse effects on standardized measures and clear positive effects according to interviews, surveys, and observations throughout the year. The question remains unexplored if the support actually has a measurable positive effect over time. As a one-year study, the findings are notable, but not as convincing as longitudinal impact. It would be interesting as a recommendation for future research to conduct a correlational study to determine whether there is a positive correlation between implementing the Portals program and above-average CAT test score over time.
The literature itself did suffice as a structure for the core courses and thematic design. Parents did report an agreement that the readings were valuable when they encouraged emotion and engagement. The use of narrative and storytelling was generative of activity and interest. For some students this meant ending the year far beyond their age-based equivalent expectations. Stories are arguably more engaging in that they are in their own context. Reading within a narrative is more compelling than disassociated ‘textbooks’ because of character, drama, and discussion between the MKO and learner. Impact on language and reading scores were clear – even among families that ‘skipped’ a book, the overall reading volume was still larger than comparative traditional settings. For instance, “only” reading 20 instead of 30 books might mean the MKO perceives the year as less than others, but is still sufficient to pull reading levels well beyond CAT average grade level performances – where typical students may only be reading 2-5 books per year. Simply put, if you want students to increase reading performance, have them read. If you want better language usage, vocabulary, comprehension, and mechanics, don’t study those things isolated from a story, instead read more stories and the “language arts” will follow.
Decrease in on-call support indicates that the support structure successfully helped new educators gain efficacy in practice. Ongoing, even if less than the onset of the year, shows ongoing professional development and trust in the Portals program. As for viability as a support structure these findings indicate a scalable model that was both equipping and creating ongoing design for learning for the adults to enhance practice over time without a compensation model of any sort. Some new educators, at least, seek support and growth for the sake of improving practice.
Projects had the anticipated result also. The variety of activity, context of it, and the situated learning within the structure and challenge of the project resulted in engagement when families had the time to engage in the project. Families did engage when interested, and skipped projects that were not interesting. The one-size-fits-many model simply did not happen when MKO’s were given freedom to select as per interest. When projects were taken on, reports were glowing around them. They too saw the contextualized activity as relevant and worthy of their time – or as Brown (2009) says, “Fertilizer” for learning. For those that did projects, they perceived it as a key for learning to be retained. Perhaps this is one of the significant advantages of homeschooling over traditional models, the time to spend on a project or endeavor. MKO’s can stop all other obligations and allow time for inquiry in a way that is impossible in a standardized model. More dedicated and isolated study of the value of projects is in the body of research, but more could be done in combination with a flexible time allowance for project pursuits.
Use of digital media also became both referential and supplementary. Families reference the media when they need more help, a different approach, or just to fill time. Use of media to support multi-age contexts was clearly happening – indicating digital media as useful for automation or learning without a human MKO. Where some argue digital media is more effective with guidance, we did not see that in actual use outside of pointing the learner at a resource. However digital media was clearly and resoundingly welcomed in the overall learning model and had the most positive perceived value from MKO’s in this study.
Finally, we did see our MKO’s report positive synergy between content and advancement of their own beliefs and values. Where this is impossible to totally separate, it is an area in need of much more research and inquiry as it is associated with overall academic success, parental confidence, and perceived worth of educational support structures from texts to school boards. Portals parents were confident that we aligned with their values. Even when readings conflicted, the partnership was sustained with easy call-in mechanisms. We wonder if trust can be maintained without clear and open lines of communication. Nationally, we see a trend of school boards and teacher unions limiting parental voice, when they should likely be learning from it. This may actually be a central cause in the rise of homeschooling also. Where distrust grows in terms of family beliefs and values, the perceived worth of the structured experience shrinks. At the least this would be more of a sociological study with seeds of interest found in our results.
Overall, the Portals pilot shows a proof of concept that structured experiences can be provided effectively to MKO’s that have no other training to use homeschool and academic approaches to teaching and learning. Namely the use of literature, projects, and digital media in such a way that allows the MKO an opportunity to control the purveyance of family beliefs and values. The resources provided a set of tools with embodied cognition to support the teacher effectively. The Portals concept is both viable and inherently scalable in design towards equipping non-trained educators to successfully complete a year of education in a variety of settings.
References
Alexander, T. (1919). The Prussian elementary schools. Macmillan.
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14. http://website.education.wisc.edu/kdsquire/manuscripts/jls-barab-squire-design.pdf
Barab, S., Zuiker, S., Warren, S., Hickey, D., Ingram-Goble, A., Kwon, E.-J., Kouper, I., Herring, S. C. (2007). Situationally embodied curriculum: Relating formalisms and contexts. Science Education, 91(5), 750-782.
Bawden, D. (2008). Origins and concepts of digital literacy. In C. Lankshear & M. Knobel (Eds.), Digital literacies: Concepts, policies and practices. essay, Peter Lang Publishing.
Bredeson, P. V. (2013). Distributed instructional leadership in urban high schools: Transforming the work of principals and department chairs through professional development. Journal of School Leadership, 23(2), 362-388.
Bruner, J. S. (1961). The act of discovery. Harvard Educational Review.
Cuban, L. (1990). Reforming again, again, and again. Educational researcher, 19(1), 3-13.
Danielson, C., & McGreal, T. L. (2000). Teacher evaluation to enhance professional practice. ASCD and ETS.
Devitt, R. (2023). 50 Top-Rated Popular Homeschool Curriculum Programs in 2023. How Do I Homeschool. Retrieved from https://howdoihomeschool.com/most-popular-homeschool-curriculum/
Dikkers, S. (2015). TeacherCraft: How teachers learn to use Minecraft in their classrooms. ETC Press.
Dikkers, S. M. (2014). The future of mobile media for learning. The New Landscape of Mobile Learning, eds. C. Miller and A. Doering. Routledge. 104-119
Dikkers, S. M. (2016). Questing as learning: Iterative course design using game inspired elements. On the Horizon 24 (1), 55-70.
Engler, C. M. (2004). The ISLLC Standards in Action: A Principal’s Handbook. Eye on Education, Inc.
Gardner, H. (1993). Multiple intelligences: The theory in practice. Basic books.
Gee, J. P. (2012). Situated language and learning: A critique of traditional schooling. Routledge.
Gee, J. P. (2007). Good video games+ good learning: Collected essays on video games, learning, and literacy. Peter Lang.
Hargraves, V. (2021). Dewey’s educational philosophy. The Education Hub. https://theeducationhub.org.nz/deweys-educational-philosophy/
Hirsch, E.D. (1987). Cultural literacy: What every American needs to know. Boston MA: Houghton Mifflin.
Hirsh-Pasek, K., & Michnick Golinkoff, R. (2008, October). Why Play = Learning. CEECD / SKC-ECD. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=0fd5f1475d49c2d383afc05c48082f8799a69d65.
Hutchins, E. (1995a). Cognition in the wild. Cambridge , MA : MIT Press.
Hutchins, E. (1995b). How a cockpit remembers its speeds. Cognitive Science, 19(3), 265-288.
Kelley, C. J. & Shaw, J. J. (2009). Learning First! A School Leader’s Guide to Closing Achievement Gaps. Corwin Press. Thousand Oaks, CA.
Kennedy, T.J., & Sundberg, C.W. (2020). 21st Century Skills. In: B. Akpan & T. J.Kennedy (Eds) Science Education in Theory and Practice. Springer Texts in Education. Springer, Cham. https://doi.org/10.1007/978-3-030-43620-9_32
Klymkowsky, M. W. (2007). Teaching without a textbook: strategies to focus learning on fundamental concepts and scientific process. CBE Life Sciences Education, 6(3), 190–193. https://doi.org/10.1187/cbe.07-06-0038
Lankshear, C., & Knobel, M. (Eds.). (2008). Digital literacies: Concepts, policies and practices. Peter Lang Publishing.
Lave, J. (1988). Cognition in practice: Mind, mathematics and culture in everyday life. Cambridge University Press.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge University Press. https://doi.org/10.1017/CBO9780511815355
Levy, P. (2015). Collective Intelligence for Educators, Educational Philosophy and Theory, 47:8, 749-754, DOI: 10.1080/00131857.2015.1053734
McLeod, S. (2023). Vygotsky’s sociocultural theory of cognitive development. Simply Psychology. https://www.simplypsychology.org/vygotsky.html
Mitchell, T. & O’Brien, J. (2020). Diversity, equity, and inclusion (DEI) matters. EDUCAUSE Review 55(2).
National Center for Montessori in the Public Sector. (2023). Montessori helps children reach their full potential in schools all around the world. https://www.public-montessori.org/montessori/
Odden, A. & Kelley, C. (1997). Paying Teachers for What They Know and Do: New and Smarter Compensation Strategies to Improve Schools. Corwin Press. Thousand Oaks, CA.
Papert S. & Harel, I. (1991). Constructionism. Ablex Publishing Corporation. https://pirun.ku.ac.th/~btun/papert/sitcons.pdf
Partnership for 21st Century Skills. (2008). 21st Century Skills, Education & Competitiveness: A Resource and Policy Guide. https://files.eric.ed.gov/fulltext/ED519337.pdf
Prensky, M. (2001). Digital natives, digital immigrants part 2: Do they really think differently?. On the horizon, 9(6), 1-6.
Ray, B. (2017). A Review of research on Homeschooling and what might educators learn? Pro-Posições. 28. 85-103. 10.1590/1980-6248-2016-0009.
Ray, B. (2021). All Education is Spiritual and Ergo Homeschooling is Resurging. Wydawnictwo Uniwersytetu Łódzkiego. https://www.ceeol.com/search/article-detail?id=970565
Resnick, L. B. (1991). Shared cognition: Thinking as social practice.
Resnick M., & Robinson K. (2017). Lifelong kindergarten: Cultivating creativity through projects, passion, peers and play. MIT Press.
Schrier, K. (2016). Learning, education and games (Vol. 2). ETC Press.
Squire, K. (2021, February 17). Situated learning theory. Educational Technology. https://educationaltechnology.net/situated-learning-theory/
Squire, K., & Dikkers, S. (2012). Amplifications of learning: Use of mobile media devices among youth. Convergence, 18(4), 445-464.
Squire, K. D. (2008). Video games and education: Designing learning systems for an interactive age. Educational technology, 17-26.
Steinkuehler, C., Compton-Lilly, C., & King, E. (2010). Reading in the context of online games.
Steinkuehler, C. A., & Williams, D. (2006). Where everybody knows your (screen) name: Online games as “third places”. Journal of Computer-Mediated Communication, 11(4), 885-909.
Tekinbas, K. S., Torres, R., Wolozin, L., Rufo-Tepper, R., & Shapiro, A. (2010). Quest to learn: Developing the school for digital kids.
The National Center for Education Statistics. (2020). Appendix tables for 2020 LongTerm Trend Reading and Mathematics Assessments at Ages 9 and 13. Retrieved online: https://www.nationsreportcard.gov/ltt/supporting_files/2020_appendix_ltt.pdf
Tomlinson, C. A., & McTighe, J. (2006). Integrating differentiated instruction & understanding by design: Connecting content and kids.
Twenge, J. M., Martin, G. N., & Spitzberg, B. H. (2019). Trends in U.S. Adolescents’ Media Use, 1976 –2016: The Rise of Digital Media, the Decline of TV, and the (Near) Demise of Print. Psychology of Popular Media Culture, 8(4), 329-345.
Vygotsky, L. S. (1987). The collected works of LS Vygotsky: Problems of the theory and history of psychology (Vol. 3). Springer Science & Business Media.
Wertsch, J. V. (1998). Mind as action. Oxford university press.
Wigfall, C. (2022). Minnesota 4th grade reading & 8th grade math scores on national tests lowest in 30 years. American Experiment. https://www.americanexperiment.org/minnesota-4th-grade-reading-8th-grade-math-scores-on-national-tests-lowest-in-30-years/
Wigfall, C. (2023). More test results show math and reading scores the lowest in decades. American Experiment. https://www.americanexperiment.org/more-test-results-show-math-and-reading-scores-the-lowest-in-decades/
Wood, D. J., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychiatry and Psychology, 17(2), 89-100. ¯