This article describes the “dreamscape” as EdVisions Schools moves forward as an education development organization. “Dreamscape” refers to the goals we have developed for the network of more than 40 Edvisions schools nationwide. Here, we describe the evolution of objectives for EdVisions school sites, the development of assessment tools to measure schools’ status and progress, and
EdVisions Schools is an education development organization and CES affiliate center that provides a program model and staff development for charter school creators. Based upon its flagship school, the Minnesota New Country School in Henderson, Minnesota, EdVisions helps create small learning environments that meet adolescent developmental needs, engage previously disengaged students, and provide an experience that designed to help develop productive citizens. Our organization has two divisions: the EdVisions Cooperative, which focuses on school development, including ongoing school coaching, and the EdVisions leader center, which supports school development through best practice dissemination, institutes, research, and a variety of technical assessment tools.
EdVisions’ approach to teaching and learning focuses on highly personalized learning in full-time advisories. Students have the opportunity to learn in different ways to achieve curriculum standards and earn graduation credits though rigorous, engaging projects that are driven by student interest and connected to the real world. Educators experience a new level of professionalism, putting them on par with other vocations. Results for schools in operation for three years or more show conclusively that the EdVisions model can have a powerful impact on students’ long-term success.
The intention of an EdVisions School is to create a personalized culture with strong relationships to support adolescent developmental growth. This is done through a multi-age advisory system whereby each student is well known personally and academically. Each student will feel a sense of belongingness. As well, EdVisions schools support adolescent needs by creating a self-directed, project-based learning program that allows autonomy, thereby creating relevance. Academic needs are supported by creating learning environments that give the students what they need when they need it. Each student is treated as an individual and has a personal learning plan, creating an atmosphere of mastery goal orientation rather than of performance goal orientation. Our prevailing philosophy is that the development of hope is the end result of our work with adolescents.
The Hope Study
In 2003, Dr. Mark Van Ryzin, a researcher from the University of Minnesota, came to EdVisions with a plan to measure relationships and relevance via a set of student self-perception surveys called the Hope Study. The Hope Study is made up of a series of surveys that ask students for their perceptions of the learning environment in regard to autonomy (more choices, voices are heard, opinions are valued), belongingness (personal and academic support from both adults and students), mastery goal orientation (intrinsically motivated, not learning for external rewards), academic press (high expectations, reach for understanding), engagement, and hope. EdVisions Schools utilize the surveys to measure the ability of a newly created or enhanced environment to provide an engaging atmosphere that supports adolescent needs in the above mentioned learning environment variables. As autonomy, belongingness, and mastery goal orientation have a strong correlative affects on engagement, the analysis of the data from the surveys gives EdVisions coaches and management an early indicator of whether the schools are re-engaging previously disengaged adolescents.
The study thus far has shown that students in EdVisions Schools do perceive higher levels of autonomy, belongingness, mastery goal orientation, and academic press, and therefore higher emotional and behavioral engagement than is found in traditional school settings. When it can be shown that engagement is rising, the Hope Study shows that hope will also rise correlatively. Hope is an indicator of an internal disposition toward resilience, persistence and goal setting in which individuals see pathways to success and feel they can be the agents of their own success. This dispositional growth provides adolescents the internal means to become successful and productive adults.
The EdVisions Design Essentials
The Hope Study will provide EdVisions coaches the ability to provide data and coaching around a set of EdVisions Design Essentials developed to support the relationships, relevance, and rigor necessary for the school to become an exemplary environment for adolescent development. The Design Essentials are incorporated into four major categories:
Academics: A personalized, project-based learning program that values the self-directed student
Learning Community: A democratic culture that values student voice and supports them with full-time, multi-age advisories and other structures
Evidence of Learning: An assessment system that values performance, application, and “using one’s mind well”
Teacher Professional Practice: A democratic system that empowers teachers to make most of the decisions in managing the school
In the first five years, by design, EdVisions allowed a large degree of autonomy for founders to create the kind of school they believed would engage adolescents and to help them acquire skills necessary for the future. EdVisions school founders had to use a project process to meet state course requirements, use authentic assessments, organize students and teachers into advisories, and manage their own schools, but we rarely imposed upon them exactly how to do these things. Consequently, some of the schools we helped create developed what we would call “hybrids,” incorporating elements of the EdVisions mode with other influences.
Some of these hybrids did not implement some of the key elements that were fundamental to the EdVisions Design Essentials, such as personal work stations and full-time advisories, and were dropped from further grants and from our network of schools. Other schools adapted the model in ways they believed they could reach the student populations that they were serving: students of color, credit deficient urban students, high levels of special education students, etc. EdVisions encouraged some of these adaptations because we believed in autonomy and empowerment of educators. It would appear rather suspicious behavior to attempt self-sufficiency for students and not allow the same for teachers.
We also knew that sites developed a wide variety of ways to assess life skills, and that the life skill that data was collected was utilized primarily as a method of personal assessment; the data rarely went beyond advisor and student. In other words, life skill data was used neither as a school improvement tool nor as a professional development tool. If data cannot be aggregated beyond a student’s personal learning plan, it does not inform the staff about how effectively the school is building those skills. EdVisions knew that we needed to place more emphasis on aggregating that data for schools to become stronger.
Consequently, the Design Essentials Rubric was created to provide coaches and school staff a means by which they can see correlations between the Hope Study variables and the Design Essentials. School improvement plans can then be created from the results of that inquiry. Data on each of the variables provide school staff members and coaches the means by which they can drill down to aspects of their environments and react to those particular relational factors. For example, if a school exhibits a low mastery goal orientation, staff members may reflect on whether or not grades are emphasized over and above understanding. If autonomy is low, they may conceive of ways to allow for more student voice or choice within the framework of the classes, projects, or school culture.
EdVisions Evaluation Plan: Additional Elements
In addition to discovering the ways that the environment affects engagement and hope, the Hope Study data will be longitudinally correlated with retention rates, high school graduation rates, college matriculation, college persistence, and ultimately college graduation rates. EdVisions eventually plans to have in place a College Transition Coordinator who will track students after high school graduation. Engagement and hope scores of individual students while in high school will be utilized in an ongoing study of students as they matriculate in post-secondary institutions. With this data, EdVisions intends to prove that high school environments that provide high levels of relationships and relevance also provide rigor and adequate internal assets for adolescent success in college and work.
The Hope Study data also will be correlated with student achievement data while students are still in the high school setting. Each EdVisions school will keep a database of engagement and hope scores alongside of reading and math scores on standardized tests, preferably the Northwest Evaluation Association’s Measures of Academic Progress (MAP) tests. This will provide EdVisions sites the ability to verify that rising engagement and hope does indeed affect basic skill acquisition. It will also provide EdVisions School staff the ability to personalize and customize each student’s learning and post-secondary plans.
We do not wish to have the school climate build engagement and hope so as to obtain good test scores. Rather engagement and hope are ends in themselves. We know that adherence to the testing culture, which often begets the teacher-led, course-driven, and time-based methodology, does limit development of hope. The correlation to test scores is more a need to provide proper academic support not for test prep, but to provide a means by which personal learning plans lead to dispositional hope.
In addition, the EdVisions Evaluation Plan calls for utilizing the College and Work Readiness Assessment (CWRA) to assess higher order thinking skills (critical thinking, analytical reasoning, problem-solving, and communication skills). Success in the CWRA is highly correlative to success on the ACT and SAT tests. This assessment will be given three times from the freshman year to the senior year to gauge the ability of students to tackle the higher level tasks needed for success in college and work in the 21st century.
In review, the EdVisions Evaluation Plan calls for each EdVisions School to:
Collect core demographic data on each student to be utilized in personal learning plans and post-secondary planning as well as providing data for individual school success rates with particular demographics;
Collect and track standardized test data on reading and math from induction to graduation, correlated with CWRA data and Hope Study data on engagement and hope, so as to provide personal planning and value-added data for determining school success;
Utilize the Hope Study annually to provide feedback on development of the environment to provide engagement and hope, correlated with success in raising reading and math levels and higher order thinking skills;
Utilize the CWRA to provide students and staff the data needed for personal post-secondary planning and value-added data for school success.
Carrying out the Plan
In order to successfully provide the services necessary to carry out this ambitious evaluation plan, EdVisions coaches will be trained in each of the evaluative factors and how to coach staff members at each site on administering the Hope Study, the CWRA, the MAP, and in the School Improvement Plan. A method for collecting data will be created within a project management tool called Project Foundry. Project Foundry was developed to aid advisors and students propose projects, track project processes, allow for reflection and journaling, track assessments of projects and other assessments, and create a student portfolio and transcript. Project Foundry is being upgrade to include in a Personal Learning Plan template that will be updated periodically by advisors in each site. This forms the basis for the Post-Secondary Plan as well. As data is gathered from standardized tests, the Hope Study, and the CWRA, it is embedded in the Project Foundry assessment data tracking mechanism and utilized to continually update the PLP. Data then can also be gathered from each EdVisions site by management for school improvement planning and for EdVisions evaluation of site development toward becoming an exemplary site.
Portions of EdVisions professional development time will be given over to teaching about the evaluative measures, the methodology for instituting them, and their analysis. Each EdVisions coach and staff member at individual sites shall be immersed in the measures, the variables, the analysis, the data, and their uses so that evaluative processes are inculcated at each school. With the use of the online capabilities of Project Foundry, EdVisions will have the ability to gather the data needed for analysis of each site, analysis of coaching, and evaluation our entire program for investors and foundation grantors. Coaches will refer to evaluative data when they coach at sites. A number of days of coaching will allow for evaluation personnel to interact directly with each site.
Where We Are at this Point: Site Reviews
As a result of an ambitious business plan and aggressive fund raising aimed at continuing to create and sustain great small schools, EdVisions Schools identified the need to assess where present sites are in regard to the “dreamscape” describe above. Hence, we are in the midst of a large-scale site review process. The review process is based upon the tools already in place, and takes in to account some of the tools and processes we believe would strengthen already existing sites.
In order to carry out the site reviews, EdVisions staff members are visiting 35 sites that have received sub-grant funds to create an EdVisions School. Site reviewers observe the school in action, interview students and staff, and collect certain data. One of the tools used to collect data is the Design Essentials Rubric, which incorporates the Design Essentials in the four categories. The Design Essentials Rubric is scored on a nine-point Likert scale, with 1-3 considered emerging, 4-6 sustaining, and 7-9 exemplary. We already know from the Hope Study and from site visit observations that the Design Essentials, when fully utilized, lead to greater engagement and achievement from students. When sites stray from the Design Essentials, various problems can emerge and schools take steps backward toward the old model of schooling.
Other data gathered by site reviewers includes how the school site uses data-driven decision-making, and which data are used to create personal learning plans, professional development plans, and school-improvement plans. Site reviewers are looking for schools to utilize not only standardized tests and state AYP criteria, but also data about how students are progressing in life skills and life-long learning skills. Each EdVisions site has been encouraged to create rubrics to score student development in process skills, higher order thinking skills, and life skills.
Additional data that site reviewers are asking for are graduation rates (always a difficult set of data to track in charter and innovative schools that don’t use seat-time), the number of graduates accepted into post-secondary degree programs, ACT and/or SAT composite scores, and student growth in measure of hope as assessed by the Hope Stoy. The first, graduation rates, are collected because funders and politicos want to know. The other categories represent information that EdVisions would like to have sites collect in order to create a future focus for students and for school improvement.
After eight years of school creation and replication, we believe that having strong personal learning plans that incorporate a wide variety of data about students, and incorporating post-secondary planning early in the enrollment process, schools will have higher retention rates and higher graduation rates. By utilizing the personalized, student/advisor co-generative curriculum, and project-based processes embedded in the Design Essentials, schools will develop strong life skills and post-secondary success. We know from the Hope Study that schools that embed the Design Essentials increase engagement, and thus build hope and achievement, both in basic skills and in life skills. The site review process was designed to collect data to prove that hypothesis.
The reason the CWRA was built into our future assessment dreamscape was to utilize a means by which higher order thinking skills could be assessed by other than subjective rubrics; especially given the fact that the rubrics devised by our schools were tremendously diverse, and were used in many different ways. The CWRA is an authentic task-driven assessment of a student’s ability to do problem-solving, use critical thinking, do analytical reasoning, and utilize good writing skills. Although there are other kinds of skills that could be assessed, these kinds of skills will serve young people well in any future endeavor. EdVisions hopes to have the CWRA piloted in the exemplary schools this spring to see if it indeed could be used to replace life skill rubrics.
Although the Hope Study has been in operation for four years, not all EdVisions sites have been using it. The review process allows us to give feedback on whether or not stronger results are coming from sites utilizing the surveys, and how well sites that are using it are using the data for school improvement. We at EdVisions are certain that if schools use the surveys as a school improvement tool, they will develop stronger advisories, stronger academic support systems, and students will have greater success in future endeavors. But schools have to use the data well. This requires not only buying into it, but also utilizing staff development and school improvement time devoted to enhancing the variables that create strong climates for engagement and hope.
Likewise the senior staff at EdVisions Schools knew that many of the early schools had not emphasized the future focus well; most of our schools were quite content to see previously disengaged students graduate from high school. Whether or not they had skills to complete college or were work ready was not a high consideration. Because of greater emphasis by funders, researchers and by the media on future success, we as an organization also need to prepare schools to look beyond high school graduation. The site reviews and assessment of school results are designed to refocus schools on long range student results. We will be asking schools to attempt tracking senior acceptances into degree programs and to undertake alumni tracking processes and surveys to determine alumni success; then for the schools to use that data for school improvement and staff development.
The site review process is in its early stages, and it is too early to discuss results. Suffice to say that we are finding that many sites could develop stronger and more formal personal learning plans, especially in how they incorporate strong post-secondary planning. Also, as we knew going in, the schools could do more to use life skill data for school improvement. We also knew that some sites using the Hope Study were not using the data to the maximum affect. And schools could improve their programs if they would find how well their students are doing after graduation. But we are also finding that the bulk of the schools we helped develop and for which we provided staff development are adhering to the model to a high degree and are doing some very positive things for students via developing strong relationships and providing relevant learning experiences.
The student interviews are the most enlightening. The student population in these schools is overwhelmingly enthusiastic and positive about their experiences. Many students say they previously hated school and ditched whenever possible. Now they are interested in what they are doing and want to talk about their projects. Their attendance is up. Most say they feel cared for and safe. EdVisions students tell us that their schools are successful in transforming them from passive to active learners who have taken responsibility for their own standards of success. As a measure of student perceptions of their environment, the Hope Study demonstrates that students are engaged and building hope, which translates into success by all measures. As we continue with this review process, we at EdVisions believe we will see strong data from strong schools, and that will enable us to help more schools get those positive results in the future.
More on the Hope Study
The elements of that the Hope Study assesses are:
Autonomy: Student choice
• Choose what to study and when
• Personalize goals with flexible timeframes
• Learn what is relevant to them and why
• Use creativity and various learning styles
• Various points of view accepted and encouraged
Belongingness: Perceived support of advisors and peers
• Advisors have ongoing interaction with students
• Advisors show care and concern for each student
• Advisors advocate for students and support their learning
• Peers accept and demonstrate respect for one another
• Peers support one another in their learning
Goal Orientation: Approach to learning
• Intrinsically motivated
• Willing to risk
• Positive attitude
• Choose effective strategies for learning
Engagement: Approach to tasks
• Strong work ethic
• Pay attention and concentrate
• Sustained effort
• Use time wisely
• High level of cognitive engagement in learning tasks
Hope: Level of optimism toward ones’ ability to be successful
• Conceptualize challenging goals and pathways to obtain them
• Develop various strategies to achieve goals
• Initiate and sustain motivation toward goals
• Demonstrate resilience in achieving their goals
EdVisions Design Elements
Academics Design Essentials
Generative self-directed project-based learning: Self-directed project-based learning, driven by constructivist pedagogy, where each student has a personalized workspace with Internet access is the leading and most visible expression of the small school.
In-depth Learning: Significant student-led interdisciplinary projects, senior and/or capstone projects, internships, or field study projects that allow for in- depth, original learning over time. Project reflection is built into design of in-depth learning.
Facility is designed for decentralized learning: Facility design supports the advisory structure, student generated curriculum, and decentralized lab/work areas.
Personal Learning Plan (PLP): PLP emphasizing each student’s aspirations and academic interests, including post-secondary planning beginning in ninth grade. PLP is reviewed each quarter.
Reading: Students and staff engage in quiet reading every day. Reading development is part of each student’s PLP and often a key component of projects. Strong advisory-based community of readers. Strong literacy ethic permeates pedagogy.
Learning Community Design Essentials
Small school with multi-age advisories in a personalized climate: Small school constructed from autonomous, full-time, multi-age advisories. School is less than 160 students; if larger than 160 students then organized by autonomous “houses” of no more than 80 students. Personalized climate; personal responsibility, belongingness, task completion, and peer support are evident in highly functioning advisories.
Democratic student learning and leadership: Student voice/consultation is vital. The balance of individual and community is informed by a substantive climate of student voice, consultation, and decision-making and leadership development.
Community connections with experts/elders: Mentoring available to all students from a wide range of community experts and elders. The school is known as a place where learning is enhanced by these community connections.
Extended Day, Year and Variable Scheduling design: Innovation around time and scheduling is often a design feature of the school.
Evidence of Learning Design Essentials
Project proposals and assessments: Project proposals articulate state, school, and self-developed standards. Multiple adults assess projects.
21st Century Skills are priority outcomes: Students are assessed for 21st century skills in interdisciplinary problem solving, lifelong learning styles, communication, collaboration, and information systems.
Electronic Project/Portfolio Management: Electronic system (Project Foundry) for projects and student portfolios.
Growth model of assessment: Hope Study: Academic test results shape PLP. Hope Study results inform school improvement plans.
Teacher Professional Practice Design Essentials
Autonomy: Autonomous school management with control over budget and staffing; individual responsibility and accountability for school finance and educational success.
Teachers and staff full partners: Teachers and staff members are full partners in the school vision and implementation and meet on a frequent basis.
Staff Evaluation and Professional Development: Teacher evaluations by peers, students, and parents; coaching/mentoring plan aligned with each teacher’s PLP.
For more on . . .
• the Northwest Evaluation Association’s Measures of Academic Progress (MAP) tests, visit www.nwea.org/system.asp.
• the College and Work Readiness Assessment (CWRA), developed by the Council for Aid to Education, visit www.cae.org/content/pro_collegework.htm.
• the online project based learning management system Project Foundry, visit www.projectfoundry.org.
Dr. Ron Newell is the Evaluation Director of EdVisions Schools. He was one of the founders of the Minnesota New Country School and has worked for nine years with EdVision Schools to replicate the model school nationwide. He is author or co-author of four books on the reform efforts of EdVisions: Passion for Learning; Democratic Learning and Leading; The Coolest School in America; and recently, Assessing What Really Matters in Schools, all from Rowman & Littlefield Education.