PD 360 Impact Assessment: 422 Title 1 Schools

School Improvement Network’s PD 360 is positioned to deliver embedded professional learning to the academic community that will improve teacher preparedness, skills and effectiveness, resulting in significant increases in student achievement in classrooms with participating educators.  Title 1 students are traditionally known to represent among the greatest challenges for showing dramatic improvement, and often struggle to close performance gaps with their non-Title 1 peers, if that gap can ever be closed at all.                

Data from this study verified that Title 1 students whose teachers participate in PD 360 experience significantly greater gains in standardized test scores than their peers whose teachers do not.  Additionally, the data show that Title 1 students outperformed and outgrew their respective districts when their teachers participated in PD 360, a remarkable finding with great implications for educational success.

Product Overview

PD 360 represents the largest on-demand professional development video library available within the academic community. The application is delivered through an Internet-based platform that is built around over 1,500 core professional training segments focused on today’s most critical topics in education.  PD 360’s indexed and searchable content library contains video learning segments that reflect research-based methods and strategies that are proven to improve teacher effectiveness and enhance student achievement.  All professional development is delivered by noted national and international experts, supported by real classroom examples, practitioner interviews and graphics, all of which are designed to engage teacher-learners, deliver crucial content and maximize instructional skills. Each content area is available in both elementary and secondary versions so that educators can focus on professional development that is specific to their student populations and their unique instructional needs. 

A license to PD 360 includes the following:

• Over 1,500 video learning segments, averaging 10 minutes in length, covering over 95 academic topics
• Each segment is delivered by the top subject matter experts in each content area
• Every video segment has multiple real best practice classroom examples
• Content is mapped to national teacher and leader competencies with topics including, but not limited to: Differentiated Instruction, Instructional Strategies, Closing the Achievement Gap, Working with Students of Special Needs, English Language Learners, Leadership and Classroom Management
• Integrated follow-up tools, reflection activities, tracking and collaboration, community discussion forums and file sharing

Study Design

The School Improvement Network commissioned the PD-360 Impact for Title 1 Schools study to demonstrate the effectiveness of the professional learning solution in 422 Title I schools that utilize PD 360 throughout the United States.  The School Improvement Network engaged the services of a third-party evaluator, Dr. Steven H. Shaha to conduct a comprehensive outcome study with the aim to provide quantitative, rigorous evidence of PD 360’s positive impact on student achievement when teachers are actively engaged in utilizing the tool.  All data reflect scores achieved in standardized tests for reading and mathematics for the academic years 2008-2009 and 2009-2010 leveraging the standardized tests for their respective states. 

Performance Results in Student Achievement

The PD 360 Impact Assessment revealed statistically meaningful advantages favoring schools with PD 360 versus schools in their respective districts that did not use the learning tool. Schools whose teachers participated in PD 360 (n=422) significantly outperformed their respective districts in student performance in both reading and mathematics. Moreover, performance data implies that the gains made by participating schools actually boosted the performance data for the entire districts in which they were nested by “pulling up the district averages.” 

 

Reading: PD 360 schools demonstrated statistically significant increases in reading proficiency over the previous performance period. Gains were measured by combining the categories “Proficient” and “Advanced” on standardized test results. While minimal gains of 0.1% were experienced across participating districts, the PD 360 schools gained an overall 4.8% for the same school years, a performance gap advantage of 4.6%. (p<.001). And that performance advantage is all the more dramatic when considering that the PD 360 schools began from a point equal to or slightly lower than their respective districts as a whole, meaning that PD 360 use resulted in a gap where none existed prior to its use.

 

 

 

 

A multiple regression analysis revealed a profile for describing predictors for PD 360 schools that experienced the greatest performance advantages versus their respective districts. Schools with the highest gains demonstrated increased levels in the percentage of users that viewed the content of PD 360 – the more teachers viewing PD 360 content, the greater the student gains for those schools.   Accordingly, it was concluded that participating in PD 360 has a significantly beneficial impact on student reading achievement levels and that teachers and schools that participate the most actively will experience the greatest advantages for their students. Furthermore, since the PD 360 schools were included in the cumulative district results, there is a statistical likelihood that the improvements experienced by the PD 360 schools positively affected the cumulative improvements for their district, as well. Thus, PD 360 schools contributed substantively to the gains experienced for their district as a whole.

 

Mathematics: As with reading, PD 360 schools consistently made statistically significant gains in mathematic proficiency rates above those experienced by their districts collectively (p<.001). In fact, the results show that the districts cumulatively actually saw a decline of -5.9% in the percent of students scoring at a Proficient and Advanced levels in mathematics while the participating PD 360 schools within those same districts experienced a gain of 7.3% for the same time period – a gap creating growth advantage of 13.2%. (p<.001). That performance advantage was further magnified when considering that the PD 360 schools began from a point equal to or slightly lower than their respective districts as a whole, thus creating a PD 360 gap where none existed prior to its use.

 

The comparative review of combined percent improvement reveal that the participating schools utilizing PD 360 not only closed the achievement gap and made meaningful gains in student academic achievement, but they were able to make performance improvements when their districts were unable to experience gains in cumulative proficiency rates in mathematics (p<.001).

 

 

 

 

The multiple regression completed for improvements in mathematics performance levels revealed a profile for describing predictors for PD 360 schools that experienced the greatest performance advantages versus their districts. As was the case with reading, schools with the highest gains demonstrated increased levels in the percentage of users that viewed the content of PD 360 – the more teachers viewing PD 360 content, the greater the student gains for those schools.   Similarly, it is important to remember that, since the PD 360 schools were included in the cumulative district results, there is a statistical likelihood that had the gains experienced by the participating PD 360 schools not been included to raise the cumulative rate, the losses experienced by the districts would have been even greater.

 

Conclusions

The Study of the Impact of Participation in PD 360 for Title 1 Schools clearly demonstrates that participating PD 360 schools experienced significant improvements in student achievement in reading and mathematics when compared to the cumulative results of their respective districts. 

Title 1 students traditionally have been among the most challenging to help see improvements in standardized test scores.  Yet in this study, Schools utilizing PD 360 experienced greater success in Title 1 student achievement in both reading and mathematics versus their non-PD 360 peers when their districts showed decline.  The result is that there are significant advantages for students who find themselves in classrooms with teachers who utilize PD 360. 

---

Steven H. Shaha, PhD, DBA, is a seasoned nationally and internationally recognized outcomes researcher with over 300 publications and presentations to his credit and over 35 educational program evaluations.  His doctorate at UCLA included substantive work at the Center for the Study of Evaluation in the Graduate School of Education.  He has affiliations with the University of Utah (Center for Public Policy and Administration) and Harvard University, as well as eight other universities nationally and internationally, and has contributed to over 20 educational grants.