Measures of Effective Teaching Project FAQ


This project will help determine ways in which effective teaching can be measured fairly and consistently.

FAQ Categories

Project Overview and Design
Data Collection
School and Teacher Eligibility and Selection
Data Security
Time Commitment
Union Support
Student Engagement

Project Overview and Design

What is the goal of the Measures of Effective Teaching project?

Great teaching is multi-dimensional. Therefore, it should be viewed through multiple measures, none of which are sufficient to stand alone. Teachers should know what expectations are for good teaching and what they can do to improve their practice. The project will study multiple measures of teacher effectiveness, including videotaped classroom observations, student surveys and other test and non-test based data and compare them to nationally recognized teaching standards. This data will help determine ways in which effective teaching can be measured fairly and consistently.

How is this work being funded?

The Bill & Melinda Gates Foundation will fund the cost of the project. There will be no additional costs to cities or school districts. The foundation will also provide funding for district-based project support staff who will support schools and teachers participating in the project and coordinate all project activities.

How long will this project last?

The research will occur during the 2009-2010 and 2010-2011 school years.

Return to top

Data Collection

What kind of data will be collected through the project?

The project will collect a variety of data:

Please tell me more about the following data:

Videotaped Classroom Observations: (Independent Researchers: Teachscape) Each year independent researchers will videotape four classroom lessons in each subject/grade combination in which teachers are participating. The researchers will use several different standards of teaching to evaluate the video: Danielson's Frameworks for Teaching, the CLASS measure from the University of Virginia, the Quality of Mathematical Instruction rubric from the University of Michigan/Harvard, the PLATO rubric from Stanford University, and a new rubric for assessing science instruction.

Click here for a short video on videotaping in the classroom.

Teacher reflections on their videotaped lessons: Teachers will provide written commentary and any relevant supporting materials to provide context about the videotaped lessons, and to share their self-reflections.

Teacher surveys of the school environment: Teachers will complete a survey about the working conditions in their schools and the instructional support they receive.

Supplemental student assessments: Many state tests assess a wide range of content but dedicate relatively few items to each individual content area. Project researchers will administer an additional test that focuses on the most important content areas for a given course of study. Use of this additional test will allow researchers to assess whether students who perform well on the broader assessment also can perform well on the conceptually deeper assessment. The test will be given in the last 30 days of each school year and will be no longer than 60 minutes per subject at the elementary/middle school level and no more than 90 minutes per subject at the high school level.

Student Feedback: (Independent Researchers: Ron Ferguson (Harvard) and Cambridge Education) Independent researchers will provide all students in participating teachers' classrooms with a student survey. For this project, researchers will adapt the Tripod Survey, which they have administered in a number of school districts to measure student perceptions of their relationship with their teachers, the classroom environment, and their teachers' ability to engage them in the material.

Teachers' Ability to Recognize and Diagnose Student Misperceptions: (Independent Researchers: Educational Testing Service) In Year 2 of the project (2010-2011), researchers will assess participating teachers' ability to recognize and diagnose common student misperceptions in their grade level and subject. The University of Michigan's Mathematical Knowledge for Teaching (MKT) rubric has been developed for this purpose in Mathematics. ETS will be developing other assessments for ELA.

Will any other data be used?

Yes. The research team will be looking at historical achievement and administrative data for the past two school years (2007-2008 and 2008-2009). Researchers may ask some teachers to validate the accuracy of this data..

Can a district substitute end-of-course assessments for supplemental assessments?

No. The project requires that the supplemental assessments be identical in all research sites.

How are classes for video selected?

Any teacher who opts into the study will be videotaped four times during each of the 2009-2010 and 2010-2011 school years: At least two of these taped lessons will be scheduled to observe focal topics (core areas of a specific subject). All participating teachers will receive at least 48 hours notice of any videotaping.

How many times will elementary school teachers instructing students in both Math and ELA be videotaped?

Elementary school teachers instructing students in both Math and ELA will be videotaped four times a year, like any other teacher participating in the project. Researchers will record instruction in both subjects on each of those occasions.

How does this project affect our schools' work with current research partners (i.e. university evaluators already doing research work)?

All schools are free to continue existing research with local university partners. Schools should be aware that this research requires some additional work. Schools should feel free to discuss the MET work with university partners. The MET project will not provide any information to these other researchers.

What will be done with the data collected through the project?

In order to inform education reform efforts nationwide, the findings of this project - as well as the tools developed through it - will be made available to the public. None of the individual, teacher-level data collected will be shared with principals or other school or district personnel. If it is determined that aggregated data would prove helpful to school districts, and if such data can be provided without identifying individual teachers, then the data will be provided at school districts' request.

Return to top

School and Teacher Eligibility and Selection

Will all schools/teachers be eligible to participate?

No. Due to research constraints, such as the need to make comparisons across project sites, not all district schools or teachers will be eligible to participate. To be eligible, each of the following must be true:

Teachers are eligible to participate in this project only if they can be assigned to teach any of the classes in their given subject/grade combination that will be assigned teachers through the lottery process. If a specialized credential is necessary to teach a particular section, and the teachers to be assigned through the lottery process do not all have the necessary credential, then that class section should not be entered into the lottery assignment process. This eligibility requirement is designed to ensure that principals retain control over the decisions that impact students and that students are assigned only those teachers deemed appropriate by their school principal. Help me to understand how this works.

Why are teachers eligible only if they can be assigned to teach any of the class sections in their given/subject grade combination that will be assigned teachers through the prescribed lottery process?

This eligibility requirement is designed to ensure that principals retain control over the decisions that impact students and that students are assigned only those teachers deemed appropriate by their school principal.

What is the lottery process by which participating teachers must be assigned to classes in Year 2 of the project (2010-2011)? Why is it necessary?

The lottery process by which participating teachers must be assigned to class sections in Year 2 of the project (2010-2011) will be implemented as follows:

The lottery process is an essential design component of the MET project because it will help to ensure that the project's results are not compromised by the composition of the classes to which participating teachers are assigned.

Will teachers with co-teaching situations be able to participate?

In most cases, the research study cannot incorporate team teaching. The researchers will consider cases in districts where a form of team teaching is pervasive and sufficient numbers of these teachers are willing to volunteer.

Will teachers who loop be able to participate?

No. Unfortunately, the research study cannot accommodate looping.

How many teachers will participate?

Approximately 3,700 teachers in a number of districts around the nation will participate in this project.

How will teachers be selected for the project?

Only eligible teachers who are nominated by their principals and sign a consent form can participate in the project. The school must also choose to participate. If researchers select the school, all teachers who volunteered will be included in the study.

Can a teacher participate even if his or her principal does not "opt in?"

No. The principal must "opt in" for a school to participate.

Can a teacher or school "opt out" of the project at any time if they no longer wish to participate?

While we encourage teachers and schools to participate for the entire two years, participation is voluntary for the entire two years, and teachers/schools are free to opt out at any time.

Can teachers in other subject areas or grades participate in the study?

No. Due to the design and nature of the research study, teachers are eligible to participate in the project only if they teach Math and/or English Language Arts in grades 4 through 8, Algebra I at the high school level, Biology (or its equivalent) at the high school, or English in grade 9.

Note: If a teacher changes the school, grade or subject they teach in the 2010-2011 school year, they may not be able to participate in the second year of the study.

Can teachers participate if their elementary school has subject specialists in Math or ELA?

Elementary school teachers who specialize in either Math or ELA may "opt in" for one subject only. There will need to be three (3) specialists per subject in order to participate.

Will schools and/or teachers be compensated for participating?

Yes. Schools and teachers participating in the project will be compensated for their time.

Participating teachers will receive a $1,500 stipend, disbursed in accordance with a schedule negotiated with their individual districts. Schools enrolled in the project will receive $1500 each.

Return to top

Data Security

Will anyone from the district, such as principals, be able to see the data for an individual teacher?

No. Researchers may not share data on individual teachers with principals or other school or district personnel. Principals will receive individual student scores on the supplemental assessments. If it is determined that aggregated data would prove helpful to school districts, and if such data can be provided without identifying individual teachers, then the data will be provided at school districts' request.

Will teachers be able to see their own data?

Participating teachers will, of course, have access to the raw video of their classroom observation sessions. Each teacher will be able to view his or her own video on a secure site within a day or two of the data collection and can use the video for professional development purposes if he or she chooses.

Each teacher will also receive the supplemental assessment results for the students in his or her class. The research team will provide districts with the results of supplemental assessments at the individual student level during the summer immediately following the administration of the assessments.

What will teachers not see?

Teachers will not receive:

Can teachers, schools and districts see the validated rubrics, student survey forms and training materials for videotape reviewers?

We are not able to provide the research tools used in the study to teachers, schools or districts during the study. However, at the end of Year 2 of the project (summer/fall 2011), the research team will provide to each district many of the research tools used as part of the project, including information about the validated videotape observation rubrics, student survey forms, and training materials given to videotape reviewers. Neither the rubrics used by the video observers nor the data compiled by the video observers pursuant to the rubrics will be shared with teachers, principals, or district staff.

Return to top

Time Commitment

What is the research timeline for the study?

Fall 2009

Winter/ Spring 2010

Fall 2010

Winter / Spring 2011

How long will the supplemental assessments take, and when will they be administered to students?

The supplemental assessments will be given in the last 30 days of each school year and will be no longer than 60 minutes per subject at the elementary/middle school level and no more than 90 minutes per subject at the high school level. Students in the upper grades could have one, two or three supplemental assessments depending on whether their ELA/English, Math, and/or Biology teachers are participating in the project. Elementary students with participating teachers will likely have two hours of supplemental assessment (60 minutes each for ELA and Math, given self-contained classrooms).

How much additional time is likely to be required of teachers

The research team has designed the data collection to minimize the burden on participating teachers and students. Researchers estimate that participating teachers will spend no more than twelve (12) additional hours on project activities over the course of a school year. Project activities also will have limited impact on student instructional time. The research team estimates that elementary students in participating teachers' classrooms will spend no more than three (3) additional hours per year on project activities, and high school students will spend no more than five (5) additional hours per year on project activities, including all assessment and survey activities.

Return to top

Union Support

What is the position of the unions in each participating district on this project?

The unions in each district fully support this project and encourage eligible teachers to participate.

Return to top

Student Engagement

How will students be assigned to classes to ensure valid and reliable research findings?

In order to validate the research being gathered, it is necessary to ensure that the project's results are not compromised by the composition of the classes to which participating teachers are assigned. This means that participating teachers in each subject/grade combination must be assigned to classes through a lottery process in Year 2 of the project (2010-2011). The lottery process will be implemented as follows:

This process will accomplish the research goals of the project and ensure that the results are not compromised by the composition of the classes of participating teachers. Moreover, since the only teachers eligible to participate in the MET project are those who can be assigned to teach any of the class sections in their given subject/grade combination that will be assigned teachers through the lottery, the study is designed to ensure that principals retain control over the decisions that impact students and that students are assigned only those teachers deemed appropriate by their school principal. Help me to understand how this works.

Will all students in the classroom take the student assessments and complete the student engagement surveys?

Yes. All students assigned to the classrooms of participating teachers will take the assessments and complete the surveys.

Will students have to consent to participation in the project?

Students and their families will be given the opportunity to "opt out" of participation. Should parents decide that they do not want their children to be included in the project's data collection activities, researchers will not collect surveys from those students and will take steps to ensure that those students do not appear in or are not identifiable in the videotaped classroom observations. Districts and/or schools will provide information about the project and the means of "opting out" of participation to students and their parents.

Return to top

“By identifying what methods work well in a classroom, we have the potential to improve outcomes for many more of our students.”
– Joel I. Klein,
Chancellor,
New York City Department of Education