TLC: Research and Evaluation

Research

To assess the progress of institutional change, each TLC institution was asked to complete an Implementation and Assessment Plan (IAP). IAPs can be thought of by the universities as a strategic plan for strengthening science teacher preparation on each campus. Institutions were asked to identify goals to increase the quantity, quality, and diversity of the science teachers they produce; to select strategies from the Analytic Framework (AF) to help achieve and sustain their institutional commitments; to identify who will have the primary responsibility for moving each of the strategies forward; and to identify success indicators, selected from the AF, to monitor progress towards success.

To test the theory of action that institutional change depends both on top leadership commitment, and faculty ownership of the actions, four key research questions were addressed:

1. Does the intensity of senior institutional management involvement impact the likelihood of institutional success in meeting its objective
   • Document involvement via teams, team leader reports and interviews
   • Document effectiveness of provosts’ meetings to stimulate specific changes at institutions
2. Does the composition of institutional teams impact the likelihood of institutional success in meeting its objectives
3. Document development and composition of teams
4. Assess relationship to outcomes across institutions
5. Document institutional processes of developing and implementing IAPs through regular contacts and systematic short surveys
6. Assess usefulness from external evaluation reports
7. Were goals achieved? How? Why not?
8. What evidence is there that change was institutionalized?
9. What were the barriers to success?
10. Were IAPs used to guide progress towards objectives? If so, were they useful?
11. What changes took place within institutions that appear to contribute to improved physics, chemistry and/or mathematics teacher preparation?

IAPs provided a common framework for gathering information that allowed us to compile and analyze the various goals and approaches taken among TLC institutions. Subsequent progress reports, data collection, meetings and site visits allowed us to document what was learned and what barriers emerged.

Evaluation

WestEd undertook the evaluation of this project. Their major question was – did the involvement of a national association, in collaboration with one or more national disciplinary societies, add value to institutions’ efforts to increase the quantity, quality and/or diversity of physics, chemistry and/or mathematics teachers produced by the institutions?