A colleague pointed out the ANU report Conceptualising and Evaluating Teacher Quality by Ingvarson and Rowe which discusses teacher merit pay schemes in some detail. It’s a bit hidden on the ANU site but is there in PDF format HERE
Among the opening statements, this gem of common sense:
A note about methodological limitations endemic to econometric research focussing on the link between teacher quality and student academic performance is appropriate here (e.g., Hanushek, 1971, 2004; Monk, 1992; Podgursky, Monroe & Watson, 2004; Rivkin, Hanushek & Kain, 2001). Since these limitations are well established, they need not be reiterated here in detail.5 In brief, however, an extensive body of work indicates that the single-level econometric models typically fitted to the available data employing general linear model (GLM) techniques under ordinary-least-squares estimation, are inappropriate on at least two counts. First, they fail conceptualise, measure and evaluate teacher quality in terms of what teachers know (subject-matter knowledge) and can do (pedagogical competence). Second, such models rarely account for the measurement, distributional and structural properties of the data for response and explanatory variables – oversights that all too frequently yield misleading interpretations of findings for both policy and practice.