BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:‘Product-orientation’ among psychometric criteria in developme
 nt and application - Professor Mark Haggard\, Department of Psychology\, U
 niversity of Cambridge
DTSTART:20140225T163000Z
DTEND:20140225T173000Z
UID:TALK49892@talks.cam.ac.uk
CONTACT:Chan Yin Wah Fiona
DESCRIPTION:Like any good science\, psychometrics comprises interplay of p
 rinciples with facts\, theories with data. Part of the theory of contempor
 ary psychometrics is provided by general applied statistics\, but other pa
 rts\, such as IRT or Rasch scaling\, are specific to the application area.
  Not all bulk users of measures of human responses are up to date with tec
 hnical and theoretical developments in psychometrics\, although there is s
 ometimes an elementary 1940s version of psychological measurement on which
  to build. This is seen in three main ways: a very limited box-ticking con
 cept of ‘validity’ (criterion form and consistency index for mixed rel
 iability and validity) prevails. Second\, amid the universal pressures for
  brevity\, the importance of the length of instruments goes largely unment
 ioned\, so is left to wishful thinking\, even though the number of trials 
 or questions is as important for statistical power as the number of partic
 ipants is. Third\, the need to examine equal-interval properties is often 
 ignored\, even though these properties are fundamental to meaningfully app
 lying the most powerful (parametric) statistical modelling techniques\, an
 d when interpreting linearity issues and interaction terms.\n\nMany small-
 sample psychological studies face a particular kind of circularity in atte
 mpting to optimise measurement. Optimising a measure for a particular or p
 urpose can overestimate effect sizes when the data are obtained using a me
 asure developed specifically for that purpose. Large N is a splendid cure 
 for this problem\, failing which\, cross-testing from development sample t
 o a different generalisation (test) sample can be used. Additionally\, to 
 avoid circularity in optimisation\, when a data set is sufficiently rich i
 t may offer various internal ad hoc opportunities\, ie validation paradigm
 s\, for construct validation in the selection and optimisation phases\, th
 at is\, to guide item selection\, scaling or weighting\; this is again the
  use of a different subset of the data\, but by variable\, rather than by 
 case. I illustrate an eclectic approach to measure development by describi
 ng the nested phases in developing a long-form set of questionnaire-based 
 health outcome measures for a randomised clinical trial\, from three datas
 ets\; these have led to the subsequent specification of medium-length (32-
 item) and short-form (14-item) measures for use by others. Providing both 
 reconciles competing objectives of applications (and hence in development)
  so as to allow diverse applications. This strategy is not overly rule-bou
 nd\, but seeks always to maximise prior constraint and replication\, and t
 o minimise capitalisation upon chance.
LOCATION:Seminar Room\, Department of Psychology\, Downing Site\, Cambridg
 e
END:VEVENT
END:VCALENDAR