|
 |
||||||||
Study design
"A good evaluation design is one that fits the
circumstances while yielding credible and useful answers to the
questions that motivate it." Rossi, Lipsey & Freeman (2004, p. 32)
A study design describes which data should be collected from which objects, when and how often, and which measures are to be taken to eliminate possible biases which might affect the results.
A disctinction can be made between three different design types can be distinguished: non-experimental designs, quasi-experimental designs and experimental designs.
Non-experimental designs
Many of the questions presented in this handbook for project evaluations are not designed to produce any changes to the study design as in principle the “only” concern is to verify whether a predefined result profile has been achieved or not (cf. Wottawa & Thierau, 2003). Thus, non-experimental study designs can be used.
The following types fall under
the category of non-experimental designs:
- Cross-sectional designs: Investigation at a single point in time (e.g. survey, document analysis).
- Longitudinal designs: The same investigation is conducted at several different times (e.g. survey of computer-related attitudes in the first-term students over several years to collect data about the change of the attitude in different ‚cohorts’).
With non-experimental designs, it cannot be determined whether the object of evaluation has been in fact the cause of the goal achievement or not. This is only relevant, for example, in the case of an analysis on the reaction level as to whether the goal of “greater satisfaction with the teaching” has been achieved by the e-learning instrument. This result can
also potentially be influenced by other factors such as by changes in expectations on the part of the students.
This means that possible positive or negative factors that fall outside the scope of the project should be kept in mind and considered in the interpretation of the results. Furthermore, the goals should be formulated as specifically as possible (what does “greater satisfaction with the teaching” mean?), and the effect of the developed product should be analysed as directly as possible, i.e. close to the actual intervention.
For further information about study design:
Wottawa & Thierau (2003, p. 114-143), Rossi, Lipsey & Freeman
(2004, p. 31-65), Weiss (1998, p. 87-96), Sapsford & Jupp (1996),
Gabler & Hoffmeyer-Zlotnik (1997), Bortz & Döring (2003).
