|
 |
||||||||
Study design
A study design describes which data should be collected from which objects, when and how often, and which measures are to be taken to eliminate possible biases which might affect the results.
The questions presented in this handbook for program evaluation do not result in requirements as to the study design as, in principle, the “only” concern is to verify whether a pre-defined result profile has been achieved or not (cf. Wottawa & Thierau, 2003). This means that, non-experimental study designs can be used.
Non-experimental design
Cross-sectional designs and longitudinal designs fall into
the category of non-experimental design:
- Cross-sectional designs: Investigation at a single point in time (e.g. survey, document analysis).
- Longitudinal designs: The same investigation is conducted at several different times (e.g. performing a network analysis at several points in time with the aim of observing the development of a network over time).
Non-experimental designs cannot predict whether the program has actually been the cause of the goal achievement or not. This is relevant if, for example, it the aim is to analyse at the effect level whether the program has achieved its goals, such as “making learning more flexible”. This result can potentially also be influenced by other factors such as by the Bologna process or other developments which are not related to the program.
Thus, possible positive or detrimental factors outside the scope of the program should be kept in mind and considered in the interpretation of the results. Furthermore, the goals should be formulated as specifically as possible (what does “making learning more flexible” mean?), and the effect of a measure should be analysed as directly as possible, i.e. close to the actual intervention
.
