Evalguide - Untersuchungsdesign

the participants are assigned to an intervention group (IG) or to a control group (CG) randomly (and IG and CG are therefore equivalent).
a specific ‚treatment’ (X) is introduced, i.e. the intervention group receives the intervention to be analysed (e.g. a training with an e-learning tool) whereas the control group does not (or receives an appropriate control treatment).
the target value (variable in question, e.g. learning performance) is measured in both groups, if needed, before the intervention, but always after the intervention (measurement = O).

The purpose is to exclude all factors which could possibly have an influence on the target value (except the treatment). The differences which arise between the intervention and control group or with respect to the learning performance can be ascribed only to the treatment. An experimental design is necessary, for example, if the added value of a learning technology is to be demonstrated relative to to “more conventional” learning systems.

One-off measurement with a control group:

IG: X --> O

CG: (X_C) --> O

Example: A group (IG) of students is asked to work with a learning program on the topic z and, afterwards, a variable in question, e.g. knowledge on the topic z, is measured. The same measurement is conducted also in the control group (CG) that has not used the learning program (or that has received a ‚control treatment’ (X_C), e.g. having learned with another teaching aid).

The control treatment should be chosen in such a way that both treatments differ only in the factors to be evaluated. For example, in a comparison of technology-based learning with non-technology-based learning, all other learning-relevant factors should be held constant. This means, for example, that both groups should be taught by the same teacher, that the contents must be the same, that the didactic design of the course has to be identical, etc. (S. O. Tergan reports in the presentation “What makes learning successful?” about a great number of learning-related components, which can be downloaded in German at http://www.evaluationsnetz.de/index.php?cat=0&id=10005000&SID / Ergebnisse des Fachgesprächs / Vortrag Lernerfolg).

Before and after measurement with a control group:

IG: O --> X --> O

CG: O --> (X_C) --> O

Example: A group (IG) of students is asked to work with a learning program on the topic z and the variable in question, e.g. knowledge on the topic z, is measured before and after the event. Before and after measurements are also carried out with a control group (CG) that has not learned with a learning program (or that has received a ‘control treatment’ (XC), e.g. having learned with another teaching aid). By contrast to one-off measurement, this design has the advantage that the learning performance can be measured reliably by constructing a difference between the before and after measurement. Furthermore, it can be verified whether both groups already differ in prior knowledge.
l Needless to say, in case of a knowledge test the same questions should not be asked in the after measurement as in the before measurement as the experience of the before test itself can bring about learning. Firstly, a set of test questions should be generated from the learning material and then they should be assigned to the before or after measurement randomly.