Abstract

This paper aims at developing the idea that designers can be benefited by experiments when testing products, services and concepts, through a literature review and the use of research examples from the authors’ academic and market experience. It discusses, apart from the method itself, (a) ecological validity and the use of experiments in naturalistic environments and (b) when laboratorial experiments are good enough to solve design problems. Both perspectives – naturalistic and laboratorial – are presented and discussed in the paper. It is suggested that the first choice regarding the experimental setting in design experiments is the artificial scenario (as opposed to naturalistic environments), due to its potential to keep the study’s internal validity high. Only when artificial settings threaten the study’s external validity a naturalistic experiment becomes a better choice. High external validity might not be worth paying the price of low internal validity. Different from other sciences, the design field requires from researchers flexibility when planning experiments, since the usual choices might not be always the best ones to be made.

Keywords

experimental research, experiments, design research, ecological validity

COinS
 
Jul 1st, 12:00 AM

Experiments in Design Research: Testing causality relations among users in naturalistic and artificial environments

This paper aims at developing the idea that designers can be benefited by experiments when testing products, services and concepts, through a literature review and the use of research examples from the authors’ academic and market experience. It discusses, apart from the method itself, (a) ecological validity and the use of experiments in naturalistic environments and (b) when laboratorial experiments are good enough to solve design problems. Both perspectives – naturalistic and laboratorial – are presented and discussed in the paper. It is suggested that the first choice regarding the experimental setting in design experiments is the artificial scenario (as opposed to naturalistic environments), due to its potential to keep the study’s internal validity high. Only when artificial settings threaten the study’s external validity a naturalistic experiment becomes a better choice. High external validity might not be worth paying the price of low internal validity. Different from other sciences, the design field requires from researchers flexibility when planning experiments, since the usual choices might not be always the best ones to be made.

 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.