Experimental design influences on stated choice outputs: An empirical study in air travel choice

Discrete choice experiments are conducted in the transport field to obtain data for investigating travel behaviour and derived measures such as the value of travel time savings. The multinomial logit (MNL) and other more advanced discrete choice models (e.g., the mixed MNL model) have often been estimated on data from stated choice experiments and applied for planning and policy purposes. Determining efficient underlying experimental designs for these studies has become an increasingly important stream of research, in which the objective is to generate stated choice tasks that maximize the collected information, yielding more reliable parameter estimates. These theoretical advances have not been rigorously tested in practice, such that claims on whether the theoretical efficiency gains translate into practice cannot be made. Using an extensive empirical study of air travel choice behaviour, this paper presents for the first time results of different stated choice experimental design approaches, in which respective estimation results are compared. We show that D-efficient designs keep their promise in lowering standard errors in estimating, thereby requiring smaller sample sizes, ceteris paribus, compared to a more traditional orthogonal design. The parameter estimates found using an orthogonal design or an efficient design turn out to be statistically different in several cases, mainly attributed to more or less dominant alternatives existing in the orthogonal design. Furthermore, we found that small designs with a limited number of choice tasks performs just as good (or even better) than a large design. Finally, we show that theoretically predicted sample sizes using the so-called S-estimates provide a good lower bound. This paper will enable practitioners in better understanding the potential benefits of efficient designs, and enables policy makers to make decisions based on more reliable parameter estimates.