Observational studies of the effects caused by treatments are always subject to the concern that an ostensible treatment effect may reflect a bias in treatment assignment, rather than an effect actually caused by the treatment. The degree of legitimate concern is strongly affected by simple decisions that an investigator makes during the design and analysis of an observational study. Poor choices lead to heightened concern; that is, poor choices make a study sensitive to small unmeasured biases where better choices would correctly report insensitivity to larger biases. Indeed, perhaps surprisingly, unambiguous evidence of the presence of unmeasured bias may increase insensitivity to unmeasured bias. These issues are discussed with the aid of some theory and a simple example of an observational study.
