A GENERAL APPROACH TO THE MISSING DATA PROBLEM

Survey data are often subject to item nonresponse; that is, a response is recorded for some but not all items in a unit. It is well known that if the values are missing at random, then only a slight penalty in terms of increased standard error is incurred by ignoring the missing data. If the missing at random assumption is not valid, as is often the case, then substantial penalty in terms of bias is incurred. This work shows how maximum likelihood estimators can sometimes be found when the item nonresponse is nonignorable. The distribution of the values for the item subject to nonresponse is modeled by assuming a prior probability distribution for the population and a probability of nonresponse conditional on the value of the item. The product of the prior distribution and the conditional distribution is the joint distribution of the item and a random variable indicating whether or not the value is recorded. This joint distribution is defined for every unit. When the value is not recorded, the joint distribution becomes the marginal probability of nonresponse. Assuming the items are independent and identically distributed, the likelihood function is the product of the joint probabilities associated with each unit. This approach is shown to be general in several senses: it is equally applicable to both response and sampling mechanisms; it may be used with any prior or conditional distribution; it applies equally well to multivariate data; and it allows for a test of the missing at random assumption. This approach requires, however, that one know enough about the data to model the conditional probability of nonresponse. Other methods developed for modeling censoring at a fixed known or unknown censoring point are special cases. Simulations and an application to the well known Current Population Survey show the maximum likelihood estimates to have good statistical properties provided the random censoring occurs on only one side of the probability distribution. In relatively small samples the estimators are approximately normally distributed, and they are robust against both lack of normality and outliers.