Data Warehousing and Decision Support at the National Agricultural Statistics Service

Abstract

Easy access to large collections of historical survey and census data and the associated metadata that describes them has long been the goal of researchers and analysts. Solutions to problems such as understanding the behavior of current survey data, respondent burden, improved statistical techniques, and data quality are often found in the careful analysis of historical data. Many questions have gone unanswered because the data were not readily available, access was limited, metadata were not well defined, or query performance was intolerably slow. This article describes the database modeling techniques that permit end users fast and easy access to large amounts of microlevel data contained in different data systems and from different time frames. Also, techniques for tracking metadata changes and standardization are discussed. A generalized dimensional model is presented that can be used for any census or survey to track the full history of the data series.