Fact Sheet FS995
Surveys can be an effective means to collect data needed for research and evaluation. However, the method is often misused and abused. The challenge is to design a survey that accomplishes its purpose and avoids the following common errors:
This fact sheet provides guidance for constructing questionnaires and developing procedures to administer them so they achieve valid and reliable results. This is not difficult if a logical process is followed.
Questionnaires are typically used for survey research, to determine the current status or "situation." They are also used to measure the difference in status "before" and "after" to determine changes that may be attributed to an educational program. Before creating a questionnaire, start by asking yourself a few important questions:
It's a good idea to start with research questions or objectives. Here are some examples:
As with determining the purpose, this should be based on the objectives of your educational program and the evaluation of its outcomes and impact. Consider which of the following you are aiming to measure:
Of course, it's possible that you might measure more than one . But the questions will be clearly different based on the information you are trying to gather. Refer to the RCE fact sheet FS869, "Measuring Impact of Educational Programs," to learn more about the types of outcomes that can be measured.
To ensure that the survey instrument you develop is appropriate for your audience, "field test" your questionnaire with other people similar to your respondents before administering the final version. This will allow you to improve unclear questions or procedures and detect errors beforehand. Following recommendations in this guide pertaining to questionnaire design and wording of questions will reduce systematic "measurement" error, which will improve the internal validity of your study.
See the RCE fact sheet FS996, "Choosing a Data Collection Method for Survey Research," for more information about the advantages and disadvantages of each method.
Use scales that provide the information needed and are appropriate for respondents. Some choices are:
Reliability is a measure of how consistent the results of using a measurement instrument (e.g. a test, questionnaire) will be. Reducing "random" error in questionnaires by removing "quirky" questions or changing their arrangement improves reliability. Various methods are available to measure the reliability of an instrument, based on the type of instrument and its purpose. Major statistics software packages provide support for reliability. The method used depends on the type of questionnaire used. Reliability is often first determined using a "pilot test" with the proposed questionnaire and might also be repeated with the final version. When using an existing, commercially available instrument, reliability measures are generally reported for the audience for which it was intended. Examples of reliability measures are:
Copyright © 2017 Rutgers, The State University of New Jersey. All rights reserved.
For more information: http://njaes.rutgers.edu.
Cooperating Agencies: Rutgers, The State University of New Jersey, U.S. Department of Agriculture, and County Boards of Chosen Freeholders. Rutgers Cooperative Extension, a unit of the Rutgers New Jersey Agricultural Experiment Station, is an equal opportunity program provider and employer.
Search This Site: