Survey best practices – (2) choose an audience and delivery method

studentsIn the last post, we discussed the importance of setting concrete objectives as a first step in developing a survey. Once you have clearly articulated your objective, the next step is to identify who you want to fill out the survey. After going through the process of developing your objective you should have some idea of what your sample population will look like.

In this step, you want to make explicit who will be included (and excluded) in your survey sample as well as decide whether information collected will be confidential (name included, but not shared with anyone) or anonymous (name not asked of respondents).

Your survey sample should reflect your survey objective. Who you decide to survey will change depending on what your research objectives are. For example, consider the required survey sample for the following objective:

To review student’s food preferences in the school cafeteria in order to better understand what students would like to eat while on campus so we can make informed decisions about food services at X institution going forward.

We obviously want to sample X institution’s students, but we need to be more specific:

  • Should we ask all enrolled students (on-campus, at satellite campuses, online learners)?
  • Should we ask only those students who actually eat on campus or all students to examine why these students choose not to eat on campus?
  • Should we ask both credit and non-credit students?

My point is not to be exhaustive with the above questions but rather to highlight the importance of considering who you want to included (and exclude) from your sample.

At this point you should also consider how much information about the respondents you want to collect and whether survey responses will be merely confidential or anonymous. Unless you plan to follow-up with respondents (and therefore need to know their name and contact details) or link their responses to another data set (for example, linking Student Engagement Survey responses to responses on an Entering Student Survey to track changes in views/attitudes over time) anonymous responses is likely sufficient.

Once you have decided on who your sample is going to be and whether their responses will be treated anonymously or confidentially, the next step is to decide on a data collection method. There are a number of options, including mail-outs, telephone, face-to-face interviews, and web-based (email) surveys. Each option has associated pros and cons such as cost, human resource (i.e. time) requirements, and response rate. The method you choose should align with the objective of the survey with consideration of the population being surveyed. Other realities such as cost and delivery deadlines may also influence this decision.

In our example, the cost and time commitment to complete a telephone, F2F, or mail-out survey would be far too great. Moreover, most students have access to email so our chosen data collection method is fairly clear – email invitations to participate in a short survey.

At this point, you should have:

  • Developed a clear and detailed survey objective
  • Developed a description of your sample, including who is included and who is NOT included
  • Made a decision whether your sample’s responses will be treated confidentially or anonymously
  • Chosen a data collection method based on your objectives and survey sample

For the next post we will get into the creation of survey items, including deciding on a measurement scale and the proper wording of questions.



Follow us on Twitter @nait_ir