Survey best practices – (1) develop and follow an objective

Creating and administering a survey is relatively easy – draft a dozen or so questions, load these questions into one of the many online survey tools available, send invitations to a group asking them to complete the survey, then review the responses.

Following these steps, however, does not guarantee reliable and useful survey results. To achieve reliable and useful survey results requires rigorous planning and testing. The first key step is to identify all objectives you have in administering the survey. For example:

Objective

To review student’s food preferences in the school cafeteria in order to better understand what students would like to eat while on campus so we can make informed decisions about food services at X institution going forward.

Notice how the first part of the objective focuses on the subject of the survey while the end of each objective identifies how the data will be used. If, at the outset of developing a survey, you are forced to consider how you will use the data once it is collected you will save yourself (and your respondents) time. Students and staff members are busymake it a rule not to waste their (or your) time asking questions if you have no plan for how to use their responses. 

An important consideration is whether a survey is the right tool to get at the information you seek. In developing your objective, you might realize that an alternative method, e.g. mine existing data (e.g. previous surveys, enrolment stats, etc.) to get your answer or host a focus group. Some of these alternative information gathering methods will be reviewed in future posts.

Once you’ve decided to use the survey methodology and developed an objective, commit the objective to memory. Alternatively: steal my strategy and write down your objective in big font on a piece of paper and refer to it as you add questions to your survey. With each question you consider adding, ask yourself: “Does this question help me reach my survey objective?” Obviously, if the answer is no, don’t include it in the survey (even if it’s a really really interesting unrelated question that you just have to know the answer to).

How do you know if a survey question fulfills the objective? It has to pass through the two parts of your objective. For example:

ObjectiveTo review student’s food preferences in the school cafeteria in order to better understand what students would like to eat while on campus so we can make informed decisions about food services at X institution going forward.

(1) Does your question relate to student’s food preference in the school cafeteria? (Is it related to the topic at hand?)

(2) Will the responses inform decisions about food services at X institution? (Are the results from this question actionable?)

In other words, can we feasibly make a change to an existing policy or service offering if the responses indicate we should?

For example, say you had the following question related to food services: “Would you want X institution to have a late night food option?”  (responses: yes, no, don’t care)

Students may want to have food services available until midnight every night but it probably doesn’t make sense financially to offer this level of service considering the small number of students that are on campus after 10 pm. Moreover, even if enough students would use this service to make it worthwhile, staffing and safety considerations may prevent hours from being extended.

In short, if a survey item is not actionable then don’t include it on the survey.

Over the next few posts, we’ll continue to outline what we have found to be best practices in survey design and implementation. If you have any other tips on survey design please feel free to leave them in the comments section! I may pull a few tips into future posts (with credit, of course).

Cheers,

David (on behalf of the @NAIT_IR team)

Advertisements