NAIT 2014 PD day presentation: A step-by-step guide to developing better, more useful surveys

summary of pd presentation

Hi all,

Apologies for the long delay since our last post! We’ve been busy working on other projects for the institution including launching the Student Engagement survey and preparing the Entering student survey to be launched Fall 2014.

We also spent some time preparing a presentation for NAIT Professional Development days (June 4th and August 28th). Our presentation focused on key steps involved in developing a survey, including:

  • Set your objectives (What do you need to know?)
  • Sampling (Who knows what you don’t know?)
  • Choose a methodology (How are we going to learn what they know?)
  • Develop strong survey questions
  • Launch survey, monitor results
  • Clean data/explore findings
  • Interpret results (through the lens of your objective)

The five key takeaways from our session were:

(1)    The importance of defining your survey objectives. This is important because it prevents scope creep (avoiding “wouldn’t it be interesting to know X….” questions) and survey length. Objectives are also important as a focusing tool during the analysis phase.

(2)    Survey development is an iterative process. Despite the step-by-step format we delivered the presentation, developing a survey is an iterative process. Throughout the survey development process (and into the analysis phase), we consider how a decision at one step will impact other decisions we have made. For example, you may refine or change your objectives based on a discussion about sampling or methodology.

(3)    Question wording is important. Developing survey questions is as much as a science (avoid double barreled and leading questions) as an art form (know your audience, consider emotional triggers that may bias responses).It’s always a good idea to pilot your survey with a few members of your sample population to ensure your questions are clearly written and easy to understand.

 (4)    Be careful when interpreting results (and reading charts from other researchers or media publications!). Remember that correlation does not equal causation (just because two factors are correlated doesn’t mean that one factor is causing the other to occur – see example below).

correlation does not equal causation

Also, always consider whether the author is purposely presenting data in a way to make a point. See Media Matters terrific post covering “A History of Dishonest Fox [News] charts”  for examples of misleading charts.

 (5)    NAIT Institutional Research department is here to help as you develop surveys or interpret results!

We’ll be running another session on August 28th. We encourage all NAIT staff members to attend!

We put together a guide for best practices in survey design that may be helpful as you develop your own surveys. You can access it on our website here.


Welcome to NAIT IR’s “By the Numbers”

HP CentreOur vision for the NAIT Institutional Research “By the Numbers” blog is to create a space to highlight interesting trends in post-secondary education. Often, these trends will relate specifically to NAIT students and staff since much of IR’s incoming data is from our own registrar’s office as well as surveys we have conducted on campus.

Our goals with NAIT By the Numbers:

  1. To promote the use of Institutional Research resources internally at NAIT, including reports, data sets, and survey tools, and, where possible, to provide support for our colleagues who are needing to conduct surveys or analyse data as part of their own work at NAIT.
  2. To engage our colleagues at NAIT and outside of the institution in a discussion about Canadian post-secondary education, focusing on the unique perspective of Canadian polytechnics.
  3. Above all, we plan to tell interesting data-driven stories about NAIT, hopefully providing some useful insight into the latest policies and trends impacting post-secondary education in Canada.

Find us on twitter @NAIT_IR

Thanks for reading!