When choosing the type of question that is best suited to your research question, one of the first steps is to stop and consider the type of data produced by each possible question type. In other words, plan your analysis before you collect your data. How is this done? One easy way is to create a three column Question x Analysis table in MS Word or Excel: Column 1: Column 2: Column 3: Enter the question type and text Enter the possible answer, scales and examples of the answer format: Open-ended text questions, dichotomous, multiple choice, rank order, scaled, or constant sum (ratio scale). Enter the type of analysis supported by the answer scale (percentages, means and standard deviations, cross tabulations, and statistical tests).
TYPES OF QUESTIONS:
More than 100 question types are available in Qualtrics, including questions that give answers in the format of openended text questions, dichotomous choices, rank order, pick-bucket-rank, multiple choice, multiple choice matrix, Likert or Semantic differential scales, constant sum, conjoint, side by side, etc. Each question has an associated measurement scale, that is either nominal (text, categorical), ordinal (rank order or ordered categories), interval (scaled without an absolute zero point), or ratio (scaled with an absolute zero point). These scales are often referred to as either discrete (nominal and poorly ordered scales), or continuous (well ordered, interval, or ratio). Each question, based on the associated answer scales, is appropriate for completing multiple types of analysis. These analyses include the computation of percentages and percentages by percentile groups, frequency counts, means and standard deviations, cross tabulations, and statistical tests (chi-square, t-test, ANOVA, regression, multivariate analyses, etc.).
If instead, we had asked respondents to estimate the probability of winning the championship (0 100%), then this would have resulted in parametric data (ratio scale). The correct analysis for this data type would include computation of means (average probability of each team winning), and variance (distribution of scores for each team). Can you identify the type of data required to do a cross-tabulation analysis? How about a t-test? And finally, can you locate the multiple regression analysis?
One 1. Discrete a. Percentage (Impossible) Variable 2. Continuous a. Median, Mode a. Mean b. Quartile Range b. Std. Deviation
Two 1. Both a. Chi Square a. Correspondence Variables Discrete b. Phi Coefficient analysis c. Contingency Coefficient d. Tetrachoric Corr. (2 rows x 2 columns) e. Yules Q (2 x 2) f. Lambda 2. One Discrete and the other Continuous a. Student t (if dependent variable is dichotomous) b. One-way ANOVA (if dependent variable is multichotomous) 3. Both a. Spearman Rank a. Pearson Correlation Continuous Correlation b. Simple Regression b. Kendalls R c. Eta Curvilinear c. Gamma Correlation
Three Variables 1. Three a. Cross Tabulation a. Multidimensioanl Scaling Discrete b. Correspondence Analysis (Multiple) 2. Two Discrete a. ANOVA (2 Way) One Continuous 3. One Discrete a. Comparison of a. Analysis of Two Continuous proportions Covariance b. Comparison of two correlations (If dependent variable multichotomy) c. Multiple Discriminant Analysis 4. Three a. Kendalls W a. First-Order Continuous Partial Correlation b. Multiple Regression c. Multiple Correlation
Four 1. K Discrete a. Cross a. Quasi-Metric Multidimen Tabulation sional Scaling b. Non-Metric MDS c. CHAID 2. One Discrete a. Prohibit Analysis K Continuous (For normal probabiltiy distribution) b. Logit Analysis (For Logistic Porbability Distribution) c. Multiple Discriminant Analysis 3. K Continuous Variables a. Second-Order Partial Correlation
b. Multiple Regression c. Multiple Correlation d. Factor Analysis e. Canonical Correlation f. Fully Metric Multidimensional Scaling