Dr. Timo Lenzner & Dr. Jan Karem Höhne
Previous research has shown that survey question characteristics, such as the shape of rating scales, can affect how respondents interpret and respond to survey questions. For example, earlier studies reported different response distributions for survey questions employing rating scales in the form of a ladder and in the form of a pyramid. The current experiment, implemented in the probability-based German Internet Panel (N = 4,377), re-visits and extends this research by examining how the two graphical layouts (ladder vs. pyramid) affect response behavior and data quality of a survey question on subjective social stratification. In line with the earlier results, we found that respondents rated their social status lower in the pyramid than in the ladder condition. No differences between the two layouts were found regarding response effort measured in terms of response times, however, the ladder layout was associated with significantly higher criterion validity. Therefore, we recommend employing the ladder instead of the pyramid layout when measuring subjective social stratification.
Reference: Lenzner, T., & Höhne, J. K. (2021). Measuring subjective social stratification: How does the graphical layout of rating scales affect response distributions, response effort, and criterion validity in web surveys? International Journal of Social Research Methodology. DOI: 10.1080/13645579.2021.1874607