Liam Satchell’s Post

View profile for Liam Satchell, graphic

Senior Lecturer in Psychology at University of Winchester

3️⃣Three Key Ideas to Understand Polling👨💻 With the election underway in the UK, there is a lot of discussion about polling. But what poll a good poll? Can we trust the numbers? Here’s the Re-Va-Ge trio from human measurement science to help explain: 🔄Reliability (consistency, repeatability) 🎯Valdity (accuracy, specificity) 👥Generalisability (generalisation) 🔄Reliability - Any data story on one dataset is not enough. Repeatability and replication of key conclusions is essential. Plenty of things can drive fluctuation in a single day a story - odd sampling, temporary mind change, or just response error. 👀LOOK FOR: polls-of-polls is much more informative than any one poll. 🎯Valdity - it’s important to know that the findings of polls are driven by their methodology. How are undecided voters handled? Pollsters have a range of analytical and question tools: •Drop them (inflating voting ratios), •Force them to pick (unlike what happens in an election) •Allocate them by weighting (break down likely direction. •Just report them. The first three don’t reflect what happens in a voting booth (if a Don’t Know goes to vote). Similarly, polls that filter by ‘likely to turnout’ may find different results to broader polls. Polls that ask approval ratings for parties/politicians may find different results to intention to vote polls. These different methdologies lead to different conclusions and the questions asked are worth checking! 👀LOOK FOR: the question put to the sample in the poll you’re reading. What do the footnotes say about ‘Don’t knows’? 👥Generalisability - Who and how many is asked and how they reflect demographics is an important part of polling. Part of this is sample size and wanting as many respondents as possible. But also check where people are sampled from. Most companies have phased out phone polling, but some still do contact people by landline - with demographic implications. Some polling errors in 2016 and 2017 were due to oversampling of certain demographics, but this lesson has been well learned with the development of new polling analysis tools > 👀LOOK FOR - *MRP* (Multi-level Regression and Post-stratification) modelling. Many polling companies put out their MRPs now (YouGov had one today). These models weight the sample polled by the regional demographics of seats. These were highly effective in the 2019 election and work well on retrospective data. These fix the generalisability issues with single samples statistically. 👨💻Elections are quite fascinating times for how the broader public engage with data. But can be confusing if you don’t regularly love talking about stratified data - like I do 😍 Hopefully the 🔄Re🎯Va👥Ge top tips help you keep an eye on any polls you might be interested in!

To view or add a comment, sign in

Explore topics