Critical Consumption of Research

One of my main goals as a professor of research methods (specifically with my practitioner students) is to make sure they are equipped to critically analyze research articles they come across, whether in the media or at work. So you can imagine my excitement when a friend shared this John Oliver video on Scientific Studies. Enjoy!

https://www.youtube.com/watch?v=0Rnq1NpHdmw 

Survey Formatting

My students are often asked to create surveys for workplace data collection efforts. In my classes I try to impress upon them that it's not so easy to do and that there must be lots of thought put into every aspect of survey planning, development/design and implementation. I just saw this post on my favorite data visualization expert's blog about survey formatting. As basic as these sound, they are so very important!

http://stephanieevergreen.com/6-tips-for-surveys/ 

Creating Diagrams

In my classes, I often ask my students to create visuals, whether it be their conceptual framework or visualizing the data they have collected. Data visualization is all the craze recently and for good reason. Visualizing one's conceptual framework and/or data/findings can be a powerful way to summarize important information. And as Stephanie Evergreen in her new book Effective data visualization: The right chart for the right data has argued "we visualize to communicate a point" and that "the research tells us that data are more persuasive when shown in graphs" (2017, p. 4).

But what tools/software can we use to create these visualizations, especially if we don't do this for a living and thus do not pay for sophisticated programs like Tableau? Some students use the charts function in MS Word, but sometimes these are limited. I just learned of a tool online that allows for diagraming and creating flow charts: www.draw.io. While I haven't used it myself, I thought I would share it in case someone else wants to give it a try.

This is yet more evidence that I am always learning things from my students!

For Qualitative Research Students

A colleague shared this fun video another faculty member created for her students of qualitative research. I really like Dr. Bhattacharya's enthusiasm for qualitative research and her use of "super hero" traits that she hopes her students will develop. Qualitative research is so special and I love that she captures how much we learn about ourselves and about the people whose stories we hope to learn and share.

https://www.youtube.com/watch?v=VRCRYfQDH4c&feature=em-share_video_user

Methodological Features and Effect Sizes

In Recent Educational Research News...

A new article of interest was recently published in the Educational Researcher, which is arguably one of the most credible journals in educational research and is an official publication of the American Educational Research Association. So basically, we trust the stuff that's in here. In this article by Alan C. K. Cheung and Robert E. Slavin, titled "How methodological features affect effect sizes in education," the authors examined the effect sizes of 645 studies and looked to see whether there were differences between the different studies in terms of methodological features. If you've taken a basic methods course or an intro level statistics class, you will (hopefully) have learned that effect sizes are a way that researchers measure the practical significance of results, usually looking at differences in outcomes between groups, often receiving different "treatments" or "interventions."

So what did the authors find? It's more complex than this but, in a nutshell, they found that:

• Smaller studies (with smaller sample sizes) had twice the effect size compared to larger studies
• Published studies had higher effect sizes when compared to unpublished studies (like dissertations)
• Quasi-experimental studies had higher effect sizes than randomized studies
• Studies using researcher developed measures had higher effect sizes than those using independent, standardized measures.

What Cheung and Slavin argue with their findings is that we can't simply look at effect sizes to the exclusion of the methodological features of that study. They state, "...it is clear that researchers as well as policy makers need to take into account research design, sample size, measures and type of publications before comparing sizes from program evaluations" (Cheung & Slavin, 2016, p. 290). In some cases, certain studies overstate effect sizes and thus, overstate program impacts. Check out the study for some specific recommendations advanced by the authors.