What work can be done in the meantime?
I am currently working through a revise and resubmit for an article and was struck by a comment from one of the reviewers (and no it wasn’t reviewer #2 😜), and the handling editor. They stated that the methods employed in the project weren’t rigorous enough. On the surface, this is a totally fair comment. I am not perfect and there are lots of ways to do any one project. So, I reached out to the handling editor to get clarity on what they meant by the comment so I could make sure to fully address it in the resubmission.
In our correspondence I learned the critique wasn’t leveled at what we did in the project, which was to administer a survey, but rather what they thought we should have done. It turns out project was just too simple. Our stated objective was to assess the perspective of one population, because their perspective had not previously been addressed in the literature. Despite this, the reviewer and the editor felt we should have included other populations to make additional comparisons and in so doing make the work more “robust”.
I’ve been thinking a lot about this comment, doing impactful work, and where academics currently find themselves with respect to the evolving funding environment. Though it is not my area of expertise, I have great appreciation and respect for randomized trials, complex statistical analyses, and Big Data. I have also been part of teams employing all these approaches and data sources at various points in my career. Working towards establishing causal relationships to maximize health outcomes is important. Trawling the great repositories of secondary data for previously unseen patterns and ideas has tremendous potential for helping us to guide resource allocation and impact the largest number of lives possible.
However, these approaches can’t answer ALL research questions. Randomized trials require a hypothesis, which typically comes from previous literature. Big Data is often collected for non-research purposes, which leaves researchers to adapt to the data that’s available. To engage in these currently coveted types of projects researchers stretch and reconfigure research questions, analytic techniques, and conclusions to make use of these valuable resources. And when they can’t stretch any further, they change the questions asked to fit neatly into what’s available or what others have used before. Again, this is not a bad thing, but sometimes I find myself asking, have we gone too far?
One of the things that I love about research is the opportunity to notice patterns in the world and ask questions that no one else is asking. I find this kind of work deeply to be satisfying and so much FUN! Unfortunately, in our current research environment these kinds of questions are often considered “pilot data” used to generate a hypothesis, leveraged for grant funding, and then woven into the story of the results from a large trial down the road. By themselves these questions are stepping stones to something bigger.
While I have always been biased to these kinds of questions, my case to you today, and in our current uncertain funding environment, is that these questions matter. What kinds of studies are we talking about here, well think scoping reviews, interviews, focus groups, locally conducted surveys with non-panel participants. All small scale and built on relationships and partnerships with people local to you. When compared to purchasing datasets, panels, or having the infrastructure to run a large trial these kinds of projects are relatively cheap and can be done with resources to which you already have access. (Something else that I increasingly believe is that this local data will give you an edge on implementation down the road, but more on that another day.)
So that’s what I’m going to be doing this summer, starting with scoping reviews and moving onto interview and focus groups, and then on to surveys (and maybe back to interview depending on what the survey says) with people in my community. Simple, straightforward, and all things I can do with my own two hands (and maybe the hands of a couple of few students 😉) right now. So I would encourage you to protect your peace, take the next best step, and get you and research program ready to hit the ground running in the fall.
Need some help getting started, check this out and come work with me!
(Words: 730)