Asking Good Questions

1/19/2011

In field research we put a lot of emphasis on asking the right questions. We try to ask the right questions in our specific research questions, as well as in the questions we ask in interviews we conducts and the surveys we send out etc. I find that I am often trying to focus my questions so that they are clear and ask exactly what I want to ask. Which is great. Being clear is something I am not so good at, but it is necessary, and I would like to be good at it some day. However, what about the idea of letting questions be open to interpretation? Someone suggested that one of the benefits of in-depth interviews is that questions are open to interpretation. Now, I guess this could mean, from my paradigm, that you are able to delve further into questions, and get more than surface level answers. However, I also thought about truly having a question open to interpretation. Let me organize my ideas into paragraphs.
When preparing for cross-cultural research, we try to ask our questions in such a way that people from the culture we are in will be able to answer the questions. If they answer a question other than the one that we asked, we try to reframe our original question so that they will understand and answer it. This way we can collect the data that our research question told us to collect. This all makes sense. And it’s good. However, the question that I raise, that has probably been raised before (I do not pretend to be insightful), is what can we learn from answers to questions other than the ones that we pose. Here is a great example of how I am not clear. What I mean is, what can we learn from answers to ‘misunderstood’ questions? If that makes any more sense. Perhaps an informant was answering the question that should have been asked, rather the one that was asked. Let me give you a silly example. In the recent political discourse, politicians and pundits have tried to tone down the dialogue so that it is less violent. An amendment to an earlier passed bill has been proposed by the Republican Party that has a title that uses the word ‘killing’ in there somewhere. Democrats immediately jumped on that and pointed out that the word is violent and should be removed from the civil discourse. This is apparently a big issue, because it has been receiving a lot of commentary lately (though I do not mean to undermine the work to get rid of violent phraseology from the civil discourse. In fact, I am all for it). On a Fox News segment, they had a Republican and a Democrat weigh in on the issue. When they asked the democrat, he expressed his thoughts on the bill and why he thought the amendment shouldn’t be passed. The Fox News correspondent leading the discussion interrupted him and told him that he answered the wrong question, and he was supposed to talk about the wording of the amendment title. They then talked to the Republican about the amendment, and he said something about how the title was appropriate because the bill would in fact kill jobs, or at the very least destroys them, or something. When the correspondent asked for the democrats final thoughts, he again talked about how the bill wouldn’t kill jobs, and he was interrupted again and told how he answered the wrong question. Then the correspondent laughed, because the democrat just didn’t get it. But when you think about it, maybe the democrat did get it. Maybe he realized that while removing vitriol from our political dialogue was important, what was really important for Americans to know was what was in the bill itself, and what is in the amendment. The correspondents correcting of the democrat made it seem like understanding the issue wasn’t important.
In my own research, I wonder if I ever seem like correspondent. Perhaps I laugh too quickly when my informants answer the question I didn’t ask. What might be gained from learning from those answers? How can we learn from those answers? Are they still data?
I don’t really know how to answer those questions really. But I guess I can try. I think we can learn more about our own paradigms and biases by really listening to misunderstood answers to questions (and yes, I still haven’t found a satisfactory way to refer to them. Deal). But what I don’t know is how one can delve more deeply into those answers other than asking “what makes you say that?” or “what about my question wording made you answer in the way you did?” I think the second question might not really be an appropriate question in most circumstances anyway.
My third question, are answers to misunderstood questions still data, is more difficult for me to answer. If the response to the question does not answer the question that was so perfectly tailored to our research question, can we still include it for analysis? The logical part of my brain says no, and I can’t really explain why, though I wish I could. However, the part of my brain that is super vague and unclear (i.e. the part of my brain that I use to communicate and write with) says, “Why can’t we just change the way we ask research questions?” Can I write an academically and ethically sound proposal to research masculinity ideology and rugby by saying that I will ask certain questions, but if my informants decide to answer something else, I will just go where their answers take me? I guess that doesn’t get my point across very well, as that sounds like what an unstructured interview is. Let me try again. Can I conduct a research project in which I will go out with a certain question (that I have spent a long time working on, what with literature reviews and such), but if I find that I am learning from answers to questions not asked, just throw my research question out the window? Maybe that is a little clearer. One more question: am I raising these questions to justify my own vagueness and inability to write a clear research question? I don’t think so, but I guess I can never really be sure of what goes on in (what psychologists would call (not that I’m saying I believe what psychologists tell me)) my subconscious.

As a complete side note, a couple is taking engagement pictures in the JFSB courtyard. It’s kind of adorable, but also makes my hipster/anti-mormonculture sensibilities cry.

1/21/2011

In continuation from the previous discussion, I want to discuss why we even write research questions now (pre-field experience) if they are just going to change once we get to the field.
On the one hand, I am very OCD, worry wart, control freak (pick your favorite term and go with it), and I want to have as much information as possible before I go in the field. This might be due to my tenure with the Boy Scouts of America (I was a counselor at a cub scout day cam) and my belief in the motto: Be Prepared. However, on the other hand, I can’t pretend that I know anything about my field of study. I know I’ve been to East London before, I have had some experience with the area, the people, some culture things, but I don’t have the foggiest idea about young rugby players in South Africa. I guess that’s why I’m studying what I’m studying. How can I go in with a question that makes all kinds of assumptions about the area, people and culture? Wouldn’t it be better if I let them define what it is I’m going to study?
I think I am at a part in my project where I am trying to balance the two as best I can. Right now my question is “What are the values that rugby coaches teach their players, and how do those values affect the masculinity ideology of said rugby players?” The wording of this question is subject to change, especially when I can define what I mean by masculinity ideology, if that is even the correct term to use, etc. Another word that I think is significant is ‘values.’ On one hand I want to define it, so that I know exactly what it is I am looking for when I get there, however, what would I be missing out on if I went into the field with my definition of values. I think it would bias my results, for sure, but it would also limit my understanding of the people. Of course, that’s just another type of bias, I guess. Anyway, I recently decided that I am going to let ‘values’ go undefined in my research question. I will still address the issue, but in my methodology section.
Instead of going into the field with a set definition of what values are, I will instead spend the first few weeks of my study having my participants define what values are being taught in rugby. I talked with Ashley briefly about a few options, like free listing, and now I need to find out more about them before I decide what I am going to do exactly. There are a few other things I need to think about since talking to Ashley:
• How do I define masculinity ideology? Is that the right term?
• Learn more about free-listing and similar methods to define values.
• Think about how many schools I’d like to work with (probably 3 at max), and how my project would be affected by how many schools I work with.
• How might I offer reciprocity to the participants in my project
o Perhaps practice some photography
o Perhaps offer that as a skill
o Perhaps buy a new camera
• Who do I want to interview? Who do I want to do free listing with (assuming I choose to do free listing)?
o Coaches?
o Players?
• Would I rather do focus groups with players?
o Parents?
o Spectators?