Do you enjoy a challenge? My assignment was to determine if specific harvest skills had been transferred from our workshop to lead farmers and then on to individual farmers in remote villages in Rwanda. (Figure 1)
Figure 1. Kirehe, Rwanda: Members from two farmer cooperatives (Kimaranzara and Tuzamure Ubukungu) holding audience response devices.
Completing this assignment required coming up with answers to four key questions:
-
How can you determine if new skills are being applied in a remote location?
-
How can you expand the size of a focus group and still ensure everyone is heard?
-
How can you get qualitative and quantitative data to analyze training outcomes?
-
How can you create an engaging experience for the participants and the interviewer to learn together?
I discovered that, with an audience response system, it is possible to collect meaningful data in the most remote places. In this article, I’ll share my experiences with you, and some lessons learned that you can apply no matter where in the world your particular challenge is located.
The “back story”
The United States Agency for International Development (USAID) has been collaborating with the government of Rwanda to ensure adequate food supply for the citizens of this landlocked nation. One strategy has been to increase the local production of critical staple crops such as maize (corn) and beans.
To this end, our company (ACDI/VOCA) had been selected to work on a program to help 25,000 farmers reduce crop loss during and after harvest. Specifically, farmers are to apply new skills to better collect, process, dry, and store their crops.
To reach this large number of farmers, we focused on existing agriculture cooperatives that could help us to facilitate cascade training. In cascade training, cooperatives identify and send lead farmers to our harvest training. After being trained, these lead farmers return to their communities and train other members of their cooperatives.
Twenty-four cooperatives were selected to participate in the program. We trained eight cooperatives in first season. After the training, we assessed the program and made adjustments before training the other sixteen cooperatives.
In January 2011, the program trained 240 lead farmers from eight cooperatives in Kirehe, a southeastern district near the Tanzania border. In March, I went to Rwanda to assess the results of the cascade training. My assignment was to determine if specific harvest skills had been transferred from our workshop to lead farmers and then on to individual farmers in the remote villages of Rwanda.
Clicking: giving everyone in the room a voice.
I used audience response devices from Turning Technologies. I had thirty responders (affectionately nicknamed “clickers”) and one handheld receiver. Individuals respond to multiple-choice questions with a clicker, and I capture the aggregate results in my handheld receiver (Figure 2).
Figure 2. Audience response devices from Turning Technologies: receiver (left) and responder/clicker (right)
I met with over 130 members from all eight cooperatives. With clickers, I was able to increase the size of my focus groups to 15-30 people. (Figure 3)
Figure 3. COACMU and COACLMA farmer cooperatives
Focus groups are an effective way of collecting information, but the focus group leader must make efforts to compensate for outspoken participants, response conforming, withdrawal, and atypical responses. Using audience response devices allowed me to address each of these concerns and to provide a more comprehensive report to the project staff.
Outspoken participants. An interviewer dreads prolonged silence after asking a question, but even worse is a participant who monopolizes the time. The interviewer will try to call on others to speak, but this can make others uncomfortable by thrusting them in the spotlight. With clickers, everyone has a chance to immediately respond. It also gives everyone more time to prepare their comments. From the responses, the interviewer can ask questions such as, “I’d like to hear from those who selected option #2” to prompt different people, even those in the minority, to speak. This strategy was effective when I was asking the participants about moisture testing. For farmers, testing moisture is critical in knowing when a crop is ready to sell. Many farmers are loudly adamant about using an older tasting technique. After asking which moisture testing method the farmers used, I noticed that two participants had selected a newer method. I was able to draw out their story for others to learn from their experience.
Response conforming. Participants might be inclined to adjust their initial responses to conform with perceived leaders or towards answers that appear to be more accepted. Clickers democratize responses so that the interviewer gets a truer impression of participant opinions. When I asked participants to tell me how many hours they spent training other farmers, verbal responses tended to norm around the first answer given. When I used clickers, I got a broader range of answers (and, typically more accurate when I compared them to time estimates given by trainees).
Withdrawal. This can occur when there are outspoken participants (see above). However, withdrawal can occur with any focus group, as people are passive while waiting for a turn to speak. Clickers allow everyone to respond immediately after a question. Participants are also eager to hear the results. Hearing how others have responded is valuable information to individuals assessing their own behaviors and performance. I have witnessed groups with clickers buzz with energy as they await the next question and, most importantly, the results.
Atypical responses. Focus groups can provide in-depth stories and examples. But, an interviewer must determine the commonality of an individual’s experience. A compelling story might be unfairly weighted as an example of the collective experience. Others in a focus group may be reluctant to provide contradictory evidence. Clickers provide a foundation of quantifiable information upon which stories can be put into context for frequency and commonality. I had one farmer tell me about the long distance he had to travel to take his crops to market. With the clickers, I was able to create ad hoc survey questions. A quick survey with clickers informed me that his situation, although a real challenge, was not a common problem.
