Document(s) 6 of 27

Flowchart: Steps in the development of a health systems research proposal

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 3: IDENTIFYING AND PRIORITISING TOPICS FOR RESEARCH

1. Identification of topic
2. Criteria for prioritising topics for research
3. Nominal group technique

I. IDENTIFICATION OF TOPIC

In the previous module, a number of research questions were presented that may be posed at the various levels of the health system.

These questions can be placed in three broad categories, depending on the type of information sought:

1. Description of the health situation, required for planning interventions

   Planners need to know, for example, the magnitude and distribution of health needs in a population as well as of services; the risk factors for certain problems and people’s awareness; the utilization patterns and cost-effectiveness of available and potential other interventions, in order to formulate adequate policies and adapt or plan interventions.

2. Information required to evaluate ongoing interventions, for example with respect to:
   • coverage of priority health needs
   • coverage of target group(s)
   • acceptability and quality
   • cost-effectiveness
   • impact on health,

   to assess progress and the need for adjustment on a routine basis.

3. Information required to define problem situations in interventions in any of the fields mentioned under 2, and to analyse possible causes in order to find solutions. These causes may include lack of or inequitable distribution of resources, vague policies, and any environmental factors affecting needs, interventions and resources (See Figure 2.1).

Although research in support of planning and evaluation (categories 1 and 2 mentioned above) is an important focus for HSR, the modules will emphasise problem-solving research, because health managers are frequently confronted with problems of this type. It is assumed, however, that research skills acquired in the present course will be of use in the broader field of planning and evaluation as well. In Modele 4, moreover, all three types will be treated.

Whether a problem situation requires research depends on three conditions:*

1. There should be a perceived difference or discrepancy between what exists and the ideal or planned situation;
2. The reason(s) for this difference should be unclear (so that it makes sense to develop research questions); and
3. There should be more than one possible answer to a question or more than one solution to the problem.

* This paragraph has been adapted from Fisher et al. (1983)

For example:

Problem situation:

In District X (pop. 145,000) sanitary conditions are poor (5% of households have latrines) and diseases connected with poor sanitation, such as hepatitis, gastro-enteritis and worms, are very common. The Ministry of Health has therefore initiated a sanitation project that aims at increasing the percentage of households with latrines by 15% each year. The project provides materials, and the population should provide labour. Two years later, less than half of the target has been reached.

Discrepancy:

35% of the households should have latrines, but only 15% do have them.

Research question:

What factors can explain this difference?

Possible answers:

1. Service-related factors, such as forgetting to adequately inform and involve the population, bottlenecks in the supply of materials, differences in training and effectiveness of sanitary staff, lack of co-operation between sectors.
2. Population-related factors, such as situations where community members lack understanding of the relationship between disease and sanitation or have other problems, for example due to poverty, which they consider more important.
3. Physical factors/ecosystems, such as hard soil, or land subjected to frequent flooding.

II. CRITERIA FOR PRIORITISING TOPICS FOR RESEARCH

Because HSR is intended to provide information for decision-making to improve health care, the selection and analysis of the research topic should involve those who are responsible for the health status of the community. This would include managers in the health- and health-related services, health care workers and community leaders, as well as researchers.

Each topic that is proposed for research has to be judged according to certain guidelines or criteria. There may be several ideas to choose from. Before deciding on a research topic, each proposed topic must be compared with all other options. The guidelines or criteria discussed on the following page can help in this process:

Criteria for selecting a research topic:

1. Relevance
2. Avoidance of duplication
3. Urgency of data needed (timeliness)
4. Political acceptability of study
5. Feasibility of study
6. Applicability of results
7. Ethical acceptability

1. Relevance

The topic you choose should be a priority problem. Questions to be asked include:

• How large or widespread is the problem?
• Who is affected?
• How severe is the problem?

Try to think of serious health problems that affect a great number of people or of the most serious problems that are faced by managers in the area of your work.

Also, consider the question of who perceives the problem as important. Health managers, health staff and community members may each look at the same problem from different perspectives. Community members, for example, may give a higher priority to economic concerns than to certain public health problems. To ensure full participation of all parties concerned, it is advisable to define the problem in such a way that all have an interest in solving it. Even within villages, opinions may differ on how important a problem is. It is therefore obligatory to discuss the problem with community leaders, as well as peripheral villagers, males as well as females, rich and poor, exploring their perceptions of the problem.

2. Avoidance of duplication

Before you decide to carry out a study, it is important that you find out whether the suggested topic has been investigated before, either within the proposed study area or in another area with similar conditions. If the topic has been researched, the results should be reviewed to explore whether major questions that deserve further investigation remain unanswered. If not, another topic should be chosen.

3. Urgency of data needed (timeliness)

How urgently are the results needed for making a decision or developing interventions at various levels (from community to policy)? Consider which research should be done first and which can be done later.

4. Political acceptability

In general it is advisable to research a topic that has the interest and support of the local/national authorities. This will increase the chance that the results of the study will be implemented. Under certain circumstances, however, you may feel that a study is required to show that the government’s policy needs adjustment. If so, you should make an extra effort to involve the policy-makers concerned at an early stage, in order to limit the chances for confrontation later.

5. Feasibility

Look at the project you are proposing and consider the complexity of the problem and the resources you will require carrying out your study. Thought should be given first to manpower, time, equipment and money that are locally available.

In situations where the local resources necessary to carry out the project are not sufficient, you might consider resources available at the national level; for example, in research units, research councils or local universities. Finally, explore the possibility of obtaining technical and financial assistance from external sources.

6. Applicability of possible results/recommendations

Is it likely that the recommendations from the study will be applied? This will depend not only on the management capability within the team and the blessing of the authorities but also on the availability of resources for implementing the recommendations. Likewise, the opinion of the potential clients and of responsible staff will influence the implementation of recommendations.

7. Ethical acceptability

We should always consider the possibility that we may inflict harm on others while carrying out research. Therefore, review the study you are proposing and consider important ethical issues such as:

• How acceptable is the research to those who will be studied? (Cultural sensitivity must be given careful consideration). Is the problem shared by target group and health staff/researchers?
• Can informed consent be obtained from the research subjects?
• Will the condition of the subjects be taken into account? For example, if individuals are identified during the study who require treatment, will this treatment be given? What if such treatment interferes with your study results?
• Will the results be shared with those who are being studied? Will the results be helpful in improving the lives or health of those studied?

These criteria can be measured by the following rating scales:

SCALES FOR RATING RESEARCH TOPICS

Relevance

1. = Not relevant
2. = Relevant
3. = Very relevant

Avoidance of duplication

1. = Sufficient information already available
2. = Some information available but major issues not covered
3. = No sound information available on which to base problem-solving

Urgency

1. = Information not urgently needed
2. = Information could be used right away but a delay of some months would be acceptable
3. = Data very urgently needed for decision-making

Political acceptability

1. = Topic not acceptable to high level policymakers
2. = Topic more or less acceptable
3. = Topic fully acceptable

Feasibility

1. = Study not feasible, considering available resources
2. = Study feasible, considering available resources
3. = Study very feasible, considering available resources

Applicability

1. = No chance of recommendations being implemented
2. = Some chance of recommendations being implemented
3. = Good chance of recommendations being implemented

Ethical acceptability

1. = Major ethical problems
2. = Minor ethical problems
3. = No ethical problems

III. NOMINAL GROUP TECHNIQUE

The nominal group technique (NGT) is a group discussion technique that is useful when one wants to obtain a consensus from a group on a topic where decision-making can be usefully guided by the perceptions and opinions of the various group members. The sequence of the group discussion is usually as follows: individual expression, followed by ‘voting’, followed by discussion, and another round of ‘voting’ followed by discussion etc. The group discussion comes to an end when the results of the last vote are not appreciably different from the last-but-one vote.

Steps in applying the nominal group technique*

Participants (between 6-10, all familiar with the content area being explored) are assembled in a quiet room. They are seated in a U-shaped setting so that all participants can see the display (board, flipchart or overhead). The moderator is a non-participant (in our case a course facilitator) who explains and then guides the participants through the process. The steps of the NGT process are summarised below:

1. Individual listing of ideas on paper.

   In HSR courses, participants of a team each write one, at most two priority problems for research on a piece of paper. This is done in complete silence to prevent the group from becoming judgmental about the ideas too soon. The sheets are collected.

2. Display of lists produced, followed by discussion.

   The facilitator takes each sheet of paper and displays all sheets on the board so that all team members can see them. The leader requests the members to briefly explain their ideas and why they suggested them.

   No comments are made by the group at this time, but as the ideas for research topics are presented the rest of the group should study them and see whether they understand what the ideas are and why they are important. If clarification is needed, this is done after all ideas have been presented. Then the participants may attempt to combine overlapping ideas (for example, two almost similar research topics could be combined into one).

3. Voting and ranking.

   After the ideas have been clarified the facilitator asks the participants to select a certain number of ideas on the display (for example, five) that they consider most important, write them on a sheet of paper and rate them. The rating system used can vary but should be fixed in advance: for instance, 5 for the most important idea, 4 for the next most important, etc. The sheets of paper are then collected. For ranking the proposed research topics we would use the criteria and scales presented in the previous section.

4. Summarising the results.

   The facilitator writes each individual rating on the display, next to the idea. All scores are added, resulting in a total score for each idea. The ideas are then ranked, according to the score they received.

   ---

   * Some of the ideas in this process are adapted from Williamson JW et al. (1981) Health Accounting for Quality Assurance. American Occupational Therapy Association.

5. Discussion of the results.

   The results of the first vote are discussed in plenary. All members are urged to contribute. The facilitator may wish to select two types of ideas for clarification: those with high votes and those with divergent votes (i.e. high as well as low weights). A few new ideas may be developed in this discussion. Also, it may be possible to identify a few ‘sleeper’ ideas among those given low votes. Sometimes such ideas may get a high vote when group members understand why the idea was brought forward.

6. Second vote and re-ranking.

   Participants are now asked to vote a second time and the whole process of ranking and discussion is repeated. Voting stops when the results from two consecutive votes do not yield a marked difference. The ranking of the revised final scores gives the order of importance of the ideas as perceived by the group.

Advantages of the NGT

• The discussion process is strictly separated from the voting process and voting may be done anonymously. This depersonalises the process and gives each member an equal vote, regardless of his verbal capacities.
• The results thus reflect input from all members of the group. The series of discussions and anonymous votes helps to minimise the chance that the results will be skewed toward the opinions of one or more dominant personalities.
• The voting process provides a useful means of aggregating individual judgements.

The NGT (or a modified version of the NGT) is particularly useful during the research process in health systems research to:

• assist a group of managers/researchers/community representatives in generating and prioritising lists of topics for which research information may be needed
• assist a research team in selecting a research topic from among alternatives proposed.

REFERENCES

Fisher A, Laing J and Stoeckel J (1983) Handbook for Family Planning Operations Design. New York: The Population Council.

For more reading about the nominal group technique the following literature is recommended:

Abramson JH (1990) Survey Methods in Community Medicine. Epidemiological Studies, Programme Evaluation, Clinical Trials. London: Churchill Livingstone/ Longman Group Ltd (4^th ed.): 188-190.

Delbecq AL, Van de Ven AH, Gustafson DH (1975) Techniques for program planning: A guide to nominal group and Delphi processes. Glenview, Illinois: Scott, Foreman.

Van de Ven AH, Delbecq AL (1972) The Nominal Group Technique. Am Jrnl of Public Health 62:337.

Trainer’s Notes

Module 3: IDENTIFYING AND PRIORITISING TOPICS FOR RESEARCH

Materials

• Flip chart and markers
• Sticky stuff or clear tape, and
• Photocopies of the rating sheet for group work, if possible.

Introduction and discussion

Discuss the process of problem identification (Part I of the module), criteria for prioritising topics for research (Part II) and the Nominal Group Technique (Part III). Be sure you are thoroughly familiar with the concepts but let the criteria as well as the definitions come, as much as possible, from the group. (Before describing the criteria listed in the module, ask the participants to brainstorm, suggesting what criteria they think are most important to consider when selecting a research topic).

Exercise: The Chobe district health team (DHT) - selecting a research project

• Divide the participants in groups of 3-4 people so that they can do the exercise in plenary with minimal displacement.
• Ask the participants to carefully read both examples. Briefly explain how to use the rating sheet at the end of the exercise. Ask the groups to rate both examples, but let certain groups start with the first and others with the second. Give them 15-20 minutes at most to complete the rating process.
• Prepare a flip chart with the list of criteria and write down the ratings of all groups for both topics. Identify the criteria on which the rating differ most (for example, if one group rates a ‘3’ for feasibility of the suicide study and another a ‘1’, ask each group to explain why it decided on its score). The differences may be due to a different understanding of the criteria or to a different perception of the problems and the proposed methodology. Special attention should be given to uniformity in the interpretation of the criteria. After completing this exercise, participants should be able to see the importance of looking at all dimensions of a problem before moving ahead to select their own topics.

Group work

• Review the logistics of supervising the group work with the other facilitators before the session starts. Choose meeting places for the four groups and be sure flip charts, sticky stuff or tape, and markers are available.
• When the selection process for choosing group topics is introduced, make sure that the participants realise they are involved in more than a ‘hypothetical exercise’. Participants should be made aware that they will be developing the topics they select into research proposals during the course and that they will carry out these projects on their return home.
• Familiarise yourself thoroughly with the selection procedures as presented in the group work. These procedures are a simplified version of the nominal group discussion technique. Often one or two priority topics emerge after one round of discussions followed by individual rating and summarising the individual scores on a flip chart, so that a decision can be made during the second round.
• If the total scores of two or three topics are very close, they may be discussed again. It may be useful, in particular, to re-examine criteria that were scored differently by the group members. Special attention should be paid in this round to the question of whether the results of the study will be applicable and whether group members feel they can realistically carry out the research within the 4-6 months allocated.
• As facilitator, you may chair this first group work session but you should not dominate the discussion. You should make sure that the procedures run smoothly and that research topics which duplicate research already completed or which are not feasible are dropped before the rating starts. You should help insure, as well, that no important proposals or initiatives are dropped because the group is not yet familiar with handling the criteria.
• At the end of the group work for selecting projects, assist the reporter in editing and writing the list of topics debated by the group on a flip chart, along with the record of the combined group rating. Ask the secretarial staff to type out lists of topics considered and voting results for possible inclusion as an annex of the final course report.

Document(s) 6 of 27