Document(s) 18 of 27

Flowchart: Steps in the development of a health systems research proposal

NB: Development of a research process is a cyclical process. The double-headed arrows indicate that the process is never linear.

Module 12: PLAN FOR DATA COLLECTION

1. Introduction
2. Stages in the data collection process

I. INTRODUCTION

Where are we now in the development of our research proposal?

Look again at the flowchart in Module 7 that introduces research design. We have just finished four crucial theoretical sessions, in which we have defined:

• what information we want to collect to answer the research questions implied by our objectives (Module 8: Variables)
• what approach we will follow to collect this information (Module 9: Study types)
• what techniques and tools we will use to collect it (Module l0: Data collection techniques)
• where we want to collect the data, how we will select our sample and how many subjects we will include in our study (Module 11: Sampling)

Now we enter a new phase in the development of our research methodology: planning our fieldwork. We have to plan concretely how we will collect the data we need (Modules 12 and 1 5), how we will analyse it (Module 13), and how we can test the most crucial parts of our methodology (Module 14). Finally, we will have to develop a work plan to budget the resources necessary to carry out the study (Module 16).

A PLAN FOR DATA COLLECTION can be made in two steps:

1. Listing the tasks that have to be carried out and who should be involved, making a rough estimate of the time needed for the different parts of the study, and identifying the most appropriate period in which to carry out the research (the present module).
2. Actually scheduling the different activities that have to be carried out each week in a work plan (Module 15).

Before the proposal development workshop is finished, a pre-test of the data collection and data analysis procedures should be made. The advantages of conducting a pre-test before we finalise our proposal is that we can draft the work plan and budget based on more realistic estimates, as well as revise the data collection tools before we submit the proposal for approval. However, further pre-testing and planning of the research have to be done in the field (Module 14).

Why should you develop a plan for data collection?

A plan for data collection should be developed so that:

• you will have a clear overview of what tasks have to be carried out, who should perform them, and the duration of these tasks;
• you can organise both human and material resources for data collection in the most efficient way; and
• you can minimise errors and delays which may result from lack of planning (for example, the population not being available or data forms being misplaced).

It is likely that while developing a plan for data collection you will identify problems (such as limited manpower), which will require modification of the proposal. Such modifications might include adjustment of the sample size or extension of the period for data collection.

II. STAGES IN THE DATA COLLECTION PROCESS

What are the main stages in the data collection process?

Three main stages can be distinguished:

Stage 1: Permission to proceed

Stage 2: Data collection

Stage 3: Data handling

Stage 1: PERMISSION TO PROCEED

Consent must be obtained from the relevant authorities, individuals and the community in which the project is to be carried out. This may involve organising meetings at national or provincial level, at district and at village level. For clinical studies this may also involve obtaining written informed consent.

Most likely the principal investigator will be responsible for obtaining permission to proceed at the various levels. The Health Research Unit in the Ministry of Health or the institution organising the course may assist in obtaining permission from the national level.

Stage 2: DATA COLLECTION

When collecting our data, we have to consider:

• Logistics: who will collect what, when and with what resources
• Quality control

1. Logistics of data collection

WHO will collect WHAT data?

When allocating tasks for data collection, it is recommended that you first list them. Then you may identify who could best implement each of the tasks. If it is clear beforehand that your research team will not be able to carry out the entire study by itself, you might plan to look for research assistants to assist in relatively simple but time-consuming tasks.

For example, in a study into the effects of improvements in delivery care on utilisation of these services the following task division could be proposed:

HOW LONG will it take to collect the data for each component of the study?

Step 1: Consider:

• The time required to reach the study area(s);
• The time required to locate the study units (persons, groups, records); If you have to search for specific informants (e.g., users or defaulters of a specific service) it might take more time to locate informants than to interview them.)
• The number of visits required per study unit. For some studies it may be necessary to visit informants a number of times, for example if the information needed is sensitive and can only be collected after informants are comfortable with the investigator or if observations have to be made more than once (for example, follow-up of pregnant mothers or malnourished children). Time needed for follow-up of non-respondents should also be considered.

Step 2: Calculate the number of interviews that can be carried out per person per day (e.g., 4)

Step 3: Calculate the number of days needed to carry out the interviews. For example:

• you need to do 200 interviews,
• your research team of 5 people can do 5 x 4 = 20 interviews per day,
• you will need 200:20 = 10 days for the interviews.

Step 4: Calculate the time needed for the other parts of the study, (for example, 10 days)

Step 5: Determine how much time you can devote to the study. Since the research team usually consists of very busy people, it is unlikely that team members can spend more than 30 working days on the entire study:

• 5 days for preparation (including pre-testing and finalising questionnaires),
• 20 days actual field work,
• 5 days data processing + preliminary analysis.

  If the team has 20 days for fieldwork, as in the example above, it could do the study without extra assistance. However, if the research team has only five days available for the interviews, they would need an additional five research assistants to help complete this part of the study.

If research assistants are required, consider to what extent local health workers can be used. They have the advantage of knowing the local situation. They should never be involved, however, in conducting interviews to evaluate the performance of their own health facility. Local staff from related services (teachers, community development) or students might help out. Sometimes village health workers or community members can collect certain parts of the data.

In WHAT SEQUENCE should data be collected?

In general, it is advisable to start with analysis of data that is already available. This is essential if the sample of respondents is to be selected from the records. Another rule of thumb is that qualitative research techniques (such as key informant interviews, focus group discussions) that are devised to focus the content of questionnaires for interviewing larger groups of informants should be carried out before finalisation of these questionnaires. FGDs designed to provide feedback on issues raised in larger surveys, should, logically, be conducted after preliminary analysis of the questionnaires.

To use time and transport efficiently, data to be drawn from different sources but in one locality should be collected at the same time. (For example, interviews with health staff in a Health Centre, observations of equipment available in the Health Centre and interviews with mothers living around the Health Centre should be scheduled together.)

WHEN should the data be collected?

The type of data to be collected and the demands of the project will determine the actual time needed for the data to be collected. Consideration should be given to:

• availability of research team members and research assistants,
• the appropriate season(s) to conduct the field work (if the problem is season-related or if data collection would be difficult during certain periods),
• accessibility and availability of the sampled population, and
• public holidays and vacation periods.

2. Ensuring quality

It is extremely important that the data we collect are of good quality, that is, reliable and valid. Otherwise we will come up with false or misleading conclusions.

In the previous modules possible sources of data distortion (bias) have been discussed which we should try to prevent. They include:

• Deviations from the sampling procedures set out in the proposal.
• Variability or bias in observations or measurements made because:

  — Our study subject changes his/her behaviour as a consequence of the research. For example, a subject may act more positively while being observed; blood pressure and pulse may increase when the subject is apprehensive.

  — We use unstandardised measuring instruments. For example, we may use unstandardised weighing scales or imprecise or no guidelines for interviewing.

  — Researchers themselves vary in what they observe or measure (observer variability). For example, researchers may be selective in their observations (observer bias); measure, question or note down answers with varying accuracy or follow different approaches (one being more open, friendly, probing than the other).

• Variations in criteria for measurement or for categorising answers because we changed them during the study.

There are a number of possible measures that can be taken to prevent and to partly correct such distortions. Remember: prevention is FAR better than cure!! Cure is usually surgery: you may have to cut out the bad parts of your data or, at best, devise crutches.

Stage 3: DATA HANDLING

Once the data have been collected and checked for completeness and accuracy, a clear procedure should be developed for handling and storing them:

• At some stage questionnaires and other research tools will have to be numbered. Decide if this should be done at the time of the interview or at the time the questionnaires are stored. Usually each tool used will get its own numbers starting from 1. However, if some data sets are linked, e.g., you interview leprosy patients as well as their relatives and neighbours to analyse their interaction and possible stigma from different perspectives, you better link the numbers. For example, if you have interviewed a patient, P9, his neighbour will be N9 and his wife or son/daughter will be R9.

• Identify the person responsible for storing data and the place where it will be stored.

• Decide how data should be stored. Record forms should be kept in the sequence in which they have been numbered.

Annex 12.1: Training research assistants (as well as other research team members with little research experience)

1. Interviewers’ tasks

During the fieldwork interviewers (or research assistants) may work independently or together with one of the researchers. If they go out independently they may have to carry out the following tasks:

• Do the sampling in the field (for example sampling of households within a village and/or sampling of individuals to be interviewed within households).
• Give a clear introduction to the interviewee concerning the purpose and procedures of the interview and ask for permission to interview. (An introductory sheet should be prepared and attached to interview guide or questionnaire, so that each research team member approaches the informants in a similar way.)
• Perform the interviews. Usually it is best to give research assistants more or less structured questionnaires to administer with clear instructions for open questions. It is not wise to assign the more difficult tasks of performing highly flexible interviews or focus group discussions to research assistants unless they are mature, experienced persons.

It is imperative that the researchers train their research assistants so they can carry out their tasks accurately and correctly, according to the procedures developed by the researchers. Research assistants should not be left to develop their own procedures; otherwise bias will be almost certain to result.

The training of the research team may take 2 to 3 days. The first day may be devoted to theory, followed by 1 or 2 days of practical training, depending on the local circumstances and the nature of the study.

2. Theoretical training

Research assistants must be thoroughly familiar with the objectives of the research project and the methodology. Therefore, it is recommended that they be provided with copies of the research protocol and that the most relevant sections be discussed thoroughly, including:

• statement of the problem,
• objectives,
• data collection tools to be used (an overview),
• sampling procedures (if sampling has to be done in the field),
• plan for data collection, and
• plan for data analysis.

It is important at this stage that the research assistants get an ample opportunity to ask questions.

Then a more in-depth discussion should follow concerning the data collection tools (interview FGD guides, questionnaires and possibly checklists) that are to be used by the research-team members. For each and every question all should know WHY the information is required.

Research assistants should be taught basic interview techniques, such as

• asking questions in a neutral manner;
• not showing by words or expression what answers one expects;
• not showing agreement, disagreement or surprise; and
• recording answers to open questions precisely as they are provided, without sifting or interpreting them.

Clear instructions should be given as well, concerning to what extent an interviewer is allowed to alter the phrasing of questions, if it seems necessary, and whether he/she should probe for answers. For questions which have pre-categorised answers it should be made clear whether the possible answers should be mentioned by the interviewer during the interview or not. (Usually they are not to be mentioned.) There should be no misunderstandings concerning how to record answers and observations.

Finally, explanations should be given concerning how an interviewer should introduce him or herself to the interviewee, what to say concerning the purpose of the study, how to ask for consent, and how to close the interview.

3. Practical training

Practical training in interviewing is essential. This may be provided in two stages. (See Module 10B part V, interview exercise.)

Firstly, role-plays can be performed, during which one trainee assumes the role of the interviewer and another plays the interviewee. Other trainees and the trainers (researchers) should observe carefully what happens and give constructive feedback right after the role-play. Then roles can be changed, until each trainee has had a chance to practice each type of interview at least once.

Secondly, a field test should be conducted, which will serve two purposes: the training of the entire research team including research assistants, and a (further) test of the data collection tools. A test of the tools is essential if a previous field-test resulted in important changes and/or if the questionnaire(s) were translated into a local language after the first field test. If the research assistants will be involved in the proper phrasing of questions this will definitely strengthen their interest and commitment.

The field test is best carried out in groups of 2 to 3 persons (see Module 14), with each team including at least one trainer and one trainee. Note that after the field test constructive critique of each interview should be made, starting from the moment the interviewee was approached up to the farewell.

4. Supervision of research assistants

Even if research assistants are used, responsibility for the research remains with the research team as a whole. To guarantee the quality of the data collected it is important to supervise the research assistants’ performance, especially at the beginning of the data collection period. If they are going out into the field independently, plans could be made to accompany them on selected visits and/or to question a small sample of the interviewees as a ‘control group’ concerning key aspects of the interview.

As a further quality control check it is important that an interviewer’s name (or interviewer’s code) appears standard on each questionnaire/checklist so that it is possible to ask for clarification if certain information is not clear.

Trainer’s Notes

Module 12: PLAN FOR DATA COLLECTION

Introduction and discussion

• Explain that we are entering a new phase in the development of the methodology, in which we go from theory to practice: the concrete planning of the data collection process.
• Of the three phases that can be distinguished in the planning for data collection (i.e., permission to proceed, actual data collection and data handling), planning for actual data collection will require most attention, both in the presentation and in the group work.
• The logistics of data collection is one major aspect of the planning: WHO will collect WHAT and WHEN. It is important that relatively inexperienced researchers give thought to this aspect of data collection before the pre-test, so that they can consciously check whether their plans are realistic.
• As the participants begin planning for data collection they should clearly distinguish and list the different components of their study. Next, they should consider the time it will take to carry out the different components, so that they can decide whether they can do the study alone or whether they need assistance.
• Take the most labour-intensive part of one of the studies being developed in the workshop and follow the steps outlined in the module to determine the time needed.
• If you do not want to favour one of the groups, you can use the example of the utilisation of delivery care provided in the module or any other example that resembles the topics participants are working on.
• Discuss the advantages and disadvantages of using research assistants and of using local health service personnel as research assistants.
• Then proceed to the second major aspect of the planning for data collection, that of ensuring quality.
• Stress the importance of preparing a fieldwork manual (irrespective of whether research assistants are recruited or not) and of using appropriate interview techniques. Refer to Annex 12.1 for more detailed guidelines on how to train interviewers.

Group work

• Make sure that participants have understood the principle of calculating the time required for each component of the study and that they calculate carefully the time needed for the most labour-intensive parts of the study, so that they can decide:

  — whether they need assistance, and

  — who would be appropriate research assistants, if required.

• Let them summarise ethical issues involved in data collection, which have been discussed during previous presentations. Participants need to include a section on ethical considerations in their research proposal (see Module 18 or Module 1 for content of the research proposal).

Document(s) 18 of 27