Document(s) 14 of 27

1. Overview of data collection techniques
2. The importance of combining different data collection techniques
3. Bias in information collection
4. Ethical considerations

I. OVERVIEW OF DATA COLLECTION TECHNIQUES

Data-collection techniques allow us to systematically collect information about our objects of study (people, objects, phenomena) and about the settings in which they occur.

In the collection of data we have to be systematic. If data are collected haphazardly, it will be difficult to answer our research questions in a conclusive way.

Example:

During a nutrition survey three different weighing scales were used in three villages. The researchers did not record which scales were used in which village. After completion of the survey it was discovered that the scales were not standardised and indicated different weights when weighing the same child. It was therefore impossible to conclude in which village malnutrition was most prevalent.

Various data collection techniques can be used such as:

• Using available information
• Observing
• Interviewing (face-to-face)
• Administering written questionnaires
• Focus group discussions
• Projective techniques, mapping, scaling

1. Using available information

Usually there is a large amount of data that has already been collected by others, although it may not necessarily have been analysed or published. Locating these sources and retrieving the information is a good starting point in any data collection effort.

For example, analysis of the information routinely collected by health facilities can be very useful for identifying problems in certain interventions or in flows of drug supply, or for identifying increases in the incidence of certain diseases.

Analysis of health information system data, census data, unpublished reports and publications in archives and libraries or in offices at the various levels of health and health-related services, may be a study in itself. Usually, however, it forms part of a study in which other data collection techniques are also used.

The use of key informants is another important technique to gain access to available information. Key informants could be knowledgeable community leaders or health staff at various levels and one or two informative members of the target group (e.g., adolescents on their sexual behaviour). They can be involved in various stages of the research, from the statement of the problem to analysis of the data and development of recommendations. Other sources of available data are newspapers and published case histories, e.g., patients suffering from serious diseases, or their relatives, telling their experiences and how they cope.*

* For example, Noerine Kaleeba (1991) We miss you all: AIDS in the family. Harare: Women and AIDS Support Network.

The advantage of using existing data is that collection is inexpensive. However, it is sometimes difficult to gain access to the records or reports required, and the data may not always be complete and precise enough, or too disorganised.

2. Observing

Observation of human behaviour is a much-used data collection technique. It can be undertaken in different ways:

• Participant observation: The observer takes part in the situation he or she observes.

  (For example, a doctor hospitalised with a broken hip, who now observes hospital procedures ‘from within’.)

• Non-participant observation: The observer watches the situation, openly or concealed, but does not participate.

Observations can be open (e.g., ‘shadowing’ a health worker with his/her permission during routine activities) or concealed (e.g., ‘mystery clients’ trying to obtain antibiotics without medical prescription). They may serve different purposes. Observations can give additional, more accurate information on behaviour of people than interviews or questionnaires. They can also check on the information collected through interviews especially on sensitive topics such as alcohol or drug use, or stigmatising diseases. For example, whether community members share drinks or food with patients suffering from feared diseases (leprosy, TB, AIDS) are essential observations in a study on stigma.

Observations of human behaviour can form part of any type of study, but as they are time consuming they are most often used in small-scale studies.

Observations can also be made on objects. For example, the presence or absence of a latrine and its state of cleanliness may be observed. Here observation would be the major research technique.

If observations are made using a defined scale they may be called measurements. Measurements usually require additional tools. For example, in nutritional surveillance we measure weight and height by using weighing scales and a measuring board. We use thermometers for measuring body temperature.

3. Interviewing

Answers to the questions posed during an interview can be recorded by writing them down (either during the interview itself or immediately after the interview) or by tape-recording the responses, or by a combination of both.

Interviews can be conducted with varying degrees of flexibility. The two extremes, high and low degree of flexibility, are described below:

• High degree of flexibility:

For example:

When studying sensitive issues such as teenage pregnancy and abortions, the investigator may use a list of topics rather than fixed questions. These may, e.g., include how teenagers started sexual intercourse, the responsibility girls and their partners take to prevent pregnancy (if at all), and the actions they take in the event of unwanted pregnancies. The investigator should have an additional list of topics ready when the respondent falls silent, (e.g., when asked about abortion methods used, who made the decision and who paid). The sequence of topics should be determined by the flow of discussion. It is often possible to come back to a topic discussed earlier in a later stage of the interview.

The unstructured or loosely structured method of asking questions can be used for interviewing individuals as well as groups of key informants. (For details concerning focus group discussions (FGDs), see Module 10C.)

A flexible method of interviewing is useful if a researcher has as yet little understanding of the problem or situation he is investigating, or if the topic is sensitive. It is frequently applied in exploratory studies. The instrument used may be called an interview guide or interview schedule.*

• Low degree of flexibility:

Less flexible methods of interviewing are useful when the researcher is relatively knowledgeable about expected answers or when the number of respondents being interviewed is relatively large. Then questionnaires may be used with a fixed list of questions in a standard sequence, which have mainly fixed or pre-categorised answers.

For example:

After a number of observations on the (hygienic) behaviour of women drawing water at a well and some key informant interviews on the use and maintenance of the wells, one may conduct a larger survey on water use and satisfaction with the quantity and quality of the water.

* Though in principle one may speak of loosely structured questionnaires, in practice the term questionnaire appears to be so hooked to tools with pre-categorised answers that we have decided to use the term interview guide for loosely structured tools. However, in reality there is often a mixture of open and pre-categorised answers (see Module 10B). In that case we will still use the term questionnaire.

4. Administering written questionnaires

A written questionnaire can be administered in different ways, such as by:

• Sending questionnaires by mail with clear instructions on how to answer the questions and asking for mailed responses;
• Gathering all or part of the respondents in one place at one time, giving oral or written instructions, and letting the respondents fill out the questionnaires; or
• Hand-delivering questionnaires to respondents and collecting them later.

The questions can be either open-ended or closed (with pre-categorised answers). (See Module 10B for details concerning design of interview guides and questionnaires.)

5. Focus group discussions (FGD)

A focus group discussion allows a group of 8 - 12 informants to freely discuss a certain subject with the guidance of a facilitator or reporter. (See Module 10C for a discussion of this technique.)

6. Projective techniques

When a researcher uses projective techniques, (s)he asks an informant to react to some kind of visual or verbal stimulus.

For example: An informant may be provided with a rough outline of the body and be asked to draw her or his perception of the conception or onset of an illness.

Another example of a projective technique is the presentation of a hypothetical question or an incomplete sentence or case/study to an informant (‘story with a gap’). A researcher may ask the informant to complete in writing sentences such as:

— If I were to discover that my neighbour had TB, I would . . .;

— If my wife were to propose that I use condoms, I would . . .

Or (s)he may ask the informant: Suppose your child suffered from diarrhoea, what would you do?

Such techniques can easily be combined with semi-structured interviews or written questionnaires. They are also very useful in FGDs to get people’s opinion on sensitive issues.

7. Mapping and scaling

Mapping is a valuable technique for visually displaying relationships and resources.

In a water supply project, for example, mapping is invaluable. It can be used to present the placement of wells, distance of the homes from the wells, other water systems, etc. It gives researchers a good overview of the physical situation and may help to highlight relationships hitherto unrecognised.

Mapping a community is also very useful and often indispensable as a pre-stage to sampling.

Scaling is a technique that allows researchers through their respondents to categorise certain variables that they would not be able to rank themselves.

For example, they may ask their informant(s) to bring certain types of herbal medicine and ask them to arrange these into piles according to their usefulness. The informants would then be asked to explain the logic of their ranking.

Mapping and scaling may be used as participatory techniques in rapid appraisals or situation analyses. In a separate volume on participatory action research, more such techniques will be presented. (Also see the literature list at end of this module.)

Rapid appraisal techniques and participatory research are approaches often used in health systems research.

Differentiation between data collection techniques and data collection tools

To avoid confusion in the use of terms, the following table points out the distinction between techniques and tools applied in data collection.

Table 10A.1: Data collection techniques and tools

Advantages and disadvantages of various data collection techniques

Table 10A.2 summarises the advantages and disadvantages of various data collection techniques.

Table 10A.2: Advantages and disadvantages of various data collection techniques

II. IMPORTANCE OF COMBINING DIFFERENT DATA COLLECTION TECHNIQUES

When discussing different data collection techniques and their advantages and disadvantages, it becomes clear that they can complement each other. A skilful use of a combination of different techniques can reduce the chance of bias (see below) and will give a more comprehensive understanding of the topic under study.

Researchers often use a combination of flexible and less flexible research techniques.

Flexible techniques, such as

• loosely structured interviews using open-ended questions,
• focus group discussions, and
• participant observation

are also called QUALITATIVE research techniques. They produce qualitative data that is often recorded in narrative form.

Structured questionnaires that enable the researcher to quantify pre- or post-categorised answers to questions are an example of QUANTITATIVE research techniques. The answers to questions can be counted and expressed numerically.

Both qualitative and quantitative research techniques are often used within a single study.

For example:

It has been observed in country X that children between 1 and 2-1/2 years, who have already started to eat independently, have unsatisfactory food intake once they fall ill. A study could be designed to address this problem, containing the following stages:

• Focus group discussions (FGDs) with 2 to 5 groups of mothers or in-depth interviews with 10 - 20 mothers, to find out whether they change the feeding practices for children in this age group when they suffer from (various) illnesses and how mothers deal with children who have no appetite when they are sick (exploratory study);
• A cross-sectional survey, testing the relevant findings of the exploratory study on a larger scale; and
• FGDs with women in the study area to discuss findings and possible questions arising from the survey and to develop possible solutions for problems detected.

In this example, the first, qualitative part of the study would be used to focus the survey on the most relevant issues (mothers’ feeding behaviours and reasons for these behaviours) and to help phrase the questions in an optimal way in order to obtain the information that is needed.

The second, quantitative part of the study would be used to find out what proportion of the mothers follow various practices and the reasons for their behaviours and whether certain categories of children (e.g., the younger ones or children from specific socio-economic categories) are more at risk than others.

The third, qualitative part of the study would provide feedback on the major findings of the survey. Do the conclusions make sense to women in the study area? Have certain aspects been overlooked when interpreting the data? What remedial action is feasible to improve practices related to feeding sick children?

It is also common to collect qualitative and quantitative data in a single questionnaire. Researchers collecting both types of data have to take care that they:

• do not include too many open-ended questions in large-scale surveys, making data analysis more complicated; and
• do not use inappropriate statistical tests on quantitative data generated by small-scale studies.

III. BIAS IN INFORMATION COLLECTION

Possible sources of bias during data collection:

1. Defective instruments, such as:

• Questionnaires with:

  — fixed or closed questions on topics about which little is known (often asking the ‘wrong things’);

  — open-ended questions without guidelines on how to ask (or to answer) them;

  — vaguely phrased questions;

  — ‘leading questions’ that cause the respondent to believe one answer would be preferred over another; or

  — questions placed in an illogical order.

• Weighing scales or other measuring equipment that are not standardised (see section 1).

These sources of bias can be prevented by carefully planning the data collection process and by pre-testing the data collection tools.

2. Observer bias:

Observer bias can easily occur when conducting observations or utilising loosely structured group- or individual interviews. There is a risk that the data collector will only see or hear things in which (s)he is interested or will miss information that is critical to the research.

Observation protocols and guidelines for conducting loosely structured interviews should be prepared, and training and practice should be provided to data collectors in using both these tools. Moreover it is highly recommended that data collectors work in pairs when using flexible research techniques and discuss and interpret the data immediately after collecting it. Another possibility - commonly used by anthropologists - is using a tape recorder and transcribing the tape word by word.

3. Effect of the interview on the informant:

This is a possible factor in all interview situations. The informant may mistrust the intention of the interview and dodge certain questions or give misleading answers. For example: in a survey on alcoholism you ask school children: ‘Does your father sometimes get drunk?’ Many will probably deny that he does, even if it is true. Such bias can be reduced by adequately introducing the purpose of the study to informants, by phrasing questions on sensitive issues in a positive way, by taking sufficient time for the interview, and by assuring informants that the data collected will be confidential (see Module 10B).

It is also important to be careful in the selection of interviewers. In a study soliciting the reasons for the low utilisation of local health services, for example, one should not ask health workers from the health centres concerned to interview the population. Their use as interviewers would certainly influence the results of the study.

4. Information bias:

Sometimes the information itself has weaknesses. Medical records may have many blanks or be unreadable. This tells something about the quality of the data and has to be recorded. For example, in a TB defaulter study the percentage of defaulters with an incomplete or missing address should be calculated.

Another common information bias is due to gaps in people’s memory; this is called memory or recall bias. A mother may not remember all details of her child’s last diarrhoea episode and of the treatment she gave two or three months afterwards. For such common diseases it is advisable to limit the period of recall, asking, for example, ‘Has your child had diarrhoea over the past two weeks?’

Module 10B and Module 12, Annex 1 provide more information on how you can prevent biases in your research.

IV. ETHICAL CONSIDERATIONS

As we develop our data collection techniques, we need to consider whether our research procedures are likely to cause any physical or emotional harm. Harm may be caused, for example, by:

• violating informants’ right to privacy by posing sensitive questions or by gaining access to records which may contain personal data;
• observing the behaviour of informants without their being aware (concealed observation should therefore always be crosschecked or discussed with other researchers with respect to ethical admissibility);
• allowing personal information to be made public which informants would want to be kept private, and
• failing to observe/respect certain cultural values, traditions or taboos valued by your informants.

Several methods for dealing with these issues may be recommended:

• obtaining informed consent before the study or the interview begins;
• not exploring sensitive issues before a good relationship has been established with the informant;
• ensuring the confidentiality of the data obtained; and
• learning enough about the culture of informants to ensure it is respected during the data collection process.

If sensitive questions are asked, for example, about family planning or sexual practices, or about opinions of patients on the health services provided, it may be advisable to omit names and addresses from the questionnaires.

REFERENCES

Abramson JH (1990, 4^th ed.) Survey Methods in Community Medicine. London: Churchill- Livingstone. (In particular Chapter 6)

Moser CA, Kalton G (1989, 2^nd ed.) Survey Methods in Social Investigation. Hants, UK: Gower Publishing Company. (In particular Chapters 10-13)

Patton MQ (1990, 2^nd ed.) Qualitative Evaluation and Research Methods. Newbury Park, USA: Sage Publications. (In particular Chapters 17-29)

Pretty JN, Guyt I, Thompson J, Scones I (1995) Participatory Learning & Action. A Trainer’s Guide. London: International Institute for environment and Development (IIED) (In particular Chapters 4 and 5 on semi-structured interviewing, diagramming, ranking and scoring.)

Trainer’s Notes

Module 10A: OVERVIEW OF DATA COLLECTION TECHNIQUES

Introduction and discussion

• Present an overview of the various data collection techniques. Give examples from the participants’ fields of interest.
• If it would be useful for the participants’ research projects, you can introduce additional research techniques using the material in Module 10C (Focus group discussions) or other sources, e.g., the volume on Participatory Action Research to be published by AFRO in the HSR series or Pretty et al. (1995) Participatory Learning and Action.
• Explain the difference between data collection techniques and data collection tools.
• Let the participants mention possible advantages and disadvantages of the various data collection techniques.
• Explain at what times qualitative research techniques are most useful and when quantitative techniques are more appropriate. Make sure that participants understand the advantages of combining quantitative and qualitative research techniques, preferably giving examples from one or more of the projects they are developing.
• Identify different possibilities for bias, using examples from the participants’ own studies.
• Let the groups come up with examples of ethical issues that might play a role in their studies.

Exercise: Selection of study types and data collection techniques

• This exercise is designed to give participants some experience in choosing types of studies appropriate for typical situations before they have to select types of studies for their own proposals.
• Stress that objectives, if well formulated, should help to determine the appropriate study type(s).
• Ask participants to divide into sub-groups of 4-5 persons to do the exercise. Each sub-group should be assigned 2 topics. Allow 30 minutes for the groups to complete their work. Let them write the answers on an overhead transparency or flipchart.
• In the plenary (30 minutes) ask each group to answer the questions posed for one topic. Let other groups who discussed the same topic comment and add their own suggestions.

(An answer sheet for the exercise is presented in Annex 10A.1, on the next page.)

Group work: Selection of study type and data collection techniques

• This group work session is an important one, as it combines both the selection of study type and choice of data collection techniques. Work closely with your group, helping them to go step by step through the process described for the group work.
• Make sure they understand why it is important to refer back to their variables and the table they developed during Module 8, as they choose the data collection techniques they need.

Annex 10A.1: Answer sheet to exercise on selection of study types and data collection techniques

* See Grosskurth H, Mosha F, Todd J, Mwijarubi E, Klokke A, Senkoro K, Mayaud P et al (1995) Impact of improved treatment of sexually transmitted diseases on HIV infection in rural Tanzania: randomised controlled trial. The Lancet 346: 530-536.

This delay of two years would form an ethical problem as soon as the preliminary results would indicate that the extra input in STD treatment in the intervention HCs indeed reduces the risk of HIV infection. If that is the case, experiments are sometimes stopped in order to allow the control group to profit from the intervention as well.

One important bias could be that STD patients in control villages would have heard about the available treatment for STD in the experimental HCs and go there for treatment. To be able to control this potential bias, the research team analysed the treatment registers in the experimental HCs for the location of all STD patients who received treatment and deducted patients from the control villages (which appeared to be exceptions).

Another bias may be caused by intensified health education on STD symptoms and treatment over radio and TV, which could influence the treatment seeking behaviour of members of the experimental as well as of the control group.

Document(s) 14 of 27