validity and reliability in qualitative research h.i.l. brink (conference paper)

• Technical Support
• Find My Rep

You are here

Reliability and Validity in Qualitative Research

• Jerome Kirk - Politics and Society, University of California, Irvine
• Marc L. Miller - Anthropology, University of Washington, USA
• Description

See what’s new to this edition by selecting the Features tab on this page. Should you need additional information or have questions regarding the HEOA information provided for this title, including what is new to this edition, please email [email protected] . Please include your name, contact information, and the name of the title for which you would like more information. For information on the HEOA, please go to http://ed.gov/policy/highered/leg/hea08/index.html .

For assistance with your order: Please email us at [email protected] or connect with your SAGE representative.

SAGE 2455 Teller Road Thousand Oaks, CA 91320 www.sagepub.com

Some contributions to issues and ever present debates on the reliability and validity of qualitative research approaches . However some concerns on the currency of the arguments given the references.

On reliability and validity in qualitative research

• PMID: 3649124
• DOI: 10.1177/019394598700900201

Publication types

• Longitudinal Studies
• Research / standards*
• Research Design

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 18, Issue 2
• Issues of validity and reliability in qualitative research
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Helen Noble 1 ,
• Joanna Smith 2
• 1 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• 2 School of Human and Health Sciences, University of Huddersfield , Huddersfield , UK
• Correspondence to Dr Helen Noble School of Nursing and Midwifery, Queens's University Belfast, Medical Biology Centre, 97 Lisburn Rd, Belfast BT9 7BL, UK; helen.noble{at}qub.ac.uk

https://doi.org/10.1136/eb-2015-102054

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Evaluating the quality of research is essential if findings are to be utilised in practice and incorporated into care delivery. In a previous article we explored ‘bias’ across research designs and outlined strategies to minimise bias. 1 The aim of this article is to further outline rigour, or the integrity in which a study is conducted, and ensure the credibility of findings in relation to qualitative research. Concepts such as reliability, validity and generalisability typically associated with quantitative research and alternative terminology will be compared in relation to their application to qualitative research. In addition, some of the strategies adopted by qualitative researchers to enhance the credibility of their research are outlined.

Are the terms reliability and validity relevant to ensuring credibility in qualitative research?

Although the tests and measures used to establish the validity and reliability of quantitative research cannot be applied to qualitative research, there are ongoing debates about whether terms such as validity, reliability and generalisability are appropriate to evaluate qualitative research. 2–4 In the broadest context these terms are applicable, with validity referring to the integrity and application of the methods undertaken and the precision in which the findings accurately reflect the data, while reliability describes consistency within the employed analytical procedures. 4 However, if qualitative methods are inherently different from quantitative methods in terms of philosophical positions and purpose, then alterative frameworks for establishing rigour are appropriate. 3 Lincoln and Guba 5 offer alternative criteria for demonstrating rigour within qualitative research namely truth value, consistency and neutrality and applicability. Table 1 outlines the differences in terminology and criteria used to evaluate qualitative research.

• View inline

Terminology and criteria used to evaluate the credibility of research findings

What strategies can qualitative researchers adopt to ensure the credibility of the study findings?

Unlike quantitative researchers, who apply statistical methods for establishing validity and reliability of research findings, qualitative researchers aim to design and incorporate methodological strategies to ensure the ‘trustworthiness’ of the findings. Such strategies include:

Accounting for personal biases which may have influenced findings; 6

Acknowledging biases in sampling and ongoing critical reflection of methods to ensure sufficient depth and relevance of data collection and analysis; 3

Meticulous record keeping, demonstrating a clear decision trail and ensuring interpretations of data are consistent and transparent; 3 , 4

Establishing a comparison case/seeking out similarities and differences across accounts to ensure different perspectives are represented; 6 , 7

Including rich and thick verbatim descriptions of participants’ accounts to support findings; 7

Demonstrating clarity in terms of thought processes during data analysis and subsequent interpretations 3 ;

Engaging with other researchers to reduce research bias; 3

Respondent validation: includes inviting participants to comment on the interview transcript and whether the final themes and concepts created adequately reflect the phenomena being investigated; 4

Data triangulation, 3 , 4 whereby different methods and perspectives help produce a more comprehensive set of findings. 8 , 9

Table 2 provides some specific examples of how some of these strategies were utilised to ensure rigour in a study that explored the impact of being a family carer to patients with stage 5 chronic kidney disease managed without dialysis. 10

Strategies for enhancing the credibility of qualitative research

In summary, it is imperative that all qualitative researchers incorporate strategies to enhance the credibility of a study during research design and implementation. Although there is no universally accepted terminology and criteria used to evaluate qualitative research, we have briefly outlined some of the strategies that can enhance the credibility of study findings.

• Sandelowski M
• Lincoln YS ,
• Barrett M ,
• Mayan M , et al
• Greenhalgh T
• Lingard L ,

Twitter Follow Joanna Smith at @josmith175 and Helen Noble at @helnoble

Competing interests None.

Read the full text or download the PDF:

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Family Med Prim Care
• v.4(3); Jul-Sep 2015

Validity, reliability, and generalizability in qualitative research

Lawrence leung.

1 Department of Family Medicine, Queen's University, Kingston, Ontario, Canada

2 Centre of Studies in Primary Care, Queen's University, Kingston, Ontario, Canada

In general practice, qualitative research contributes as significantly as quantitative research, in particular regarding psycho-social aspects of patient-care, health services provision, policy setting, and health administrations. In contrast to quantitative research, qualitative research as a whole has been constantly critiqued, if not disparaged, by the lack of consensus for assessing its quality and robustness. This article illustrates with five published studies how qualitative research can impact and reshape the discipline of primary care, spiraling out from clinic-based health screening to community-based disease monitoring, evaluation of out-of-hours triage services to provincial psychiatric care pathways model and finally, national legislation of core measures for children's healthcare insurance. Fundamental concepts of validity, reliability, and generalizability as applicable to qualitative research are then addressed with an update on the current views and controversies.

Nature of Qualitative Research versus Quantitative Research

The essence of qualitative research is to make sense of and recognize patterns among words in order to build up a meaningful picture without compromising its richness and dimensionality. Like quantitative research, the qualitative research aims to seek answers for questions of “how, where, when who and why” with a perspective to build a theory or refute an existing theory. Unlike quantitative research which deals primarily with numerical data and their statistical interpretations under a reductionist, logical and strictly objective paradigm, qualitative research handles nonnumerical information and their phenomenological interpretation, which inextricably tie in with human senses and subjectivity. While human emotions and perspectives from both subjects and researchers are considered undesirable biases confounding results in quantitative research, the same elements are considered essential and inevitable, if not treasurable, in qualitative research as they invariable add extra dimensions and colors to enrich the corpus of findings. However, the issue of subjectivity and contextual ramifications has fueled incessant controversies regarding yardsticks for quality and trustworthiness of qualitative research results for healthcare.

Impact of Qualitative Research upon Primary Care

In many ways, qualitative research contributes significantly, if not more so than quantitative research, to the field of primary care at various levels. Five qualitative studies are chosen to illustrate how various methodologies of qualitative research helped in advancing primary healthcare, from novel monitoring of chronic obstructive pulmonary disease (COPD) via mobile-health technology,[ 1 ] informed decision for colorectal cancer screening,[ 2 ] triaging out-of-hours GP services,[ 3 ] evaluating care pathways for community psychiatry[ 4 ] and finally prioritization of healthcare initiatives for legislation purposes at national levels.[ 5 ] With the recent advances of information technology and mobile connecting device, self-monitoring and management of chronic diseases via tele-health technology may seem beneficial to both the patient and healthcare provider. Recruiting COPD patients who were given tele-health devices that monitored lung functions, Williams et al. [ 1 ] conducted phone interviews and analyzed their transcripts via a grounded theory approach, identified themes which enabled them to conclude that such mobile-health setup and application helped to engage patients with better adherence to treatment and overall improvement in mood. Such positive findings were in contrast to previous studies, which opined that elderly patients were often challenged by operating computer tablets,[ 6 ] or, conversing with the tele-health software.[ 7 ] To explore the content of recommendations for colorectal cancer screening given out by family physicians, Wackerbarth, et al. [ 2 ] conducted semi-structure interviews with subsequent content analysis and found that most physicians delivered information to enrich patient knowledge with little regard to patients’ true understanding, ideas, and preferences in the matter. These findings suggested room for improvement for family physicians to better engage their patients in recommending preventative care. Faced with various models of out-of-hours triage services for GP consultations, Egbunike et al. [ 3 ] conducted thematic analysis on semi-structured telephone interviews with patients and doctors in various urban, rural and mixed settings. They found that the efficiency of triage services remained a prime concern from both users and providers, among issues of access to doctors and unfulfilled/mismatched expectations from users, which could arouse dissatisfaction and legal implications. In UK, a care pathways model for community psychiatry had been introduced but its benefits were unclear. Khandaker et al. [ 4 ] hence conducted a qualitative study using semi-structure interviews with medical staff and other stakeholders; adopting a grounded-theory approach, major themes emerged which included improved equality of access, more focused logistics, increased work throughput and better accountability for community psychiatry provided under the care pathway model. Finally, at the US national level, Mangione-Smith et al. [ 5 ] employed a modified Delphi method to gather consensus from a panel of nominators which were recognized experts and stakeholders in their disciplines, and identified a core set of quality measures for children's healthcare under the Medicaid and Children's Health Insurance Program. These core measures were made transparent for public opinion and later passed on for full legislation, hence illustrating the impact of qualitative research upon social welfare and policy improvement.

Overall Criteria for Quality in Qualitative Research

Given the diverse genera and forms of qualitative research, there is no consensus for assessing any piece of qualitative research work. Various approaches have been suggested, the two leading schools of thoughts being the school of Dixon-Woods et al. [ 8 ] which emphasizes on methodology, and that of Lincoln et al. [ 9 ] which stresses the rigor of interpretation of results. By identifying commonalities of qualitative research, Dixon-Woods produced a checklist of questions for assessing clarity and appropriateness of the research question; the description and appropriateness for sampling, data collection and data analysis; levels of support and evidence for claims; coherence between data, interpretation and conclusions, and finally level of contribution of the paper. These criteria foster the 10 questions for the Critical Appraisal Skills Program checklist for qualitative studies.[ 10 ] However, these methodology-weighted criteria may not do justice to qualitative studies that differ in epistemological and philosophical paradigms,[ 11 , 12 ] one classic example will be positivistic versus interpretivistic.[ 13 ] Equally, without a robust methodological layout, rigorous interpretation of results advocated by Lincoln et al. [ 9 ] will not be good either. Meyrick[ 14 ] argued from a different angle and proposed fulfillment of the dual core criteria of “transparency” and “systematicity” for good quality qualitative research. In brief, every step of the research logistics (from theory formation, design of study, sampling, data acquisition and analysis to results and conclusions) has to be validated if it is transparent or systematic enough. In this manner, both the research process and results can be assured of high rigor and robustness.[ 14 ] Finally, Kitto et al. [ 15 ] epitomized six criteria for assessing overall quality of qualitative research: (i) Clarification and justification, (ii) procedural rigor, (iii) sample representativeness, (iv) interpretative rigor, (v) reflexive and evaluative rigor and (vi) transferability/generalizability, which also double as evaluative landmarks for manuscript review to the Medical Journal of Australia. Same for quantitative research, quality for qualitative research can be assessed in terms of validity, reliability, and generalizability.

Validity in qualitative research means “appropriateness” of the tools, processes, and data. Whether the research question is valid for the desired outcome, the choice of methodology is appropriate for answering the research question, the design is valid for the methodology, the sampling and data analysis is appropriate, and finally the results and conclusions are valid for the sample and context. In assessing validity of qualitative research, the challenge can start from the ontology and epistemology of the issue being studied, e.g. the concept of “individual” is seen differently between humanistic and positive psychologists due to differing philosophical perspectives:[ 16 ] Where humanistic psychologists believe “individual” is a product of existential awareness and social interaction, positive psychologists think the “individual” exists side-by-side with formation of any human being. Set off in different pathways, qualitative research regarding the individual's wellbeing will be concluded with varying validity. Choice of methodology must enable detection of findings/phenomena in the appropriate context for it to be valid, with due regard to culturally and contextually variable. For sampling, procedures and methods must be appropriate for the research paradigm and be distinctive between systematic,[ 17 ] purposeful[ 18 ] or theoretical (adaptive) sampling[ 19 , 20 ] where the systematic sampling has no a priori theory, purposeful sampling often has a certain aim or framework and theoretical sampling is molded by the ongoing process of data collection and theory in evolution. For data extraction and analysis, several methods were adopted to enhance validity, including 1 st tier triangulation (of researchers) and 2 nd tier triangulation (of resources and theories),[ 17 , 21 ] well-documented audit trail of materials and processes,[ 22 , 23 , 24 ] multidimensional analysis as concept- or case-orientated[ 25 , 26 ] and respondent verification.[ 21 , 27 ]

Reliability

In quantitative research, reliability refers to exact replicability of the processes and the results. In qualitative research with diverse paradigms, such definition of reliability is challenging and epistemologically counter-intuitive. Hence, the essence of reliability for qualitative research lies with consistency.[ 24 , 28 ] A margin of variability for results is tolerated in qualitative research provided the methodology and epistemological logistics consistently yield data that are ontologically similar but may differ in richness and ambience within similar dimensions. Silverman[ 29 ] proposed five approaches in enhancing the reliability of process and results: Refutational analysis, constant data comparison, comprehensive data use, inclusive of the deviant case and use of tables. As data were extracted from the original sources, researchers must verify their accuracy in terms of form and context with constant comparison,[ 27 ] either alone or with peers (a form of triangulation).[ 30 ] The scope and analysis of data included should be as comprehensive and inclusive with reference to quantitative aspects if possible.[ 30 ] Adopting the Popperian dictum of falsifiability as essence of truth and science, attempted to refute the qualitative data and analytes should be performed to assess reliability.[ 31 ]

Generalizability

Most qualitative research studies, if not all, are meant to study a specific issue or phenomenon in a certain population or ethnic group, of a focused locality in a particular context, hence generalizability of qualitative research findings is usually not an expected attribute. However, with rising trend of knowledge synthesis from qualitative research via meta-synthesis, meta-narrative or meta-ethnography, evaluation of generalizability becomes pertinent. A pragmatic approach to assessing generalizability for qualitative studies is to adopt same criteria for validity: That is, use of systematic sampling, triangulation and constant comparison, proper audit and documentation, and multi-dimensional theory.[ 17 ] However, some researchers espouse the approach of analytical generalization[ 32 ] where one judges the extent to which the findings in one study can be generalized to another under similar theoretical, and the proximal similarity model, where generalizability of one study to another is judged by similarities between the time, place, people and other social contexts.[ 33 ] Thus said, Zimmer[ 34 ] questioned the suitability of meta-synthesis in view of the basic tenets of grounded theory,[ 35 ] phenomenology[ 36 ] and ethnography.[ 37 ] He concluded that any valid meta-synthesis must retain the other two goals of theory development and higher-level abstraction while in search of generalizability, and must be executed as a third level interpretation using Gadamer's concepts of the hermeneutic circle,[ 38 , 39 ] dialogic process[ 38 ] and fusion of horizons.[ 39 ] Finally, Toye et al. [ 40 ] reported the practicality of using “conceptual clarity” and “interpretative rigor” as intuitive criteria for assessing quality in meta-ethnography, which somehow echoed Rolfe's controversial aesthetic theory of research reports.[ 41 ]

Food for Thought

Despite various measures to enhance or ensure quality of qualitative studies, some researchers opined from a purist ontological and epistemological angle that qualitative research is not a unified, but ipso facto diverse field,[ 8 ] hence any attempt to synthesize or appraise different studies under one system is impossible and conceptually wrong. Barbour argued from a philosophical angle that these special measures or “technical fixes” (like purposive sampling, multiple-coding, triangulation, and respondent validation) can never confer the rigor as conceived.[ 11 ] In extremis, Rolfe et al. opined from the field of nursing research, that any set of formal criteria used to judge the quality of qualitative research are futile and without validity, and suggested that any qualitative report should be judged by the form it is written (aesthetic) and not by the contents (epistemic).[ 41 ] Rolfe's novel view is rebutted by Porter,[ 42 ] who argued via logical premises that two of Rolfe's fundamental statements were flawed: (i) “The content of research report is determined by their forms” may not be a fact, and (ii) that research appraisal being “subject to individual judgment based on insight and experience” will mean those without sufficient experience of performing research will be unable to judge adequately – hence an elitist's principle. From a realism standpoint, Porter then proposes multiple and open approaches for validity in qualitative research that incorporate parallel perspectives[ 43 , 44 ] and diversification of meanings.[ 44 ] Any work of qualitative research, when read by the readers, is always a two-way interactive process, such that validity and quality has to be judged by the receiving end too and not by the researcher end alone.

In summary, the three gold criteria of validity, reliability and generalizability apply in principle to assess quality for both quantitative and qualitative research, what differs will be the nature and type of processes that ontologically and epistemologically distinguish between the two.

Source of Support: Nil.

Conflict of Interest: None declared.

• Upload File
• Most Popular
• Art & Photos
• VALIDITY AND RELIABILITY IN QUALITATIVE RESEARCH · VALIDITY AND RELIABILITY IN QUALITATIVE...

Click here to load reader

VALIDITY AND RELIABILITY IN QUALITATIVE RESEARCH · VALIDITY AND RELIABILITY IN QUALITATIVE RESEARCH H.I.L. Brink (Conference Paper) Paper delivered at SA Society of Nurse Researchers’

Upload hadiep

Embed Size (px) 344 x 292 429 x 357 514 x 422 599 x 487

Citation preview

VALIDITY AND RELIABILITY IN QUALITATIVE RESEARCH

H.I.L. Brink (Conference Paper)

Paper delivered at SA Society o f Nurse Researchers’ Workshop-RAU 19Marchl993

INTRODUCTION

Validity and reliability are key aspects of all research. Meticulous attention to these two aspects can make the difference between good research and poor research and can help to assure that fellow scientists accept findings as credible and trustworthy. This is particulaary v ita l in q u a lita tiv e work, w here the researcher’s subjectivity can so readily cloud the interpretation of the data, and where research findings are often questioned or viewed with scepticism by the scientific community.

So those of us doing qualitative studies need to be especially sensitive to the issues of validity and reliability in our projects. We need to be attuned to the multiple factors that pose risks to the validity of our findings; and plan and implement various tactics or strategies into each stage of the research project to avoid or weaken these threatening factors. We need to be aware that the tactics or strategies used to address validity and reliability in qualitative research are not the same as in quantitative research.

The very nature of qualitative research methods does not lend to statistical or empirical calculations of validity. The qualitative researcher seeks basically the same ends through different methods which are better suited to a human subject matter. A large num ber of authors focusing on qualitative research methods have suggested tactics or strategies the researcher can employ to enhance the truthfulness or validity of qualitative findings (Chenitz & Swanson 1986, Crabtree & Miller 1992, Field & Morse 1985, Le Comple & Goetz 1982, Morse 1991, Sandelowski 1986 and Corbin & Strauss 1990).

The purpwse of this pap>er is to address the major risks and threats to validity and reliability in qualitative studies and in particular the tactics and strategies suggested by various qualitative researchers for avoiding or weakening the potential risks and threats. However, to refresh your memory and ensure that we all attach the same meaning to validity, reliability and qualitative research, attention will first be given to the definition and clarification of these key concepts.

THE CONCEPTS QUALITATIVE RESEARCH, VALIDITY AND RELIABILITY

The term qualitative research is really an umbrella term representing a variety of research approaches which share certain common elements. Qualitative researchers are not interested in causal laws but in people’s belief, experience and meaning systems from the perspective of the people. Methods used are more subjective than in quantitative research and do not include statistical analysis and empirical calculation. Phenomena are viewed holistically and in their social context Included under this unbrella term are such methods as grounded theory, phenomenology and ethnonursing-the three approaches earmarked for discussion today.

Validity in research is concerned with the accuracy and truthfulness of scientific findings (Le Comple and Goetz 1982: 32). A valid study should demonstrate what actually exists and a valid instrument or measure should actually measure what it is supposed to measure.

There are many types of validity and many names have been used to define the different types of validity. Campbell and Stanley (1966) have defined two major forms of validity that encompass the many types. They refer to "internal" and "external" validity, terms which are today used in most nursing research textbooks. Denzin (1970) used the distinction between internal and external validity and applied it to qualitative research. Intem^ validity is the term used to refer to the extent to which research findings are a true reflection or representation of reality rather than being the effects of extraneous variables. External validity addresses the degree or extent to which such representations or reflections of reality are legitim ately applicable across groups.

Reliability is concerned with the consistency, stability and repeatability of the informant’s accounts as well as the investigators’ abUity to collect and record information accurately (Selltiz et al 1976:182). It refers to the ability of a research method to yield consistently the same results over repeated testing periods. In other words, it requires that a researcher using the same or comparable methods obtained the same or comparable results every time he uses the methods on the same or comparable subjects. It further requires that the researcher has developed consistent responses or habits

in using the method and scoring or rating its results and that factors related to subjects and testing procedures have been managed to reduce measurement error.

Many qualitative researchers avoid the terms vali^ty and reliability and use terms such as credibility, trustworthliness, truth, value, applicability, consistency and confirmabUity, when referring to criteria for evaluating the scientific merit of qualitative research (Glaser and Strauss 1967, Leininger 1991, Lincoln & Guba 1985).

RISKS OR THREATS TO VALIDITY AND RELIABILITY

As stated previously researchers need to be attuned to the multiple factors that pose risks to the validity and reliability of their findings and plan and implement tactics or strategies to avoid or counter them. One of the key factors affecting validity and reliability is error. Error is inherent in all investigations and is inversely related to validity and reliability. The greater the degree of error the less accurate and truthful the results. Researchers thus must be especially watchful of the sources of error when planning and implementing their studies. For convenience sake the major sources of error can be categorised as follows:

(1) the researcher

(2) the subjects participating in the project

(3) the situation or social context

(4) the methods of data collection and analysis

THE RESEARCHER AS A RISK TO VALIDITY AND RELIABILITY

In a qualitative study the data-gathering instrument is frequently the researcher himself. Thus questions ofresearcher bias and researcher competency, if unchecked, may influence the trustw orthiness of data considerably. The very presence of the researcher may affect the validity of the data provided by subjects. When a new member is introduced into an interaction reactive effects can be expected. Participants may behave abnormally (Argyris 1952). They may seek to reveal themselves in the best possible light or withhold or distort certain information; in other words the researcher has created social behaviours in others that would normally not

Curationis, Vol. 16, No. 2, June 1993 35

have occurred. Based on her extensive fieldwork, Leinger holds that researchers need to be trusted before they will be able to obtain any accurate reliable or credible data (Leininger 1991: 92).

Le Comple & Goetz refer to research findings which hold that what the reseacher sees and reports is a function of the position he occupies within the participant group, the status accorded to them, and the role behaviour expected of them. The status position of the researcher can be that of an outsider or that of a participant group member. The status position can prevent the researcher from obtaining certain information and unless he is aware of this, invalid interpretation of the data may result. On the other hand, if the researcher becomes totally a part of the group there is a danger of "going native" or assuming the attitudes and behaviours of those under study. The researcher may then lose the ability to look objectively at what is happening and may develop bias towards the point of view of the group.

Researcher bias may also be introduced by the tendency of the researchers to observe subjects and interpret findings in the light of their own values, the tendency to selectively observe and record certain data at the expense of other data. The physical appearance of the researcher may influence the situation as also his dress and demeanour and personal attributes. For example, subjects may respond differently to males and females or male and female researchers may treat the subject differently.

WHAT CAN BE DONE TO DECREASE OR ELIMINATE RESEARCHER EFFECTS?

The first step in decreasing bias is to be aware of the possibility of introducing bias at various points of the research process. Field & Morse recom m end that researchers undergo extensive and rigorous training as interviewers and observers before undertaking qualitative studies. Researchers need to be trained in a manner that encourages an objective view of the phenomena under study. Furthermore, eveiy researcher should examine and declare his underlying values and assumptions in light of the research situation so that they can be considered when reading the research

Several writers recommend that the researcher spends a period of time in the situation before data collection starts. The researcher will then become sensitised to the situation and at the same time the subjects have the opportunity to become used to the presence of the researcher (Field & Morse 1985, Le Comple & Goetz 1992, Miles & Huberman 1984).

Leininger (1991: 11) suggests that the researcher should always assess and gauge his relationships with the subjects being studied in order to enter or get close to the people or situation under study, or to move from a stranger or distrusted person to a trusted and fnendly person during the research process.

She recommends the use of the stranger to filed model which she has developed and validated over many years. The purpose of this model is to serve as an assessment or reflection guide for the researcher to become consciously aware of his own behaviours, feelings and responses in relation to the behaviour and experiences of subjects and as he starts to collect data, for confirmation of truths.

Once data collection starts this should be done over a long period of time. The researcher as participant observer either "lives" with the subjects as anthropologists do or spends time visiting the research site regularly over a long period of time. When subjects are interviewed over time, their responses to the same questions on the same topic should be answered with the same information. This is a type of test-retest of the same informant on the same material. The threat of "going native" or becoming so enmeshed with subjects that researchers lose their own perspective can be offset by distancing oneself fix)m the subjects atregular intervals ie spending time away from the site, spread out site visits and discussing data with colleagues.

When field researchers are working alone, particularly when they are still unfamiliar with the setting, it is advisable that they enlist the aid of an informant who observes the occasion also. The researcher then records the activity on the spot and then reviews the written record with the informant for completeness and comprehensiveness of coverage. In some cases participant informants serve as arbiters, reviewing the days production of field notes to correct researcher misperceptions and misinterpretations. Commonly the researcher requests reactions to working analysis or processed materials from the informants. In this confirmation may be sought for various levels of the collection and analysis process (Le Comple & Goetz 1982:42).

THE PARTICIPATING SUBJECTS AS RISKS TO VALIDITY AND RELIABILITY

The truth of responses is a key concern when data are obtained through questionnaires and interviews. Bias may be introduced because of particular responses or characteristics of the informants. Informants may want to make things seem better or worse than they are. Hospitalised patients who are questioned about the quality of their care may indicate that the care is wonderful because they fear reprisal of staff. Conversely, they may respond that their care is awful. Informants may also attempt to please the researcher by responding in the way that they believe he expects. They may also fear that by giving negative responses, they will be placed in a devalued position by the researcher. Informants may also be unwilling to share certain information with the researcher and deliberately withhold or distort i t

The researcher can attempt to increase the validity of responses in such a setting

(1) by making sure that informants are very clear on the nature of the research eg. why the researcher is there, what heis studying, how he will collect data and what he wiU do with i t

(2) by first building a trust-relationship with the subjects and staying in that setting for a long period of time

(3) by interviewing the same informant on several occasions and making observations more than once and over time

(4) by comparing the results obtained with other evidence

(5) by confirming findings and analysis with informant (the danger with this technique is that subjects may become sensitised to the researcher’s inferences and provide the answer that support the researcher’s point)

(6) by keeping accurate and detailed fieldnotes to note the variations in̂ responses over the course of time

(7) by showing fieldnotes to a second outside researcher. Another researcher is often much quicker to see where or how a fieldworker is being misled or coopted.

Informant bias may also be introduced by factors within subjects themselves such as fatigue, motivation or anxiety, duration of recall, mood, attention span, state of health and whether or not they are in pain.

To overcome this bias the researcher conducts repeated interviews at different times and in different settings and then compares results.

Another informant bias which is quoted as a particular problem in qualitative research (Miles & Huberman 1984: 230, Sandelowski 1986: 32) is the "elite bias" (overweighting data from articulate, well-informed, usually high status informants and under rrepresenting data from intractable, less articulated lower-status ones).

The researcher can build in safeguards against this bias by good planning of selection of informants, by looking purposefully for contrasting cases (negative, extrem e, countervailing and by carefully considering contrasting views).

THE SOCIAL CONTEXT AS A RISK TO VALIDITY AND RELIABILITY

The social context under which the data are gathered is an important consideration in establishing validity and reliability of data. Individuals may behave differently under differing social circumstances, for example, when alone with the researcher they may

36 Curationis, Vol. 16, No. 2, June 1993

provide different information than when they are in a group, or patients may provide different information within the health care context than what they reveal in their home neighbourhood. The researcher who is mindful of this will interview the same inform ants and make observations of behaviour in a variety of settings to make comparisons of similarities and differences before attributing meaning.

He will also specify the physical, social and interpersonal contexts within which data are gathered. Particular attention to privacy may also be of value in certain studies when subjects hesitate to answer accurately if they fear they may be overheard by others in the environment

RISKS TO VALIDITY AND RELIABILITY PERTAINING TO DATA COLLECTION AND ANALYSIS

Because reliability and validity depend on the potential for subsequent researchers to reconstruct original strategies, the researcher who presents a vague account of his design is putting himself at risk of being accused of invalid and unreliable findings.

Researchers should therefore strive to present their methods clearly, that is precisely identify and thoroughly describe all strategies used to collect data and carefully document their field notes in the context of what was being observed to enable fellow researchers to form valid judgement. Many of the risks in data co llec tio n p e rta in to the researcher, respondents and social context and have already been dealt with.

Another major risk is sampling bias. The subjects under study may be overrepresenting or underpresenting the phenomena under study. The researcher may be overreliant on accessible and elite informants. He/she may be more likely to see confirming instances of original beliefs and perceptions than to see d iscon firm ing instances even when disconfirming instances are more fequent From one or two concrete vivid instances he may assume that there are dozens more, but may fail to verify this (Miles & Huberman 1984: 231).

In qualitative research sample selection is based on the ability of the subject to provide data relevant to the research question. To avoid inaccurate or insufficient data, the researcher must use his/her judgement based up)on the best available evidence to choose subjects who know enough, can recall enough, and are able to respwnde precisely to questions asked.

Secondly, the researcher should choose subjects who are able to report events not directly observable or accessible to the investigator. If the research is carried out in unfamiliar environment with an unfamiliar group of people in an unfamiliar culture, the

researcher should seek a panel of experts to assist with finding appropriate informants.

Thirdly, the researcher should do systematic "theoretical sampling" or in other words, continue to select subjects according to the findings that emerge in the course of the study. During this phase he should establish typicality or atypicality of observed events, behaviours or responses, extreme views or contrasting views. He should persist with theoretical sampling until no new information is obtained (until saturation or redundancy has been reached). There should be support for construction of a core category or several core categories which repeatedly occur while less and less new information emerges.

Another risk to representativeness of data results from the researcher’s non-continuous presence. The researcher has to infer what is happening when he is not there and usually offers plausible reasons rather than evidence. To balance this risk once again multiple sources, m ultip le m ethods, m ultiple investigators (judge panel) varying and multiple repetitions of measurement over time are recommended.

In m ost qualita tiv e approaches and particularly phenomenology, grounded theory and ethnomethods data analysis occurs simultaneously with data collection. All these methods use a series of similar steps for analysing which begin at the onset of the data collection phase.

Typical steps are coding for categories and themes and making memos about the context and variations in the phenomena under study, developing names for categories and elaborating classification systems and testing them within the data as they are collected. The findings at any point in this process wiU provide some direction for further data collection and the direction that the analysis may take. Judgements and inferences are made by the researcher. Major threats to the validity of data during this phase are firstly what Miles and Huberman call the "holistic fallacy" that tends to make data look more patterned or regular or congruent than they are and the tendency of the researcher to selectively observe and record certain data at the expense of other data. To avoid such selective inattention, the data analysis procedures should be exposed to a judge panel. The judge panel is selected on the basis of knowledge of content or knowledge of the research project.

Other means of providing validity and reliability are the use of the constant comparative method and the search for alternative hypothesis or negative cases (Hutchinson 1986: 116-117), checking that descriptions, explanations or theories about the data contain the typical and atypical elements of the data and obtaining validation from the subjects themselves (Sandelowski 1986: 35). Field and Morse (19895: 120) recommend that following strategies to reduce

threats to internal reliability with data analysis:-

(1) Low inference descriptors (verbatim accounts of information provided by informants to the researcher). Use of mechanical recording enhances the accuracy of such transcripts.

(2) Participant reviews of findings and peer examination.

In conclusion I would like to recapitulate the major critical strategies, suggested by leading qualitative researchers from various fields as essential for producing trustworthy and believable findings in qualitative research (Benner (1985), Brink & Wood (1988), Corbin & Strauss (1990), Field & Morse(1985), Glaser & Strauss (1967), Kirk & MiUer (1986), Kuzel & Like (1991), Lather(1986), Le Comple & Goetz (1982), Leininger(1991), Lincoln & Guba (1985), Miles & H uberm an (1984), M orse (1991), Sandelowski (1981).

1. TRIANGULATIONTriangulation refers to the use of two or more data sources, m ethods, investigators, theoretical perspectives and approaches to analysis in the study of a single phenomenon and then validating the congruence among them. The major goal of triangulation is to c ircum vent the personal biases of investigators and overcome the deficiencies intrinsic to single-investigator, single-theory, or single-method study thus increasing the validity of the study (Denzil 1989).

2. MULTIPLE repetitions of measurement over or long period of time, at different points of time, in different situations or settings and by different persons.

3. EXPERT CONSENSUAL VALIDATION FROM OTHERS familiar with the topic under study at various stages of the research p rocess. This involves independent analysis of the data by others (eg research colleagues, a judge panel or participant inform ants. (For example, following each initial category generation during analysis - independent fellow researchers or other experts in the field are invited to generate their own category system. This is them compared with the researchers).

4. MEMBER CHECKSThis refers to recycling of analysis back to informants. (Conclusions are given to the informants and feedback is requested about the accuracy of the content. This ensures that the researcher and the informant are viewing the data consistently.

5. SEARCHING FOR DISCONFIRMING EVIDENCEIn this step there is an active search for disconfirmation of what is believed to be true. A proposition deserves some degree of trust only when it has survived serious attempts to

Curationis, VoL 16, No. 2, June 1993 37

falsify i t (Cronbach in Lather 1986: 67). Searching for disconfirm ing evidence involves both purposive sampling and prolonged engagement with informants in the field. Purposeful sampling allows the researcher to include informants who may differ from key informants in critical ways. This purposeful sampling of individuals and the inclusion of conflicting as well as complementary accounts strengthens the description. One varies or contrasts the conditions as methodically as possible in order to determine what has an impact on the phenomenon in question (Corbin & Strauss 1990).

6. CHECKING FOR REPRESENT­ATIVENESST his step includes checking for the representativeness of the data as a whole, of the coding categories and of the examples used to analyze and present the data.

7. THICK DESCRIPTIONQualitative research, like its quantitative cousins, can be systematically evaluated only if their criteria and procedures are made explicit Hence validity and reliability can only be judged if a very detailed account of the context or setting within which the study took place and a thorough description of the procedures from the beginning to the end is given. Most qualitative researchers refer to this as thick description. However Lincoln and Guba (1985) use the term auditability. Auditability means that any reader or another researcher can follow the progression of events in the study and understand their logic. Such an account starts with the researcher recogn ising and d isc lo sing in itia l assumptions, suppositions and values thatmay have in flu en ced data gathering and processing. The researcher furthermore describes, explains and justifies.

(1) how he/she becomes interested in the subject matter of the study

(2) the specific purpose(s) of the study

(3) how subjects or pieces of evidence came to be included in the study and how they were approached, how theoretical sampling was done

(4) the impact, the subjects and the researcher/s had on each other

(5) how the data were collected ie

• the number and types of data collection methods and procedures used in the study

• the number of times data are collected on the same subject using the same method

• how often and over what period were data collected on tiie same and different informants

• the number of investigators collecting the same and different information on the same and different subjects; and

(6) when and how data were analysed

(7) the nature of the setting(s) in which data were collected

(8) how various elements of the data were weighted

(9) the inclusiveness and exclusiveness of the categories developed to con the data and

(10) how reliability and validity were answered

The account also needs to include the relationship between field notes and the conclusions based upon them. Memos, data display charts, indicating coding instructions and the actual placem ent of data into categories and the way different elements of the data were linked to each other, should be kept available.

Corbin and Strauss (1990:20) strongly urge investigators to adhere to the major criteria for answering credibility unless there are exceptional reasons for not doing so. In such unusual cases, researchers should know precisely how and why they depart from the criteria, say so in their writing, and submit the credibility of their findings to the reader. If we want to develop a valid, truthful and believable account of our qualitative studies it will be wise to take note of and implement the critical strategies discussed unless otherw ise indicated.

LIST OF SOURCES

Argyris, C. 1952. Diagnosing differences against the outsider. Journal o f Social Issues 8 (no 3): 24-34.

B enner, P. 1985. Q uality o f life: A phenomenological perspective. Advances in Nursing Science 8 (no 1): 1-14.

Campbell, D.T. & Stanley, J.C. 1963. Experimental and quasi-experimental designs for research. Chicago: Rand McNally.

Chenitz, W.C. & Swanson, J.M. 1986. From practice to grounded theory. Menlo Park CA: Addison Wesley.

Corbin, J. & Strauss A. 1990. Grounded theory research: Procedure, canons and evaluative criteria. Qualitative Sociology 13 (no 1): 3-21.

Crabtree, B.F. & MiUer, W. 1992. Doing qualitative research. Newbury Park, CA: Sage.

Denzin, N.K. 1970. The research act: A theoretical introduction to sociological methods. Chicago: Aldine Publishing Co.

Field, P.A. & Morse, J.M. 1985. Nursing Research. The application o f qualitative

approaches. Beckenham Kent: Croom Hehn.

Glaser, B. * Strauss A. 1967. The discovery o f grounded theory: Strategies fo r qualitative research. Chicago: Aldine Publishing Co.

Kirk, J. & Miller M. 1986. Reliability, validity and qualitative research. Beverly Hills CA: Sage.

Kuzel, A.J. & Like, R.C. 1991. Standards of trustworthiness for qualitative studies in primary care. In P. Norton, M. Stewart, F. Dudiver, M. Bass & E Dunn (eds). Primary Care Research: Traditional and innovative approaches, (p .138-158). Newbury park CA: Sage.

Lather, P. 1986. Issues of validity in openly ideological research: Between a rock and a soft place. Interchange 17: 63-84.

Le Comple, M.D. & Goetz, J.P. 1982. Problems of reliability and validity in ethnographic research. R eview o f Educational Research 52 (no 1): 31-60.

Leininger, M. 1991. Culture care, diversity and universality: A theory o f nursing. New York: N.LJ'f.

Lincoln, Y. & Guba, G. 1985. Naturalistic inquiry. Beverly Hills: Sage.

Miles, M.B. & Huberman, A.M. 1984. Qualitative data analysis: A sourcebook o f new methods. Beverly HiUs CA: Sage.

M orse, j.M . 1991. Qualitative nursing research: A contemporary dialogue. Rockville, Maryland: Aspen.

Oiler, C. 1982. The phenomenological approach in nursing research. Nursing Research?)! {no 3Q-. 178-181.

Reason, P. & Rowan, J. 1981. human inquiry: A sourcebook o f new paradigm research. New York: John Wiley & Sons.

Sandelowski, M. 1986. The problem of rigor in qualitative research. Advances in Nursing Science 8 (no 3): 27-37.

Seltiz, C. & Wrightsman, L.C. & Cook, W.S. 1976. Research m ethods in socia l relations. 3rd edition. New York: Holt Rinehart & Winston.

Strauss, A. & Corbin, J. 1990. Basics o f qualitative research. Newbury Park CA: Sage.

H. BrinkPrcfessor, Department o f Nursing Science

38 Curationis, Vol. 16, No. 2, June 1993

Measurement,Reliability &Validity

Chapter 6 validity & reliability

Reliability and Validity - DissertationRecipes.comdissertationrecipes.com/.../2011/04/Reliability-and-Validity-final.pdf · Validity Validity denotes the extent to which an instrument

Presentation Validity & Reliability

Reliability and Validity - Association of American Medical ... - Measuring educational... · Speak of reliability of the scores of an instrument ... Reliability and Validity . Validity

Reliability or Validity

5 - reliability and validity of qualitative research ( Dr. Abdullah Al-Beraidi - Dr. Ibrahim Althonayan - Dr.Ramzi)

Reliability and validity

Reliability Validity

Testing Reliability &Validity

Reliability and Inter-rater Reliability in Qualitative ...andreaforte.net/McDonald_Reliability_CSCW19.pdf · Reliability and validity are features of empirical research that date

VALIDITY AND RELIABILITY - Noida International Universityniu.edu.in/soe/Validity-and-Reliability-BED108.pdf · 2020-04-29 · VALIDITY AND RELIABILITY ... A test’s construct validity

Validity and Reliability of Observation and Data ... · Validity & Reliability/ 6 Validity and reliability of observation and data collection in biographical research Summary The

Validity in qualitative research revisited

Reliability and Validity in qualitative research

Validity + Reliability

Validity and Reliability of Qualitative Assessments based on self-reported statements

Variables, Validity & Reliability

Validity and Reliability

Reliability And Validity

Reliability & validity

Chapter 4: Reliability and Validity · 2014-09-10 · Chapter 4: Reliability and Validity Validity Validity The Standards for Educational and Psychological Testing define validity

Validity/Reliability Matters

Quality in Qualitative Studies the Case of Validity Reliability and Generalizability

RUNNING HEAD: Reliability and Validity Coefficients Reliability …gknafl/research/FA/rel/reliability.paper.pdf · RUNNING HEAD: Reliability and Validity Coefficients Reliability

Validity & reliability seminar

Reliability, Validity, & Scaling

Validity and Reliability of Observation and Data ...eurotas.org/wp-content/uploads/pdf/valideng_full.pdf · Validity & Reliability/ 2 Validity and Reliability of Observation and Data

Validity, Reliability, & Sampling