definition of comparative research methodology

Communication

iResearchNet

Custom Writing Services

Comparative research.

A specific comparative research methodology is known in most social sciences. Its definition often refers to countries and cultures at the same time, because cultural differences between countries can be rather small (e.g., in Scandinavian countries), whereas very different cultural or ethnic groups may live within one country (e.g., minorities in the United States). Comparative studies have their problems on every level of research, i.e., from theory to types of research questions, operationalization, instruments, sampling, and interpretation of results.

The major problem in comparative research, regardless of the discipline, is that all aspects of the analysis from theory to datasets may vary in definitions and/or categories. As the objects to compare usually belong to different systemic contexts, the establishment of equivalence and comparability is thus a major challenge of comparative research. This is often “operationalized” as functional equivalence, i.e., the functionality of the research objects within the different system contexts must be equivalent. Neither equivalence nor its absence, “bias,” can be presumed. It has to be analyzed and tested for on all the different levels of the research process.

Equivalence And Bias

Equivalence has to be analyzed and established on at least three levels: on the levels of the construct, the item, and the method (van de Vijver & Tanzer 1997). Whenever a test on any of these levels shows negative results, a cultural bias is supposable. Thus, bias on these three levels can be described as the opposite of equivalence. Van de Vijver and Leung (1997) define bias as the variance within certain variables or indicators that can only be caused by culturally unspecific measurement. For example, a media content analysis could examine the amount of foreign affairs coverage in one variable, by measuring the length of newspaper articles. If, however, newspaper articles in country A are generally longer than they are in country B, irrespective of their topic, the result of a sum or mean index of foreign affairs coverage would almost inevitably lead to the conclusion that the amount of foreign affairs coverage in country A is higher than in country B. This outcome would be hardly surprising and not in focus with the research question, because the countries’ average amount of foreign affairs coverage is not related to the national average length of articles. To avoid cultural bias, the results must be standardized or weighted, for example by the mean article length.

To find out whether construct equivalence can be assumed, the researcher will generally require external data and rather complex procedures of culture-specific construct validation(s). Ideally, this includes analyses of the external structure, i.e., theoretical references to other constructs, as well as an examination of the latent or internal structure. The internal structure consists of the relationships between the construct’s sub-dimensions. It can be tested using confirmatory factor analyses, multidimensional scaling, or item analyses. Equivalence can be assumed if the construct validation for every culture has been successful and if the internal and external structures are identical in every country. However, it has to be stated that it is hardly possible to prove construct equivalence beyond any doubts (Wirth & Kolb 2004).

Even with a given construct equivalence, bias can still occur on the item level. The verbalization of items in surveys and of definitions and categories in content analyses can cause bias due to culture-specific connotations. Item bias is mostly evoked by bad, in the sense of nonequivalent, translation or by culture-specific questions and categories (van de Vijver & Leung 1997). Compared to the complex procedures discussed in the case of construct equivalence, the testing for item bias is rather simple (once construct equivalence has been established): Persons from different cultures who take the same positions or ranks on an imaginary construct scale must show the same answering attitude toward every item that measures the construct. Statistically, the correlation of the single items with the total (sum) score have to be identical in every culture, as test theory generally uses the total score to estimate the position of any individual on the construct scale. In brief, equivalence on the item level is established whenever the same sub-dimensions or issues can be used to explain the same theoretical construct in every country (Wirth & Kolb 2004).

When the instruments are ready for application, method equivalence comes to the fore. Method equivalence consists of sample equivalence, instrument equivalence, and administration equivalence. Violation of any of these equivalences produces a method bias. Sample equivalence refers to an equivalent selection of subjects or units of analysis. Instrument equivalence deals with the examination of whether people in every culture agree to take part in the study equivalently, and whether they are used to the instruments equivalently (Lauf & Peter 2001). Finally, a bias on the administration level can occur due to culturespecific attitudes of the interviewers that might produce culture-specific answers. Another source of administration bias could be found in socio-demographic differences between the various national interviewer teams (van de Vijver & Tanzer 1997).

The Role Of Theory

Theory plays a major role in three dimensions when looking for a comparative research strategy: theoretical diversity, theory drivenness, and contextual factors (Wirth & Kolb 2004). Swanson (1992) distinguishes between three principal strategies of dealing with international theoretical diversity. A common possibility is called the avoidance strategy. Many international comparisons are made by teams that come from one culture or nation only. Usually, their research interests are restricted to their own (scientific) socialization. Within this monocultural context, broad approaches cannot be applied and “intertheoretical” questions cannot be answered. This strategy includes atheoretical and unitheoretical (referring to one national theory) studies with or without contextualization (van den Vijver & Leung 2000; Wirth & Kolb 2004).

The pretheoretical strategy tries to avoid cultural and theoretical bias in another way: these studies are undertaken without a strict theoretical background until results are to be interpreted. The advantage of this strategy lies in the exploration, i.e., in developing new theories. Although, following the strict principles of critical rationalism, because of the missing theoretical background the proving of theoretical deduced hypotheses is not applicable (Popper 1994). Most of the results remain on a descriptive level and never reach theoretical diversity. Besides, the instruments for pretheoretical studies must be almost “holistic,” in order to integrate every theoretical construct conceivable for the interpretation. These studies are mostly contextualized and can, thus, become rather extensive (Swanson 1992).

Finally, when a research team develops a meta-theoretical orientation to build a framework for the basic theories and research questions, the data can be analyzed using different theoretical backgrounds. This meta-theoretical strategy allows the extensive use of all data and contextual factors, producing, however, quite a variety of often very different results, which are not easily summarized in one report (Swanson 1992). It is obvious that the higher is the level of theoretical diversity, the greater has to be the effort for construct equivalence.

Research Questions

Van de Vijver and Leung (1996, 1997) distinguish between two types of research questions: structure-oriented questions are mostly interested in the relationship between certain variables, whereas level-oriented questions focus on the parameter values. If, for example, a knowledge gap study analyzes the relationship between the knowledge gained from television news by recipients with high and low socio-economic status (SES) in the UK and the US, the question is structure oriented, because the focus is on a national relationship of knowledge indices and the mean gain of knowledge is not taken into account. Usually, structure-oriented data require correlation or regression analyses. If the main interest of the study is a comparison of the mean gain of knowledge of people with low SES in the UK and the US, the research question is level oriented, because the two knowledge indices of the two nations are to be compared. In this case, one would most probably use analyses of variance. The risk for cultural bias is the same for both kinds of research questions.

Emic And Etic Strategies Of Operationalization

Before the operationalization of an international comparison, the research team has to analyze construct equivalence to prove comparability. In the case of missing internal construct equivalence, the construct cannot be measured equivalently in every country. The decision of whether or not to use the same instruments in every country does not have any impact on this problem of missing construct equivalence. An emic approach could solve this problem. The operationalization for the measurement of the construct(s) is developed nationally, so that the culture-specific adequacy of each of the national instruments will be high. Comparison on the construct level remains possible, even though the instruments vary culturally, because functional equivalence has been established on the construct level by the culture-specific measurement. In general, this procedure will even be possible if national instruments already exist.

As measurement differs from culture to culture, the integration of the national results can be very difficult. Strictly speaking, this disadvantage of emic studies only allows for the interpretation of a structure-oriented outcome with a thorny validation process. It has to be proven that the measurements with different indicators on different scales really lead to data on equivalent constructs. By using external reference data from every culture, complex weighting and standardization procedures can possibly lead to valid equalization of levels and variance (van de Vijver & Leung 1996, 1997). In research practice, emic measuring and data analysis is often used to cast light on cultural differences.

If construct equivalence can be assumed after an in-depth analysis, an etic modus operandi could be recommended. In this logic, approaching the different cultures by using the same or a slightly adapted instrument is valid because the constructs are “functioning” equally in every culture. Consequently, an emic proceeding should most probably come to similar instruments in every culture. Reciprocally, an etic approach must lead to bias and measurement artifacts when applied under the circumstances of missing construct equivalence.

It is obvious that the advantages of emic proceedings are not only the adequate measurement of culture-specific elements, but also the possible inclusion of, e.g., idiographic elements of each culture. Thus, this approach can be seen as a compromise of qualitative and quantitative methodologies. Sometimes comparative researchers suggest analyzing cultural processes in a holistic way without crushing them into variables; psychometric, quantitative data collection would be suitable for similar cultures only. As an objection to this simplification, one should remember the emic approach’s potential to provide the researchers with comparable data, as described above. In contrast, holistic analyses produce culture-specific outcomes that will not be comparable; the problem of equivalence and bias has only been moved to the interpretation of results.

Adaptation Of The Instruments

Difficulties in establishing equivalence are regularly linked to linguistic problems. How can a researcher try to establish functional equivalence without knowledge of every language of the cultures under examination? For a linguistic adaptation of the theoretical background as well as for the instruments, one can again discriminate between “more etic” and “more emic” approaches.

Translation-oriented approaches produce two translated versions of the text: one in the “foreign” language and one after retranslation into the original language. The latter version can be compared to the original version to evaluate the translation. Note that this method produces eticly formed instruments, which can only work whenever functional equivalence has been established on every superior level. Van de Vijver and Tanzer (1997) call this procedure application of an instrument in another language. In a “more emic” cultural adaptation, cultural singularities can be included if, e.g., culture-specific connotations are counterbalanced by a different item formulation.

Purely emic approaches develop entirely culture-specific instruments without translation. Two assembly approaches are available (van de Vijver & Tanzer 1997). First, in order to maintain the committee approach, an international interdisciplinary group of experts of the cultures, languages, and research field decides whether the instruments are to be formed culture-specifically or whether a cultural adaptation will be sufficient. Second, the dual-focus approach tries to find a compromise between literal, grammatical, syntactical, and construct equivalence. Native speakers and/or bilinguals arrange the different language versions together with the research team in a multistep procedure (Erkut et al. 1999).

Usually, researchers use personal preference and accessibility of data to select the countries or cultures to study. This kind of forming of an atheoretical sample avoids many problems (but not cultural bias!). At the same time, it ignores some advantages. Przeworski and Teune (1970) suggest two systematic and theory-driven approaches. The quasiexperimental most similar systems design tries to stress cultural differences. To minimize the possible causes for the differences, those countries are chosen that are the “most similar,” so that the few dissimilarities between these countries are most likely to be the reason for the different outcomes. Whenever the hypotheses highlight intercultural similarities, the most different systems design is appropriate. Here, in a kind of turned-around quasi-experimental logic, the focus lies on similarities between cultures, even though these differ in the greatest possible way (Kolb 2004; Wirth & Kolb 2004).

Random sampling and representativeness play a minor role in international comparisons. The number of states in the world is limited and a normal distribution for the social factors under examination, i.e., the precondition of random sampling, cannot be assumed. Moreover, many statistical methods meet problems when applied under the condition of a low number of cases (Hartmann 1995).

Data Analysis And Interpretation Of Results

Given the presented conceptual and methodological problems of international research, special care must be taken over data analysis and the interpretation of results. As the implementation of every single variable of relevance is hardly accomplishable in international research, the documentation of methods, work process, and data analysis is even more important here than in single-culture studies. Thus, the evaluation of the results must ensue in additional studies. At any rate, an intensive use of different statistical analyses beyond the “general” comparison of arithmetic means can lead to further validation of the results and especially of the interpretation. Van de Vijver and Leung (1997) present a widespread summary of data analysis procedures, including structureand level-oriented approaches, examples of SPSS syntax, and references.

Following Przeworski’s and Teune’s research strategies (1970), results of comparative research can be classified into differences and similarities between the research objects. For both types, Kohn (1989) introduces two separate ways of interpretation. Intercultural similarities seem to be easier to interpret, at first glance. The difficulties emerge when regarding equivalence on the one hand (i.e., there may be covert cultural differences within culturally biased similarities), and when regarding the causes of similarities on the other. The causes will be especially hard to determine in the case of “most different” countries, as different combinations of different indicators can theoretically produce the same results. Esser (2000) refers to diverse theoretical backgrounds that will lead either to differences (e.g., action-theoretically based micro-research) or to similarities (e.g., system-theoretically oriented macro-approaches). In general, the starting point of Przeworski and Teune (1970) seems to be the easier way to come to interesting results and interpretations, using the quasi-experimental approach for “most similar systems with different outcome.” In addition to the advantages of causal interpretation, the “most similar” systems are likely to be equivalent from the top level of the construct to the bottom level of indicators and items. “Controlling” other influences can minimize methodological problems and makes analysis and interpretation more valid.

References:

• Erkut, S., Alarcón, O., García Coll, C., Tropp, L. R., & Vázquez García, H. A. (1999). The dual-focus approach to creating bilingual measures. Journal of Cross-Cultural Psychology, 30(2), 206 –218.
• Esser, F. (2000). Journalismus vergleichen: Journalismustheorie und komparative Forschung [Comparing journalism: Journalism theory and comparative research]. In M. Löffelholz (ed.), Theorien des Journalismus: Ein diskursives Handbuch [Journalism theories: A discoursal handbook]. Wiesbaden: Westdeutscher, pp. 123 –146.
• Esser, F., & Pfetsch, B. (eds.) (2004). Comparing political communication: Theories, cases, and challenges. Cambridge: Cambridge University Press.
• Hartmann, J. (1995). Vergleichende Politikwissenschaft: Ein Lehrbuch [Comparative political science: A textbook]. Frankfurt: Campus.
• Kohn, M. L. (1989). Cross-national research as an analytic strategy. In M. L. Kohn (ed.), Crossnational research in sociology. Newbury Park, CA: Sage, pp. 77–102.
• Kolb, S. (2004). Voraussetzungen für und Gewinn bringende Anwendung von quasiexperimentellen Ansätzen in der kulturvergleichenden Kommunikationsforschung [Precondition for and advantageous application of quasi-experimental approaches in comparative communication research]. In W. Wirth, E. Lauf, & A. Fahr (eds.), Forschungslogik und – design in der Kommunikationswissenschaft, vol. 1: Einführung, Problematisierungen und Aspekte der Methodenlogik aus kommunikationswissenschaftlicher Perspektive [Logic of inquiry and research designs in communication research, vol. 1: Introduction, problematization, and aspects of methodology from a communications point of view]. Cologne: Halem, 2004, pp. 157–178.
• Lauf, E., & Peter, J. (2001). Die Codierung verschiedensprachiger Inhalte: Erhebungskonzepte und Gütemaße [Coding of content in different languages: Concepts of inquiry and quality indices]. In E. Lauf & W. Wirth (eds.), Inhaltsanalyse: Perspektiven, Probleme, Potentiale [Content analysis: Perspectives, problems, potentialities]. Cologne: Halem, pp. 199 –217.
• Popper, K. R. (1994). Logik der Forschung [Logic of inquiry], 10th edn. Tübingen: Mohr.
• Przeworski, A., & Teune, H. (1970). The logic of comparative social inquiry. Malabar, FL: Krieger.
• Swanson, D. L. (1992). Managing theoretical diversity in cross-national studies of political In J. G. Blumler, J. M. McLeod, & K. E. Rosengren (eds.), Comparatively speaking: Communication and culture across space and time. Newbury Park, CA: Sage, pp. 19 –34.
• Vijver, F. van de, & Leung, K. (1996). Methods and data analysis of comparative research. In J. W. Berry, Y. H. Poortinga, & J. Pandey (eds.), Handbook of cross-cultural research. Boston, MA: Allyn and Bacon, pp. 257–300.
• Vijver, F. van de, & Leung, K. (1997). Methods and data analysis of cross-cultural research. Thousand Oaks, CA: Sage.
• Vijver, F. van de, & Leung, K. (2000). Methodological issues in psychological research on culture. Journal of Cross-Cultural Psychology, 31(1), 33 –51.
• Vijver, F. van de, & Tanzer, N. K. (1997). Bias and equivalence in cross-cultural assessment: An overview. European Journal of Applied Psychology, 47(4), 263 –279.
• Wirth, W., & Kolb, S. (2004). Designs and methods of comparative political communication research. In F. Esser, & B. Pfetsch (eds.), Comparing political communication: Theories, cases, and challenges. Cambridge: Cambridge University Press, pp. 87–111.

Characteristics of a Comparative Research Design

Hannah richardson, 28 jun 2018.

Comparative research essentially compares two groups in an attempt to draw a conclusion about them. Researchers attempt to identify and analyze similarities and differences between groups, and these studies are most often cross-national, comparing two separate people groups. Comparative studies can be used to increase understanding between cultures and societies and create a foundation for compromise and collaboration. These studies contain both quantitative and qualitative research methods.

Explore this article

• Comparative Quantitative
• Comparative Qualitative
• When to Use It
• When Not to Use It

1 Comparative Quantitative

Quantitative, or experimental, research is characterized by the manipulation of an independent variable to measure and explain its influence on a dependent variable. Because comparative research studies analyze two different groups -- which may have very different social contexts -- it is difficult to establish the parameters of research. Such studies might seek to compare, for example, large amounts of demographic or employment data from different nations that define or measure relevant research elements differently.

However, the methods for statistical analysis of data inherent in quantitative research are still helpful in establishing correlations in comparative studies. Also, the need for a specific research question in quantitative research helps comparative researchers narrow down and establish a more specific comparative research question.

2 Comparative Qualitative

Qualitative, or nonexperimental, is characterized by observation and recording outcomes without manipulation. In comparative research, data are collected primarily by observation, and the goal is to determine similarities and differences that are related to the particular situation or environment of the two groups. These similarities and differences are identified through qualitative observation methods. Additionally, some researchers have favored designing comparative studies around a variety of case studies in which individuals are observed and behaviors are recorded. The results of each case are then compared across people groups.

3 When to Use It

Comparative research studies should be used when comparing two people groups, often cross-nationally. These studies analyze the similarities and differences between these two groups in an attempt to better understand both groups. Comparisons lead to new insights and better understanding of all participants involved. These studies also require collaboration, strong teams, advanced technologies and access to international databases, making them more expensive. Use comparative research design when the necessary funding and resources are available.

4 When Not to Use It

Do not use comparative research design with little funding, limited access to necessary technology and few team members. Because of the larger scale of these studies, they should be conducted only if adequate population samples are available. Additionally, data within these studies require extensive measurement analysis; if the necessary organizational and technological resources are not available, a comparative study should not be used. Do not use a comparative design if data are not able to be measured accurately and analyzed with fidelity and validity.

• 1 San Jose State University: Selected Issues in Study Design
• 2 University of Surrey: Social Research Update 13: Comparative Research Methods

About the Author

Hannah Richardson has a Master's degree in Special Education from Vanderbilt University and a Bacheor of Arts in English. She has been a writer since 2004 and wrote regularly for the sports and features sections of "The Technician" newspaper, as well as "Coastwach" magazine. Richardson also served as the co-editor-in-chief of "Windhover," an award-winning literary and arts magazine. She is currently teaching at a middle school.

Related Articles

Research Study Design Types

Correlational Methods vs. Experimental Methods

Different Types of Methodologies

Quasi-Experiment Advantages & Disadvantages

What Are the Advantages & Disadvantages of Non-Experimental...

Independent vs. Dependent Variables in Sociology

Methods of Research Design

Qualitative Research Pros & Cons

How to Form a Theoretical Study of a Dissertation

What Is the Difference Between Internal & External...

Difference Between Conceptual & Theoretical Framework

The Advantages of Exploratory Research Design

What Is Quantitative Research?

What is a Dissertation?

What Are the Advantages & Disadvantages of Correlation...

What Is the Meaning of the Descriptive Method in Research?

How to Use Qualitative Research Methods in a Case Study...

How to Tabulate Survey Results

How to Cross Validate Qualitative Research Results

Types of Descriptive Research Methods

Regardless of how old we are, we never stop learning. Classroom is the educational resource for people of all ages. Whether you’re studying times tables or applying to college, Classroom has the answers.

• Accessibility
• Terms of Use
• Privacy Policy
• Copyright Policy
• Manage Preferences

© 2020 Leaf Group Ltd. / Leaf Group Media, All Rights Reserved. Based on the Word Net lexical database for the English Language. See disclaimer .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Clin Oncol

Methods in Comparative Effectiveness Research

Katrina armstrong.

From the Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA.

Comparative effectiveness research (CER) seeks to assist consumers, clinicians, purchasers, and policy makers to make informed decisions to improve health care at both the individual and population levels. CER includes evidence generation and evidence synthesis. Randomized controlled trials are central to CER because of the lack of selection bias, with the recent development of adaptive and pragmatic trials increasing their relevance to real-world decision making. Observational studies comprise a growing proportion of CER because of their efficiency, generalizability to clinical practice, and ability to examine differences in effectiveness across patient subgroups. Concerns about selection bias in observational studies can be mitigated by measuring potential confounders and analytic approaches, including multivariable regression, propensity score analysis, and instrumental variable analysis. Evidence synthesis methods include systematic reviews and decision models. Systematic reviews are a major component of evidence-based medicine and can be adapted to CER by broadening the types of studies included and examining the full range of benefits and harms of alternative interventions. Decision models are particularly suited to CER, because they make quantitative estimates of expected outcomes based on data from a range of sources. These estimates can be tailored to patient characteristics and can include economic outcomes to assess cost effectiveness. The choice of method for CER is driven by the relative weight placed on concerns about selection bias and generalizability, as well as pragmatic concerns related to data availability and timing. Value of information methods can identify priority areas for investigation and inform research methods.

INTRODUCTION

The desire to determine the best treatment for a patient is as old as the medical field itself. However, the methods used to make this determination have changed substantially over time, progressing from the humoral model of disease through the Oslerian application of clinical observation to the paradigm of experimental, evidence-based medicine of the last 40 years. Most recently, the field of comparative effectiveness research (CER) has taken center stage 1 in this arena, driven, at least in part, by the belief that better information about which treatment a patient should receive is part of the answer to addressing the unsustainable growth in health care costs in the United States. 2 , 3

The emergence of CER has galvanized a re-examination of clinical effectiveness research methods, both among researchers and policy organizations. New definitions have been created that emphasize the necessity of answering real-world questions, where patients and their clinicians have to pick from a range of possible options, recognizing that the best choice may vary across patients, settings, and even time periods. 4 The long-standing emphasis on double-blinded, randomized controlled trials (RCTs) is increasingly seen as impractical and irrelevant to many of the questions facing clinicians and policy makers today. The importance of generating information that will “assist consumers, clinicians, purchasers, and policy makers to make informed decisions” 1 (p29) is certainly not a new tenet of clinical effectiveness research, but its primacy in CER definitions has important implications for research methods in this area.

CER encompasses both evidence generation and evidence synthesis. 5 Generation of comparative effectiveness evidence uses experimental and observational methods. Synthesis of evidence uses systematic reviews and decision and cost-effectiveness modeling. Across these methods, CER examines a broad range of interventions to “prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.” 1 (p29)

EXPERIMENTAL METHODS

RCTs became the gold standard for clinical effectiveness research soon after publication of the first RCT in 1948. 6 An RCT compares outcomes across groups of participants who are randomly assigned to different interventions, often including a placebo or control arm ( Fig 1 ). RCTs are widely revered for their ability to address selection bias, the correlation between the type of intervention received and other factors associated with the outcome of interest. RCTs are fundamental to the evaluation of new therapeutic agents that are not available outside of a trial setting, and phase III RCT evidence is required for US Food and Drug Administration approval. RCTs are also important for evaluating new technology, including imaging and devices. Increasingly, RCTs are also used to shed light on biology through correlative mechanistic studies, particularly in oncology.

Experimental and observational study designs. In a randomized controlled trial, a population of interest is screened for eligibility, randomly assigned to alternative interventions, and observed for outcomes of interest. In an observational study, the population of interest is assigned to alternative interventions based on patient, provider, and system factors and observed for outcomes of interest.

However, traditional approaches to RCTs are increasingly seen as impractical and irrelevant to many of the questions facing clinicians and policy makers today. RCTs have long been recognized as having important limitations in real-world decision making, 7 including: one, RCTs often have restrictive enrollment criteria so that the participants do not resemble patients in practice, particularly in clinical characteristics such as comorbidity, age, and medications or in sociodemographic characteristics such as race, ethnicity, and socioeconomic status; two, RCTs are often not feasible, either because of expense, ethical concerns, or patient acceptance; and three, given their expense and enrollment restrictions, RCTs are rarely able to answer questions about how the effect of the intervention may vary across patients or settings.

Despite these limitations, there is little doubt that RCTs will be a major component of CER. 8 Furthermore, their role is likely to grow with new approaches that increase their relevance in clinical practice. 9 Adaptive trials use accumulating evidence from the trials to modify trial design of the trial to increase efficiency and the probability that trial participants benefit from participation. 10 These adaptations can include changing the end of the trial, changing the interventions or intervention doses, changing the accrual rate, or changing the probability of being randomly assigned to the different arms. One example of an adaptive clinical trial in oncology is the multiarm I-Spy2 trial, which is evaluating multiple agents for neoadjuvant breast cancer treatment. 11 The I-Spy2 trial uses an adaptive approach to assigning patients to treatment arms (where patients with a tumor profile are more likely to be assigned to the arm with the best outcomes for that profile), and data safety monitoring board decisions are guided by Bayesian predicted probabilities of pathologic complete response. 12 , 13 Other examples of adaptive clinical trials in oncology include a randomized trial of four regiments in metastatic prostate cancer, where patients who did not respond to their initial regimen (selected based on randomization) were then randomly assigned to the remaining three regimens, 14 and the CALGB (Cancer and Leukemia Group B) 49907 trial, which used Bayesian predictive probabilities of inferiority to determine the final sample size needed for the comparison of capecitabine and standard chemotherapy in elderly women with early-stage breast cancer. 15 Pragmatic trials relax some of the traditional rules of RCTs to maximize the relevance of the results for clinicians and policy makers. These changes may include expansion of eligibility criteria, flexibility in the application of the intervention and in the management of the control group, and reduction in the intensity of follow-up or procedures for assessing outcomes. 16

OBSERVATIONAL METHODS

The emergence of comparative effectiveness has led to a renewed interest in the role of observational studies for assessing the benefits and harms of alternative interventions. Observational studies compare outcomes between patients who receive different interventions through some process other than investigator randomization. Most commonly, this process is the natural variation in clinical care, although observational studies also can take advantage of natural experiments, where higher-level changes in care delivery (eg, changes in state policy or changes in hospital unit structure) lead to changes in intervention exposure between groups. Observational studies can enroll patients by exposure (eg, type of intervention) using a cohort design or outcome using a case-control design. Cohort studies can be performed prospectively, where participants are recruited at the time of exposure, or retrospectively, where the exposure occurred before participants are identified.

The strengths and limitations of observational studies for clinical effectiveness research have been debated for decades. 7 , 17 Because the incremental cost of including an additional participant is generally low, observational studies often have relatively large numbers of participants who are more representative of the general population. Large, diverse study populations make the results more generalizable to real-world practice and enable the examination of variation in effect across patient subgroups. This advantage is particularly important for understanding effectiveness among vulnerable populations, such as racial minorities, who are often underrepresented in RCT participants. Observational studies that take advantage of existing data sets are able to provide results quickly and efficiently, a critical need for most CER. Currently, observational data already play an important role in influencing guidelines in many areas of oncology, particularly around prevention (eg, nutritional guidelines, management of BRCA1/2 mutation carriers) 18 , 19 and the use of diagnostic tests (eg, use of gene expression profiling in women with node-negative, estrogen receptor–positive breast cancer). 20 However, observational studies also have important limitations. Observational studies are only feasible if the intervention of interest is already being used in clinical practice; they are not possible for evaluation of new drugs or devices. Observational studies are subject to bias, including performance bias, detection bias, and selection bias. 17 , 21 Performance bias occurs when the delivery of one type of intervention is associated with generally higher levels of performance by the health care unit (ie, health care quality) than the delivery of a different type of intervention, making it difficult to determine if better outcomes are the result of the intervention or the accompanying higher-quality health care. Detection bias occurs when the outcomes of interest are more easily detected in one group than another, generally because of differential contact with the health care system between groups. Selection bias is the most important concern in the validity of observational studies and occurs when intervention groups differ in characteristics that are associated with the outcome of interest. These differences can occur because a characteristic is part of the decision about which treatment to recommend (ie, disease severity), which is often termed confounding by indication, or because it is correlated with both intervention and outcome for another reason. A particular concern for CER of therapies is that some new agents may be more likely to be used in patients for whom established therapies have failed and who are less likely to be responsive to any therapy.

There are two main approaches for addressing bias in observational studies. First, important potential confounders must be identified and included in the data collection. Measured confounders can be addressed through multivariate and propensity score analysis. A telling example of the importance of adequate assessment of potential confounders was found through examination of the observational studies of hormone replacement therapy (HRT) and coronary heart disease (CHD). Meta-analyses of observational studies had long estimated a substantial reduction in CHD risk with the use of postmenopausal HRT. However, the WHI (Women's Health Initiative) trial, a large, double-blind RCT of postmenopausal HRT, found no difference in CHD risk between women assigned to HRT or placebo. Although this apparent contradiction is often used as general evidence against the validity of observational studies, a re-examination of the observational studies demonstrated that studies that adjusted for measures of socioeconomic status (a clear confounder between HRT use and better health outcomes) had results similar to those of the WHI, whereas studies that did not adjust for socioeconomic status found a protective effect with HRT 22 ( Fig 2 ). The use of administrative data sets for observational studies of comparative effectiveness is likely to become increasingly common as health information technology spreads, and data become more accessible; however, these data sets may be particularly limiting in their ability to include data on potential confounders. In some cases, the characteristics that influence the treatment decision may not be available in the data (eg, performance status, tumor gene expression), making concerns about confounding by indication too high to proceed without adjusting data collection or considering a different question.

Meta-analysis of observational studies of hormone replacement therapy (HRT) and coronary artery disease incidence comparing studies that did and did not adjust for socioeconomic status (SES). Data adapted. 22

Second, several analytic approaches can be used to address differences between groups in observational studies. The standard analytic approach involves the use of multivariable adjustment through regression models. Regression allows the estimation of the change in the outcome of interest from the difference in intervention, holding the other variables in the model (covariates) constant. Although regression remains the standard approach to analysis of observational data, regression can be misleading if there is insufficient overlap in the covariates between groups or if the functional forms of the variables are incorrectly specified. 23 Furthermore, the number of covariates that can be included is limited by the number of participants with the outcome of interest in the data set.

Propensity score analysis is another approach to the estimation of an intervention effect in observational data that enables the inclusion of a large number of covariates and a transparent assessment of the balance of covariates after adjustment. 23 – 26 Propensity score analysis uses a two-step process, first estimating the probability of receiving a particular intervention based on the observed covariates (the propensity score) and estimating the effect of the intervention within groups of patients who had a similar probability of receiving the intervention (often grouped as quintiles of propensity score). The degree to which the propensity score is able to represent the differences in covariates between intervention groups is assessed by examining the balance in covariates across propensity score categories. In an ideal situation, after participants are grouped by their propensity for being treated, those who receive different interventions have similar clinical and sociodemographic characteristics—at least for the characteristics that are measured ( Table 1 ). Rates of the outcomes of interest are then compared between intervention groups within each propensity score category, paying attention to whether the intervention effect differs across patients with a different propensity for receiving the intervention. In addition, the propensity score itself can be included in a regression model estimating the effect of the intervention on the outcome, a method that also allows for additional adjustment for covariates that were not sufficiently balanced across intervention groups within propensity score categories.

Hypothetic Example of Propensity Score Analysis Comparing Two Intervention Groups, A and B

The use of propensity scores for oncology clinical effectiveness research has become increasingly popular over the last decade, with six articles published in Journal of Clinical Oncology in 2011 alone. 27 – 32 However, propensity score analysis has limitations, the most important of which is that it can only include the variables that are in the available data. If a factor that influences the intervention assignment is not included or measured accurately in the data, it cannot be adequately addressed by a propensity score. For example, in a prior propensity score analysis of the association between active treatment and prostate cancer mortality among elderly men, we were able to include only the variables available in Surveillance, Epidemiology, and End Results–Medicare linked data in our propensity score. 33 The data included some of the factors that influence treatment decisions (eg, age, comorbidities, tumor grade, and size) but not others (eg, functional status, prostate-specific antigen score). Furthermore, the measurement of some of the available factors was imperfect—for example, assessment of comorbidities was based on billing codes, which can underestimate actual comorbidity burden and provide no information about the severity of the comorbidity. Thus, although the final result demonstrating a fairly strong association between active treatment and reduced mortality was quite robust based on the data that were available, it is still possible that the association represents unaddressed selection factors where healthier men underwent active treatment. 34

Instrumental variable methods are a third analytic approach that estimate the effect of an intervention in observational data without requiring the factors that differ between the intervention groups to be available in the data, thereby addressing both measured and unmeasured confounders. 35 The goal underlying instrumental variable analysis is to identify a characteristic (called the instrument) that strongly influences the assignment of patients to intervention but is not associated with the outcomes of interest (except through the intervention). In essence, an instrumental variable approach is an attempt to replicate an RCT, where the instrument is randomization. 36 Common instruments include the patterns of treatment across geographic areas or health care providers, the distance to a health care facility able to provide the intervention of interest, or structural characteristics of the health care system that influence what interventions are used, such as the density of certain types of providers or facilities. The analysis involves two stages: first, the probability of receiving the intervention of interest is estimated as a function of the instrument variable and other covariates; second, a model is built predicting the outcome of interest based on the instrument-based intervention probability and the residual from the first model.

Instrumental variable analysis is commonly used in economics 37 and has increasingly been applied to health and health care. In oncology, instrumental variable approaches have been used to examine the effectiveness of treatments for lung, prostate, bladder, and breast cancers, with the most common instruments being area-level treatment patterns. 38 – 42 One recent analysis of prostate cancer treatment found that multivariable regression and propensity score methods resulted in essentially the same estimate of effect for radical prostatectomy, but an instrumental variable based on the treatment pattern of the previous year found no benefit from radical prostatectomy, similar to the estimate from a recently published trial. 41 , 43 However, concerns also exist about the validity of instrumental variable results, particularly if the instrument is not strongly associated with the intervention, or if there are other potential pathways by which the instrument may influence the outcome. Although the strength of the association between the instrument and the intervention assignment can be tested in the analysis, alternative pathways by which the instrument may be associated with the outcome are often not identified until after publication. A recent instrumental variable analysis used annual rainfall as the instrument to demonstrate an association between television watching and autism, arguing that annual rainfall is associated with the amount of time children watch television but is not otherwise associated with the risk of autism. 44 The findings generated considerable controversy after publication, with the identification of several other potential links between rainfall and autism. 45 Instrumental variable methods have traditionally been unable to examine differences in effect between patient subgroups, but new approaches may improve their utility in this important component of CER. 46 , 47

SYSTEMATIC REVIEWS

For some decisions faced by clinicians and policy makers, there is insufficient evidence to inform decision making, and new studies to generate evidence are needed. However, for other decisions, evidence exists but is sufficiently complex or controversial that it must be synthesized to inform decision making. Systematic reviews are an important form of evidence synthesis that brings together the available evidence using an organized and evaluative approach. 48 Systematic reviews are frequently used for guideline development and generally include four major steps. 49 First, the clinical decision is identified, and the analytic framework and key questions are determined. Sometimes the decision may be straightforward and involve a single key question (eg, Does drug A reduce the incidence of disease B?), but other times the question may be more complicated (eg, Should gene expression profiling be used in early-stage breast cancer?) and involve multiple key questions. 50 Second, the literature is searched to identify the relevant studies using inclusion and exclusion criteria that may include the timing of the study, the study design, and the location of the study. Third, the identified studies are graded on quality using established criteria such as the CONSORT criteria for RCTs 51 and the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) criteria for observational studies. 52 Studies that do not meet a minimum quality threshold may be excluded because of concern about the validity of the results. Fourth, the results of all the studies are collated in evidence tables, often including key characteristics of the study design or population that might influence the results. Meta-analytic techniques may be used to combine results across studies when there is sufficient homogeneity to make a single-point estimate statistically valid. Alternatively, models may be used to identify the study or population factors that are associated with different results.

Although systematic reviews are a key component of evidence-based medicine, their role in CER is still uncertain. The traditional approach to systematic reviews has often excluded observational studies because of concerns about internal validity, but such exclusions may greatly limit the evidence available for many important comparative effectiveness questions. CER is designed to inform real-world decisions between available alternatives, which may include multiple tradeoffs. Inclusion of information about harms in comparative effectiveness systematic reviews is desirable but often challenging because of limited data. Finally, systematic reviews are rarely able to examine differences in intervention effects across patient characteristics, another important step for achieving the goals of CER.

DECISION AND COST-EFFECTIVENESS ANALYSIS

Another evidence synthesis method that is gaining increasing traction in CER is decision modeling. Decision modeling is a quantitative approach to evidence synthesis that brings together data from a range of sources to estimate expected outcomes of different interventions. 53 The first step in a decision model is to lay out the structure of the decision, including the alternative choices and the clinical and economic outcomes of those alternatives. 54 Ensuring that the structure of the model holds true to the clinical scenario of interest without becoming overwhelmed by minor possible variations is critical for the eventual impact of the model. 55 Once the decision structure is determined, a decision tree or simulation model is created that incorporates the probabilities of different outcomes over time and the change in those probabilities from the use of different interventions. 56 , 57 To calculate the expected outcomes, a hypothetic cohort of patients is run through each of the decision alternatives in the model. Estimated outcomes are generally assessed as a count of events in the cohort (eg, deaths, cancers) or as the mean or median life expectancy among the cohort. 58

Decision models can also include information about the value placed on each of the outcomes (often referred to as utility) as well as the health care costs incurred by the interventions and the health outcomes. A decision model that includes cost and utility is often referred to as a cost-benefit or cost-effectiveness model and is used in some settings to compare value across interventions. The types of costs that are included depend on the perspective of the model, with a model from the societal perspective including both direct and indirect medical costs (eg, loss of productivity), a model from a payer (ie, insurer) perspective including only direct medical costs, and a model from a patient perspective including the costs experienced by the patient. Future costs are discounted to address the change in monetary value over time. 59 Sensitivity analyses are used to explore the impact of different assumptions on the model results, a critical step for understanding how the results should be used in clinical and policy decisions and for the development of future evidence-generation research. These sensitivity analyses often use a probabilistic approach, where a distribution is entered for each of the inputs and the computer samples from those distributions across a large number of simulations, thereby creating a confidence interval around the estimated outcomes of the alternative choices.

Decision models have several strengths in CER. They can link multiple sources of information to estimate the effect of different interventions on health outcomes, even when there are no studies that directly assess the effect of interest. Because they can examine the effect of variation in different probability estimates, they are particularly useful for understanding how patient characteristics will affect the expected outcomes of different interventions. Decision models can also estimate the impact of an intervention across a population, including the effect on economic outcomes. Decision and cost-effectiveness analyses have been used frequently in oncology, particularly for decisions with options that include the use of a diagnostic or screening test (eg, bone mineral density testing for management of osteoporosis risk), 60 involve significant tradeoffs (eg, adjuvant chemotherapy), 61 or have only limited empirical evidence (eg, management strategies in BRCA mutation carriers). 62

However, decision models also have several limitations that have limited their impact on clinical and policy decision making in the United States to date and are likely to constrain their role in future CER. Often, model results are highly sensitive to the assumptions of the model, and removing bias from these assumptions is difficult. The potential impact of conflicts of interest is high. Decision models require data inputs. For many decisions, data are insufficient for key inputs, requiring the use of educated guesses (ie, expert opinion). The measurement of utility has proven particularly challenging and can lead to counterintuitive results. In the end, decision analysis is similar to other comparative effectiveness methods—useful for the right question as long as results are interpreted with an understanding of the methodologic limitations.

SELECTION OF CER METHODS

The choice of method for a comparative effectiveness study involves the consideration of multiple factors. The Patient-Centered Outcomes Research Institute Methods Committee has identified five intrinsic and three extrinsic factors ( Table 2 ), including internal validity, generalizability, and variation across patient subgroups as well as the feasibility and time urgency. 63 The importance of these factors will vary across the questions being considered. For some questions, the concern about selection bias will be too great for observational studies, particularly if a strong instrument cannot be identified. Many questions about aggressive versus less aggressive treatments may fall into this category, because the decision is often correlated with patient characteristics that predict survival but are rarely found in observational data sets (eg, functional status, social support). For other questions, concern about selection bias will be less pressing than the need for rapid and efficient results. This scenario may be particularly relevant for the comparison of existing therapies that differ in cost or adverse outcomes, where the use of the therapy is largely driven by practice style. In many cases, the choice will be pragmatic based on what data are available and the feasibility of conducting an RCT. These choices will increasingly be informed by the value of information methods 64 – 66 that use economic modeling to provide guidance about where and how investment in CER should be made.

Factors That Influence Selection of Study Design for Patient-Centered Outcome Research

In reality, the questions of CER are not new but are simply more important than ever. Nearly 50 years ago, Sir Austin Bradford Hill spoke about the importance of a broad portfolio of methods in clinical research, saying “To-day … there are many drugs that work and work potently. We want to know whether this one is more potent than that, what dose is right and proper, for what kind of patient.” 7 (p109) This call has expanded beyond drugs to become the charge for CER. To fulfill this charge, investigators will need to use a range of methods, extending the experience in effectiveness research of the last decades “to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.” 1 (p29)

Supported by Award No. UC2CA148310 from the National Cancer Institute.

The content is solely the responsibility of the author and does not necessarily represent the official views of the National Cancer Institute or the National Institutes of Health.

Author's disclosures of potential conflicts of interest and author contributions are found at the end of this article.

AUTHOR'S DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The author(s) indicated no potential conflicts of interest.

• Search Menu
• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Urban Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Papyrology
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Evolution
• Language Reference
• Language Acquisition
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Media
• Music and Religion
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Science
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Lifestyle, Home, and Garden
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Neuroanaesthesia
• Clinical Neuroscience
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Ethics
• Business Strategy
• Business History
• Business and Technology
• Business and Government
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic History
• Economic Systems
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Natural Disasters (Environment)
• Social Impact of Environmental Issues (Social Science)
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• International Political Economy
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Behaviour
• Political Economy
• Political Institutions
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Theory
• Politics and Law
• Public Policy
• Public Administration
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Developmental and Physical Disabilities Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

9 Comparative Method of Legal Research: Nature, Process, and Potentiality

• Published: January 2020
• Cite Icon Cite
• Permissions Icon Permissions

The relevance of comparative legal research (CLR) consists in comparative evaluation of human experience occurring in legal systems of different jurisdictions. It involves a logical and inductive method of reasoning. It is valuable as it brings out the advantages and disadvantages of different approaches, procedures, and institutions. It has a long history and strong tendency of growth in contemporary times. Appropriate choice of comparative components and choice of the criterion of comparison become crucial for the success of comparative legal research. Functionalist study and cultural immersion give social dimension to the comparison. In the context of multicultural society and march of globalization, CLR has attained great scope and potentiality. Its limitations include inadequate understanding of the social background of other jurisdiction, language barrier, and so on. Its attainments are many in various fields of law.

Signed in as

Institutional accounts.

• Google Scholar Indexing
• GoogleCrawler [DO NOT DELETE]

Personal account

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code

Institutional access

• Sign in with a library card Sign in with username/password Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Sign in through your institution

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Sign in with a library card

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Our books are available by subscription or purchase to libraries and institutions.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Encyclopedia of Behavioral Medicine pp 526–528 Cite as

Comparative Effectiveness Research

• J. Rick Turner 2  
• Reference work entry
• First Online: 20 October 2020

57 Accesses

CER ; Comparative effectiveness methodology

The definition of Comparative Effectiveness Research (CER) for the Federal Coordinating Council reads as follows ( HHS.gov ):

Comparative effectiveness research is the conduct and synthesis of systematic research comparing different interventions and strategies to prevent, diagnose, treat, and monitor health conditions. The purpose of this research is to inform patients, providers, and decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances. To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations. Defined interventions compared may include medications, procedures, medical and assistive devices and technologies, behavioral change strategies, and delivery system interventions. This research necessitates the development, expansion, and use...

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

References and Further Reading

Blumenthal, J. A. (2011). New frontiers in cardiovascular behavioral medicine: comparative effectiveness of exercise and medication in treating depression. Cleveland Clinical Journal of Medicine, 78 (Suppl. 1), S35–S43.

Article   Google Scholar  

Blumenthal, J. A., Califf, R., Williams, R. S., & Hindman, M. (1983). Cardiac rehabilitation: A new frontier for behavioral medicine. Journal of Cardiac Rehabilitation, 3 , 637–656.

Google Scholar  

Bonham, A. C., & Solomon, M. Z. (2010). Moving comparative effectiveness research into practice: Implementation science and the role of academic medicine. Health Affairs (Millwood), 29 , 1901–1905.

Concato, J., Lawler, E. V., Lew, R. A., Gaziano, J. M., Aslan, M., & Huang, G. D. (2010). Observational methods in comparative effectiveness research. American Journal of Medicine, 123 (12 Suppl. 1), e16–e23.

Article   PubMed   Google Scholar  

Hoffman, B., Babyak, M., Craighead, W. E., Sherwood, A., Doraiswamy, P. M., Coons, M. J., et al. (2010). Exercise and pharmacotherapy in patients with major depression: One-year follow-up of the SMILE study. Psychosomatic Medicine, 73 , 127–133.

Article   PubMed   PubMed Central   Google Scholar  

Huang, G. D., Ferguson, R. E., Peduzzi, P. N., & O’Leary, T. J. (2010). Scientific and organizational collaboration in comparative effectiveness research: The VA cooperative studies program model. American Journal of Medicine, 123 (12 Suppl. 1), e24–e31.

Institute of Medicine. (2008). In J. Eden, B. Wheatley, B. McNeil, & H. Sox (Eds.), Knowing what works in health care: A roadmap for the nation . Washington, DC: National Academies Press.

Kupersmith, J., & Ommaya, A. K. (2010). The past, present, and future of comparative effectiveness research in the US Department of Veterans Affairs. American Journal of Medicine, 123 (12. Suppl 1), e3–e7.

O’Connell, J. M., & Griffin, S. (2011). Overview of methods in economic analyses of behavioral interventions to promote oral health. Journal of Public Health Dentistry, 71 (Suppl. 1), S101–S118.

Rich, E. C., Bonham, A. C., & Kirch, D. G. (2011). The implications of comparative effectiveness research for academic medicine. Academic Medicine, 86 , 684–688.

Sox, H. C., & Greenfield, S. (2009). Comparative effectiveness research: A report from the institute of medicine. Annals of Internal Medicine, 151 , 203–205.

United States Health and Human Services. HHS.gov.. Accessed December 14th, 2011, from http://www.hhs.gov/recovery/programs/cer/draftdefinition.html .

Wilensky, G. R. (2006). Developing a center for comparative effectiveness information. Health Affairs (Millwood), 25 , w572–w585.

Download references

Author information

Authors and affiliations.

Campbell University College of Pharmacy and Health Sciences, Buies Creek, NC, USA

J. Rick Turner

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to J. Rick Turner .

Editor information

Editors and affiliations.

Behavioral Medicine Research Center, Department of Psychology, University of Miami, Miami, FL, USA

Marc D. Gellman

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this entry

Cite this entry.

Turner, J.R. (2020). Comparative Effectiveness Research. In: Gellman, M.D. (eds) Encyclopedia of Behavioral Medicine. Springer, Cham. https://doi.org/10.1007/978-3-030-39903-0_1001

Download citation

DOI : https://doi.org/10.1007/978-3-030-39903-0_1001

Published : 20 October 2020

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-39901-6

Online ISBN : 978-3-030-39903-0

eBook Packages : Medicine Reference Module Medicine

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• From Our Archives

Comparative Politics: Method and Research

In the early 1950s, the nascent political science subfield of comparative politics wrestled with questions of method and whether to approach comparing nation-states via abstract concepts or a problem-oriented focus. To begin addressing these concerns, the SSRC convened an interuniversity research seminar in which political scientists began to create a framework for the field that ultimately led to the formation of the Committee on Comparative Politics. Roy Macridis , in this report, summarizes the seminar’s discussion, which included the relative merits of area studies approaches to more abstract theorizing. The conversation clearly tilted toward starting with conceptual schemes independent of context, and so exemplifies the impact of behavioralism that Michael Desch illustrates in his Items essay .

The study of comparative politics has been primarily concerned thus far with the formal institutions of foreign governments, particularly of Western Europe. In this sense it has been not only limited but also primarily descriptive and formalistic. Its place in the field of political science has been ill-defined. Is the student of comparative politics properly concerned primarily with description of the formal institutions of various polities, or with undertaking comparisons? If with the latter, what is the meaning of comparison? Is it confined to the description of differences in various institutional arrangements? If comparison is to be something more than description of formal institutional differences, what are its aims, scope, and methods? Should the student of comparative politics attempt to compare total configurations? If not, then he has to develop a precise notion of what can be isolated from the total configuration of a system or systems and compared.

The above questions illustrate the difficulties and the challenge confronting the student of comparative politics. The problem of comparative method revolves around the discovery of uniformities, and the examination of variables in the context of uniformities between various systems. But even so, what are the particular clusters of states that have a degree of uniformity that makes the comparison and understanding of variations possible? Does the concept of “area,” as used, provide us with such a hunting ground for the study of “difference,” against a background of “uniformity”? Is the concept of culture a more acceptable one? Or does the similarity of social and economic contextual elements provide a better opportunity to compare and understand variations? What degree of homogeneity is required for comparison, and are comparisons between systems not showing the desired degree of homogeneity impossible?

The members of the seminar agreed that it would be extremely ambitious to attempt to answer these methodological questions in detail and at the same time indulge in empirical investigation. It was thought that if we could do some spadework on the methodological issues, we would invite comments and suggestions from the members of the profession and increase awareness of the need for making one’s methodological position explicit prior to undertaking comparative empirical investigation.

Comparability and uniqueness

Questions of the purpose and nature of comparison arose repeatedly, and the members of the seminar devoted considerable time to discussing them. It was agreed that the general problem of comparison is extraordinarily difficult for political science since it is unlikely that we will ever find two or more societies that are identical in all respects except for a single, variable factor. Consequently, the possibility of comparing one variable or even a set of variables against identical conditions is illusory. The alternative would seem to be comparison—at different levels of abstraction and complexity—of wider or narrower segments of the political process.

The following tentative classification of levels of comparative analysis was suggested: (1) comparison of a single problem limited to political systems that are homogeneous in character and operation; (2) comparison of several elements or clusters of elements in relation to political systems that are fairly homogeneous; (3) comparison of institutions or segments of the political process irrespective of “homogeneity”; (4) comparison of political systems as such. These four levels of comparison require increasingly higher levels of abstraction. At the fourth level, some such approach as that of “ideal types” would seem to be called for.

In general, two points of view were expressed throughout the seminar discussions. The first saw the need of a conceptual scheme that not only precisely defines the categories under which data may be collected, but also indicates the criteria of relevance to be adopted and the variables that are to be related hypothetically for the purpose of comparative study. According to the other view, given the present state of comparative studies, comparability ought to be derived primarily from the formulation of problems with limited and manageable proportions. This disagreement should not obscure the area of agreement reached by the members of the seminar.

For example, it was agreed: (1) Comparison involves abstraction, and concrete situations or processes can never be compared as such. To compare means to select certain types of concepts, and in selection we have to “distort” the unique and the concrete. (2) Prior to any comparison it is necessary not only to establish categories and concepts, but also to determine criteria of relevance of the particular components of a social and political situation to the problem under analysis, e.g., relevance of social stratification to family system, or of sun spots to political instability. (3) It is necessary to establish criteria for the adequate representation of the particular components that enter into a general analysis of a problem. (4) The formulation of hypothetical relations and their investigation with empirical data can never lead to proof. A hypothesis or a series of hypothetical relations would be considered verified only as long as not falsified. (5) Hypothetical relations rather than single hypotheses should be formulated. The connecting link between general hypothetical series and particular social relations should be provided by specifying the conditions under which any or all the possibilities enumerated in the series are expected to take place. (6) Finally, one of the greatest dangers in hypothesizing is the projection of possible relationships ad infinitum . This can be avoided by the orderly collection of data prior to hypothesizing. Such collection may in itself lead us to recognize irrelevant relations (climate and the electoral system, language and industrial technology, etc.).

The members of the seminar, therefore, substantially rejected the arguments in favor of uniqueness, and argued that comparison between institutions not only is possible but may eventually provide, through a multiple approach, a general theory of politics and a general theory of political change. Before this development can be realized, the following research approaches should be emphasized and undertaken in as orderly a way as possible: (1) elaboration of a tentative and even rough classificatory scheme or schemes; (2) conceptualization at various levels of abstraction, preferably at the manageable level of the problem-oriented approach; (3) formulation of single hypotheses or hypothetical series that may be suggested by the formulation of either a classificatory scheme or sets of problems; (4) constant reference of hypotheses to empirical data for the purpose of falsification and the formulation of new hypotheses.

Approaches to the comparative study of political systems

Conceptual schemes . The seminar members believed that formulation of a conceptual scheme for the comparative study of politics would help provide a classificatory table and permit the elaboration of hypotheses. Comparison, it was agreed, must proceed from a definition of politics as a universally discoverable social activity. The function of politics is to provide society with social decisions having the force and status of legitimacy. A social decision has the “force of legitimacy” if the collective regularized power of the society is brought to bear against deviations, and if there is a predominant disposition among those subject to the decision to comply. As for the means of enforcing decisions, every society, generally speaking, has a determinate organization that enjoys a monopoly of legitimate authority (or political ultimacy). Moreover, the characteristic distinguishing between political relationships and other relationships is the existence of this framework of legitimacy. Conceptions of legitimacy or “legitimacy myths” are highly varied ways in which people justify coercion and conformity, as well as the ways by which a society rationalizes its ascription of political ultimacy, and the beliefs that account for a predisposition to compliance with social decisions.

But the legitimacy myth only defines the conditions of obedience. Within its framework there is the political process itself, through which numerous groups having political aspirations (policy-aspiration groups and power-aspiration groups) strive for recognition and elevation to the position of legitimacy. The factors that determine which power-aspiration group is to be invested with legitimacy, to the exclusion of all others, are the effective power factors in the system.

It was suggested, then, that the general modes of politics, for the purpose of analysis, would be as follows: Political processes are the struggle among power-aspiration and policy-aspiration groups competing for the status of legitimacy; the outcome is determined by the society’s structure of effective power, and the end state, legitimacy, is the political reflection of its general value system.

The major components of the political process, which should provide a fairly coherent classification scheme as well as the possibility of formulating hypothetical relations, are the following: the “elective” process of the system, its “formal” deliberative process, its “informal” deliberative process, its structure of “influence,” and its structure of “power.” The major tasks, envisioned under this scheme, in the analysis of political systems are: (1) to analyze the legitimacy myth of the society in terms of specific content and relationship to the society’s general myth structure; (2) to inquire into the system’s political aspirations, political processes, and effective power factors; (3) to analyze both the complexity and ultimacy of decision-making systems in the society, specifically the conditions under which political ultimacy is either diffused or concentrated, and the relationships between subsidiary and ultimate decision-making systems; (4) to provide for a theory of change through the study of “formal” and “informal” processes.

Alternative approaches . The general agreement on the usefulness of a conceptual scheme was coupled with an equally strong emphasis on the need for alternative approaches. It was thought that the present state of comparative politics calls for a “pluralistic” rather than a unitary approach, and that for each of the alternative approaches suggested, the same degree of methodological rigor should be followed as in the development of a conceptual scheme. The alternative approaches agreed upon were: the problem approach, the elaboration of a classificatory scheme or checklist to aid in more coherent and more systematic compilation of data, and the area approach.

The problem approach

The study of comparative politics cannot wait for the development of a comprehensive conceptual scheme. Instead of aiming toward universality it may be advisable to adopt a more modest approach. The members of the seminar agreed that the “problem approach” is a step in this direction. It was pointed out that the formulation of a problem in itself has some of the characteristics of a conceptual scheme. It directs research toward various aspects of the political process and at the same time calls for an ordering of empirical data and the formulation of hypotheses or series of hypothetical relations. Furthermore, this approach is flexible enough so that it can lead the research worker to examination of questions that have a varying degree of comprehensiveness in terms of both theory and empirical orientation and investigation.

Three types of problem approaches were suggested and discussed: (1) Narrow-range theory, which involves a relatively low degree of abstraction; i.e., it applies to homogeneous cultural contexts and deals with a limited number of variables. (2) Middle-range theory, which is conceived to include problems of fairly general importance, involving a relatively high degree of generalization, but remaining below the level of a truly general theory of politics. (3) Policy-oriented theory, which deals with the immediate practical solution of important problems and is consequently focused on problems originating in pressing conflict situations or in an overwhelming need for policy action.

Four criteria by which to select problems were suggested: the intrinsic interest of the problem to political scientists; its ability to eliminate certain key difficulties in the comparative method and the analytical utility of comparison; its capacity for advancing research beyond the current level of inquiry in the field; its probable and eventual significance for the formation of a general theory of comparative politics.

It was agreed that the formulation of the problem should be as clear and logically coherent as possible, and that it should be presented in the following form:

a. The problem must be stated precisely; it must be stated in such a form as to lead immediately to hypotheses; it must be analyzed into its component elements; its variables and the relations between them must be spelled out; and all this must be done in operationally meaningful terms. b. Its relations to a possible general theory of politics must be described. How would the problem fit into a more general theoretical orientation and what more general questions can its solution illuminate? c. The manner in which the problem calls for the use of comparative method must be demonstrated, and the level of abstraction that comparison would involve must be analyzed. d. Outline a recommended research technique for dealing with the problem and justify the recommendation. e. Enumerate possible alternative research techniques.

It was not considered the function of the seminar to state exhaustively narrow-range, middle-range, and policy-oriented problems, but a few typical ones were suggested for the purpose of illustration: An analysis of the relations between the power of dissolution and ministerial stability in parliamentary systems would fall in the realm of narrow-range theory. A study of the political consequences of rapid industrialization in underdeveloped areas of the world would be in the realm of middle-range theory. The following are policy-oriented problems: the development of constitutional government in colonial areas; how to deal with political instability in France; how to dissociate colonial nationalism from Soviet-inspired leadership and ideology; determination of policies of constitutional regimes toward totalitarian parties, e.g., the Communist Party.

A classificatory scheme

The seminar concluded that a checklist might facilitate assembling data in an orderly fashion under commonly formulated concepts. It decided to develop such a checklist for purposes of illustration. The following broad categories and subdivisions were specified:

(1) The setting of politics: an enumeration of the most significant contextual factors of all political systems, i.e., geographic patterning, economic structure, transportation and communication patterns, sociological structure and minorities, cultural patterns, values and value systems, and the record of social change.

(2) The sphere of politics: the actual and potential sphere of political decisions: conditions determining the sphere of decision making, limits on political decisions, major types of decision making, and potential changes in the sphere of decision making.

(3) Who makes decisions: Who are the “elite” supposed to be? To whom does the community impute prestige and what are the prevalent prestige images and symbols? Who actually makes the effective political decisions if they are not made by those who are supposed to make them?

(4) How decisions are made: formulation of problems, agencies and channels of decision making, some major characteristics of decision-making procedure.

(5) Why are decisions obeyed: the enforcement of decisions, compliance, consent, types of consent, ecology of compliance, measurement of compliance.

(6) Practical politics: types, purpose, organization, and techniques of policy-aspiration groups; types, goals, organization, techniques, and influence and effectiveness of power-aspiration groups.

(7) The performance of the system: stability, adjustment, and change, and their conditions, relationship between formal and informal processes, manifestations of instability and stability.

Area study and comparative politics

The third alternative approach to the study of comparative politics is the more systematic use of the area concept. However, neither geographic, historical, economic, nor cultural similarities constitute prima facie evidence of the existence of similar political characteristics. If the concept of an area is to be operationally meaningful for purposes of comparison, it should correspond to some uniform political patterns against which differences may be studied comparatively and explained.

The definition of an area on the basis of culture was considered to be worth detailed discussion. It was suggested that although primarily used by the anthropologists it might be adapted to the needs of political scientists.

To make the concept operationally more meaningful for political science, it was suggested that an attempt be made to define the area concept with reference to “political traits” or “trait complexes” or “problem configuration patterns,” in terms analogous to those used by anthropologists when they analyze the concept of culture in terms of “traits” or “trait complexes.” Such an approach to the definition of an area has not yet been undertaken despite its promise for comparative study. The very search for common political traits and patterns will call for classification and conceptualization. Once similar traits or patterns have been distinguished and have been related to geographically delimited units, the area concept will be of great value since certain political processes can be compared within the area against a common background of similar trait configuration. In this sense it was thought that future research should be directed toward developing in great detail classificatory schemes within areas.

International relations

The last topic considered was the relationship between comparative politics and international politics. It was pointed out that contemporary study of international politics has entered a new stage. Since the national interest is now a central concept of international politics, what tests or criteria are to be used in identifying the interests that shape the foreign policy of any state? Is the student of international politics to accept, at face value, the definition of national interests given by statesmen? Or does the examination of power factors, geography, historical development, etc. offer the student of international politics certain rough tests for defining the “objective” interests of the state?

Assuming that the concept of national interest provides a focal point for investigation, two approaches seem possible, either separately or in combination: (1) To determine analytically what the national interest “ought to be” under certain conditions. One could then compare this evaluation of the national interest with its “actual” definition as provided by the actions and pronouncements of the particular nation. (2) To describe, in terms of the following categories, the reasons why nations define their national interest in certain terms and not in others: survival prerequisites; objective physical conditions—geography, natural resources, tradition, past decisions, value systems, etc.; institutional channels through which the national interest is defined and set—i.e., interest programs and images and how they are defined in the political process; policy interplay between independent political units and their respective interests, including the pattern of reaction within each; the subjective pluralism of the society, by which the content of the interest images is set. The first four categories are in the nature of factors that set limits on the definition of the national interest. The last category defines the possibilities in substantive terms. On the basis of this scheme, it was suggested, there might be a useful “division of labor” or cooperation between students of international politics and of comparative politics. The second and fourth categories could best be handled by students of international politics, the others by specialists in comparative government.

More generally, however, a cooperative effort between students of international politics and comparative politics should be centered in the following areas of mutual interest: (1) The process of decision making has become a function in international politics through existing organizations. How does this decision-making take place? Is it accompanied by any broad legitimacy ideas or myths that transcend the national states? (2) The concept of national interest provides for meaningful concepts for the study of foreign policy. The concept will have to be broken down into component parts, some of which would be studied by students of comparative politics, while others remain within the domain of international politics. Given analogous conditions, generally speaking, the definition of national interest varies by individual states. Determination of what accounts for this variation seems to be the proper task of the specialist in comparative politics. (3) Study of the focal point at which the states meet—diplomacy and negotiations through which conflicts are resolved or common objectives realized—is a cooperative task that admits no arbitrary allocation of duties, for the action of each state depends upon domestic conditions, internal images, and traditional forces. Relations between states, on the other hand, have repercussions on domestic myths, images, authority symbols, and institutions. Domestic and international politics are in this sense complementary factors. (4) The student of international politics may also join with the student of comparative politics in attempting to define and study an area. Certain uniform outlooks and behavior patterns may be due to similar experiences shared by a number of states that can be geographically identified. The study of such uniformities and differences is primarily a joint task. (5) Finally, goals of foreign policy may be jointly studied. In the context of national interest, foreign policy is to be conceived as a dynamic interplay of the given or chosen goals, the organic elements that set limiting conditions on the selection of goals, and the selection of strategy or means for the achievement of the given or chosen goals.

This article is a summary of a longer report on the proceedings of the interuniversity summer research seminar on comparative politics held at Northwestern University during July and August 1952. Plans for the seminar were described briefly in the March 1952 issue of Items , p. 7. While the author is responsible for the statements appearing here, they summarize a collective product and at times reproduce lines of thought expressed originally by the participants both during the meetings and in their respective essays. Members of the seminar were: Samuel H. Beer and Harry Eckstein, Harvard University; George I. Blanksten and Roy C. Macridis, Northwestern University; Karl W. Deutsch, Massachusetts Institute of Technology; Kenneth W. Thompson, University of Chicago; and Robert E. Ward, University of Michigan. Richard Cox of the University of Chicago acted as rapporteur.

Roy C. Macridis (1918–1991) taught political science at Brandeis University since 1965. During the Second World War, Macridis served with the Office of Strategic Services and migrated to the United States in 1944. He was a member of the Council’s Committee on Comparative Politics from 1954 to 1958.

This essay originally appeared in Items Vol. 6, No. 4 in December of 1952. Visit our archives to view the original as it first appeared in the print editions of Items .

You may also like

Democratic politics in africa and the caribbean, postcolonial urban apartheid, the conference on problems of area research in contemporary africa, bringing the state back in: a report on current comparative research on the relationship between states and social structures.