research on observational studies

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is an Observational Study? | Guide & Examples

What Is an Observational Study? | Guide & Examples

Published on March 31, 2022 by Tegan George . Revised on June 22, 2023.

An observational study is used to answer a research question based purely on what the researcher observes. There is no interference or manipulation of the research subjects, and no control and treatment groups .

These studies are often qualitative in nature and can be used for both exploratory and explanatory research purposes. While quantitative observational studies exist, they are less common.

Observational studies are generally used in hard science, medical, and social science fields. This is often due to ethical or practical concerns that prevent the researcher from conducting a traditional experiment . However, the lack of control and treatment groups means that forming inferences is difficult, and there is a risk of confounding variables and observer bias impacting your analysis.

Table of contents

Types of observation, types of observational studies, observational study example, advantages and disadvantages of observational studies, observational study vs. experiment, other interesting articles, frequently asked questions.

There are many types of observation, and it can be challenging to tell the difference between them. Here are some of the most common types to help you choose the best one for your observational study.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

There are three main types of observational studies: cohort studies, case–control studies, and cross-sectional studies .

Cohort studies

Cohort studies are more longitudinal in nature, as they follow a group of participants over a period of time. Members of the cohort are selected because of a shared characteristic, such as smoking, and they are often observed over a period of years.

Case–control studies

Case–control studies bring together two groups, a case study group and a control group . The case study group has a particular attribute while the control group does not. The two groups are then compared, to see if the case group exhibits a particular characteristic more than the control group.

For example, if you compared smokers (the case study group) with non-smokers (the control group), you could observe whether the smokers had more instances of lung disease than the non-smokers.

Cross-sectional studies

Cross-sectional studies analyze a population of study at a specific point in time.

This often involves narrowing previously collected data to one point in time to test the prevalence of a theory—for example, analyzing how many people were diagnosed with lung disease in March of a given year. It can also be a one-time observation, such as spending one day in the lung disease wing of a hospital.

Observational studies are usually quite straightforward to design and conduct. Sometimes all you need is a notebook and pen! As you design your study, you can follow these steps.

Step 1: Identify your research topic and objectives

The first step is to determine what you’re interested in observing and why. Observational studies are a great fit if you are unable to do an experiment for practical or ethical reasons , or if your research topic hinges on natural behaviors.

Step 2: Choose your observation type and technique

In terms of technique, there are a few things to consider:

• Are you determining what you want to observe beforehand, or going in open-minded?
• Is there another research method that would make sense in tandem with an observational study?
• If yes, make sure you conduct a covert observation.
• If not, think about whether observing from afar or actively participating in your observation is a better fit.
• How can you preempt confounding variables that could impact your analysis?
• You could observe the children playing at the playground in a naturalistic observation.
• You could spend a month at a day care in your town conducting participant observation, immersing yourself in the day-to-day life of the children.
• You could conduct covert observation behind a wall or glass, where the children can’t see you.

Overall, it is crucial to stay organized. Devise a shorthand for your notes, or perhaps design templates that you can fill in. Since these observations occur in real time, you won’t get a second chance with the same data.

Step 3: Set up your observational study

Before conducting your observations, there are a few things to attend to:

• Plan ahead: If you’re interested in day cares, you’ll need to call a few in your area to plan a visit. They may not all allow observation, or consent from parents may be needed, so give yourself enough time to set everything up.
• Determine your note-taking method: Observational studies often rely on note-taking because other methods, like video or audio recording, run the risk of changing participant behavior.
• Get informed consent from your participants (or their parents) if you want to record:  Ultimately, even though it may make your analysis easier, the challenges posed by recording participants often make pen-and-paper a better choice.

Step 4: Conduct your observation

After you’ve chosen a type of observation, decided on your technique, and chosen a time and place, it’s time to conduct your observation.

Here, you can split them into case and control groups. The children with siblings have a characteristic you are interested in (siblings), while the children in the control group do not.

When conducting observational studies, be very careful of confounding or “lurking” variables. In the example above, you observed children as they were dropped off, gauging whether or not they were upset. However, there are a variety of other factors that could be at play here (e.g., illness).

Step 5: Analyze your data

After you finish your observation, immediately record your initial thoughts and impressions, as well as follow-up questions or any issues you perceived during the observation. If you audio- or video-recorded your observations, you can transcribe them.

Your analysis can take an inductive  or deductive approach :

• If you conducted your observations in a more open-ended way, an inductive approach allows your data to determine your themes.
• If you had specific hypotheses prior to conducting your observations, a deductive approach analyzes whether your data confirm those themes or ideas you had previously.

Next, you can conduct your thematic or content analysis . Due to the open-ended nature of observational studies, the best fit is likely thematic analysis .

Step 6: Discuss avenues for future research

Observational studies are generally exploratory in nature, and they often aren’t strong enough to yield standalone conclusions due to their very high susceptibility to observer bias and confounding variables. For this reason, observational studies can only show association, not causation .

If you are excited about the preliminary conclusions you’ve drawn and wish to proceed with your topic, you may need to change to a different research method , such as an experiment.

• Observational studies can provide information about difficult-to-analyze topics in a low-cost, efficient manner.
• They allow you to study subjects that cannot be randomized safely, efficiently, or ethically .
• They are often quite straightforward to conduct, since you just observe participant behavior as it happens or utilize preexisting data.
• They’re often invaluable in informing later, larger-scale clinical trials or experimental designs.

Disadvantages

• Observational studies struggle to stand on their own as a reliable research method. There is a high risk of observer bias and undetected confounding variables or omitted variables .
• They lack conclusive results, typically are not externally valid or generalizable, and can usually only form a basis for further research.
• They cannot make statements about the safety or efficacy of the intervention or treatment they study, only observe reactions to it. Therefore, they offer less satisfying results than other methods.

The key difference between observational studies and experiments is that a properly conducted observational study will never attempt to influence responses, while experimental designs by definition have some sort of treatment condition applied to a portion of participants.

However, there may be times when it’s impossible, dangerous, or impractical to influence the behavior of your participants. This can be the case in medical studies, where it is unethical or cruel to withhold potentially life-saving intervention, or in longitudinal analyses where you don’t have the ability to follow your group over the course of their lifetime.

An observational study may be the right fit for your research if random assignment of participants to control and treatment groups is impossible or highly difficult. However, the issues observational studies raise in terms of validity , confounding variables, and conclusiveness can mean that an experiment is more reliable.

If you’re able to randomize your participants safely and your research question is definitely causal in nature, consider using an experiment.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

An observational study is a great choice for you if your research question is based purely on observations. If there are ethical, logistical, or practical concerns that prevent you from conducting a traditional experiment , an observational study may be a good choice. In an observational study, there is no interference or manipulation of the research subjects, as well as no control or treatment groups .

The key difference between observational studies and experimental designs is that a well-done observational study does not influence the responses of participants, while experiments do have some sort of treatment condition applied to at least some participants by random assignment .

A quasi-experiment is a type of research design that attempts to establish a cause-and-effect relationship. The main difference with a true experiment is that the groups are not randomly assigned.

Exploratory research aims to explore the main aspects of an under-researched problem, while explanatory research aims to explain the causes and consequences of a well-defined problem.

Experimental design means planning a set of procedures to investigate a relationship between variables . To design a controlled experiment, you need:

• A testable hypothesis
• At least one independent variable that can be precisely manipulated
• At least one dependent variable that can be precisely measured

When designing the experiment, you decide:

• How you will manipulate the variable(s)
• How you will control for any potential confounding variables
• How many subjects or samples will be included in the study
• How subjects will be assigned to treatment levels

Experimental design is essential to the internal and external validity of your experiment.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). What Is an Observational Study? | Guide & Examples. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/methodology/observational-study/

Is this article helpful?

Tegan George

Other students also liked, what is a research design | types, guide & examples, guide to experimental design | overview, steps, & examples, naturalistic observation | definition, guide & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• Research Process

What is Observational Study Design and Types

• 4 minute read
• 108.9K views

Table of Contents

Most people think of a traditional experimental design when they consider research and published research papers. There is, however, a type of research that is more observational in nature, and it is appropriately referred to as “observational studies.”

There are many valuable reasons to utilize an observational study design. But, just as in research experimental design, different methods can be used when you’re considering this type of study. In this article, we’ll look at the advantages and disadvantages of an observational study design, as well as the 3 types of observational studies.

What is Observational Study Design?

An observational study is when researchers are looking at the effect of some type of intervention, risk, a diagnostic test or treatment, without trying to manipulate who is, or who isn’t, exposed to it.

This differs from an experimental study, where the scientists are manipulating who is exposed to the treatment, intervention, etc., by having a control group, or those who are not exposed, and an experimental group, or those who are exposed to the intervention, treatment, etc. In the best studies, the groups are randomized, or chosen by chance.

Any evidence derived from systematic reviews is considered the best in the hierarchy of evidence, which considers which studies are deemed the most reliable. Next would be any evidence that comes from randomized controlled trials. Cohort studies and case studies follow, in that order.

Cohort studies and case studies are considered observational in design, whereas the randomized controlled trial would be an experimental study.

Let’s take a closer look at the different types of observational study design.

The 3 types of Observational Studies

The different types of observational studies are used for different reasons. Selecting the best type for your research is critical to a successful outcome. One of the main reasons observational studies are used is when a randomized experiment would be considered unethical. For example, a life-saving medication used in a public health emergency. They are also used when looking at aetiology, or the cause of a condition or disease, as well as the treatment of rare conditions.

Case Control Observational Study

Researchers in case control studies identify individuals with an existing health issue or condition, or “cases,” along with a similar group without the condition, or “controls.” These two groups are then compared to identify predictors and outcomes. This type of study is helpful to generate a hypothesis that can then be researched.

Cohort Observational Study

This type of observational study is often used to help understand cause and effect. A cohort observational study looks at causes, incidence and prognosis, for example. A cohort is a group of people who are linked in a particular way, for example, a birth cohort would include people who were born within a specific period of time. Scientists might compare what happens to the members of the cohort who have been exposed to some variable to what occurs with members of the cohort who haven’t been exposed.

Cross Sectional Observational Study

Unlike a cohort observational study, a cross sectional observational study does not explore cause and effect, but instead looks at prevalence. Here you would look at data from a particular group at one very specific period of time. Researchers would simply observe and record information about something present in the population, without manipulating any variables or interventions. These types of studies are commonly used in psychology, education and social science.

Advantages and Disadvantages of Observational Study Design

Observational study designs have the distinct advantage of allowing researchers to explore answers to questions where a randomized controlled trial, or RCT, would be unethical. Additionally, if the study is focused on a rare condition, studying existing cases as compared to non-affected individuals might be the most effective way to identify possible causes of the condition. Likewise, if very little is known about a condition or circumstance, a cohort study would be a good study design choice.

A primary advantage to the observational study design is that they can generally be completed quickly and inexpensively. A RCT can take years before the data is compiled and available. RCTs are more complex and involved, requiring many more logistics and details to iron out, whereas an observational study can be more easily designed and completed.

The main disadvantage of observational study designs is that they’re more open to dispute than an RCT. Of particular concern would be confounding biases. This is when a cohort might share other characteristics that affect the outcome versus the outcome stated in the study. An example would be that people who practice good sleeping habits have less heart disease. But, maybe those who practice effective sleeping habits also, in general, eat better and exercise more.

Language Editing Plus Service

Need help with your research writing? With our Language Editing Plus service , we’ll help you improve the flow and writing of your paper, including UNLIMITED editing support. Use the simulator below to check the price for your manuscript, using the total number of words of the document.

Clinical Questions: PICO and PEO Research

Paper Retraction: Meaning and Main Reasons

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Non-Experimental Research

32 Observational Research

Learning objectives.

• List the various types of observational research methods and distinguish between each.
• Describe the strengths and weakness of each observational research method. 

What Is Observational Research?

The term observational research is used to refer to several different types of non-experimental studies in which behavior is systematically observed and recorded. The goal of observational research is to describe a variable or set of variables. More generally, the goal is to obtain a snapshot of specific characteristics of an individual, group, or setting. As described previously, observational research is non-experimental because nothing is manipulated or controlled, and as such we cannot arrive at causal conclusions using this approach. The data that are collected in observational research studies are often qualitative in nature but they may also be quantitative or both (mixed-methods). There are several different types of observational methods that will be described below.

Naturalistic Observation

Naturalistic observation  is an observational method that involves observing people’s behavior in the environment in which it typically occurs. Thus naturalistic observation is a type of field research (as opposed to a type of laboratory research). Jane Goodall’s famous research on chimpanzees is a classic example of naturalistic observation. Dr.  Goodall spent three decades observing chimpanzees in their natural environment in East Africa. She examined such things as chimpanzee’s social structure, mating patterns, gender roles, family structure, and care of offspring by observing them in the wild. However, naturalistic observation  could more simply involve observing shoppers in a grocery store, children on a school playground, or psychiatric inpatients in their wards. Researchers engaged in naturalistic observation usually make their observations as unobtrusively as possible so that participants are not aware that they are being studied. Such an approach is called disguised naturalistic observation .  Ethically, this method is considered to be acceptable if the participants remain anonymous and the behavior occurs in a public setting where people would not normally have an expectation of privacy. Grocery shoppers putting items into their shopping carts, for example, are engaged in public behavior that is easily observable by store employees and other shoppers. For this reason, most researchers would consider it ethically acceptable to observe them for a study. On the other hand, one of the arguments against the ethicality of the naturalistic observation of “bathroom behavior” discussed earlier in the book is that people have a reasonable expectation of privacy even in a public restroom and that this expectation was violated. 

In cases where it is not ethical or practical to conduct disguised naturalistic observation, researchers can conduct  undisguised naturalistic observation where the participants are made aware of the researcher presence and monitoring of their behavior. However, one concern with undisguised naturalistic observation is  reactivity. Reactivity refers to when a measure changes participants’ behavior. In the case of undisguised naturalistic observation, the concern with reactivity is that when people know they are being observed and studied, they may act differently than they normally would. This type of reactivity is known as the Hawthorne effect . For instance, you may act much differently in a bar if you know that someone is observing you and recording your behaviors and this would invalidate the study. So disguised observation is less reactive and therefore can have higher validity because people are not aware that their behaviors are being observed and recorded. However, we now know that people often become used to being observed and with time they begin to behave naturally in the researcher’s presence. In other words, over time people habituate to being observed. Think about reality shows like Big Brother or Survivor where people are constantly being observed and recorded. While they may be on their best behavior at first, in a fairly short amount of time they are flirting, having sex, wearing next to nothing, screaming at each other, and occasionally behaving in ways that are embarrassing.

Participant Observation

Another approach to data collection in observational research is participant observation. In  participant observation , researchers become active participants in the group or situation they are studying. Participant observation is very similar to naturalistic observation in that it involves observing people’s behavior in the environment in which it typically occurs. As with naturalistic observation, the data that are collected can include interviews (usually unstructured), notes based on their observations and interactions, documents, photographs, and other artifacts. The only difference between naturalistic observation and participant observation is that researchers engaged in participant observation become active members of the group or situations they are studying. The basic rationale for participant observation is that there may be important information that is only accessible to, or can be interpreted only by, someone who is an active participant in the group or situation. Like naturalistic observation, participant observation can be either disguised or undisguised. In disguised participant observation , the researchers pretend to be members of the social group they are observing and conceal their true identity as researchers.

In a famous example of disguised participant observation, Leon Festinger and his colleagues infiltrated a doomsday cult known as the Seekers, whose members believed that the apocalypse would occur on December 21, 1954. Interested in studying how members of the group would cope psychologically when the prophecy inevitably failed, they carefully recorded the events and reactions of the cult members in the days before and after the supposed end of the world. Unsurprisingly, the cult members did not give up their belief but instead convinced themselves that it was their faith and efforts that saved the world from destruction. Festinger and his colleagues later published a book about this experience, which they used to illustrate the theory of cognitive dissonance (Festinger, Riecken, & Schachter, 1956) [1] .

In contrast with undisguised participant observation ,  the researchers become a part of the group they are studying and they disclose their true identity as researchers to the group under investigation. Once again there are important ethical issues to consider with disguised participant observation.  First no informed consent can be obtained and second deception is being used. The researcher is deceiving the participants by intentionally withholding information about their motivations for being a part of the social group they are studying. But sometimes disguised participation is the only way to access a protective group (like a cult). Further, disguised participant observation is less prone to reactivity than undisguised participant observation. 

Rosenhan’s study (1973) [2]   of the experience of people in a psychiatric ward would be considered disguised participant observation because Rosenhan and his pseudopatients were admitted into psychiatric hospitals on the pretense of being patients so that they could observe the way that psychiatric patients are treated by staff. The staff and other patients were unaware of their true identities as researchers.

Another example of participant observation comes from a study by sociologist Amy Wilkins on a university-based religious organization that emphasized how happy its members were (Wilkins, 2008) [3] . Wilkins spent 12 months attending and participating in the group’s meetings and social events, and she interviewed several group members. In her study, Wilkins identified several ways in which the group “enforced” happiness—for example, by continually talking about happiness, discouraging the expression of negative emotions, and using happiness as a way to distinguish themselves from other groups.

One of the primary benefits of participant observation is that the researchers are in a much better position to understand the viewpoint and experiences of the people they are studying when they are a part of the social group. The primary limitation with this approach is that the mere presence of the observer could affect the behavior of the people being observed. While this is also a concern with naturalistic observation, additional concerns arise when researchers become active members of the social group they are studying because that they may change the social dynamics and/or influence the behavior of the people they are studying. Similarly, if the researcher acts as a participant observer there can be concerns with biases resulting from developing relationships with the participants. Concretely, the researcher may become less objective resulting in more experimenter bias.

Structured Observation

Another observational method is structured observation . Here the investigator makes careful observations of one or more specific behaviors in a particular setting that is more structured than the settings used in naturalistic or participant observation. Often the setting in which the observations are made is not the natural setting. Instead, the researcher may observe people in the laboratory environment. Alternatively, the researcher may observe people in a natural setting (like a classroom setting) that they have structured some way, for instance by introducing some specific task participants are to engage in or by introducing a specific social situation or manipulation.

Structured observation is very similar to naturalistic observation and participant observation in that in all three cases researchers are observing naturally occurring behavior; however, the emphasis in structured observation is on gathering quantitative rather than qualitative data. Researchers using this approach are interested in a limited set of behaviors. This allows them to quantify the behaviors they are observing. In other words, structured observation is less global than naturalistic or participant observation because the researcher engaged in structured observations is interested in a small number of specific behaviors. Therefore, rather than recording everything that happens, the researcher only focuses on very specific behaviors of interest.

Researchers Robert Levine and Ara Norenzayan used structured observation to study differences in the “pace of life” across countries (Levine & Norenzayan, 1999) [4] . One of their measures involved observing pedestrians in a large city to see how long it took them to walk 60 feet. They found that people in some countries walked reliably faster than people in other countries. For example, people in Canada and Sweden covered 60 feet in just under 13 seconds on average, while people in Brazil and Romania took close to 17 seconds. When structured observation  takes place in the complex and even chaotic “real world,” the questions of when, where, and under what conditions the observations will be made, and who exactly will be observed are important to consider. Levine and Norenzayan described their sampling process as follows:

“Male and female walking speed over a distance of 60 feet was measured in at least two locations in main downtown areas in each city. Measurements were taken during main business hours on clear summer days. All locations were flat, unobstructed, had broad sidewalks, and were sufficiently uncrowded to allow pedestrians to move at potentially maximum speeds. To control for the effects of socializing, only pedestrians walking alone were used. Children, individuals with obvious physical handicaps, and window-shoppers were not timed. Thirty-five men and 35 women were timed in most cities.” (p. 186).

Precise specification of the sampling process in this way makes data collection manageable for the observers, and it also provides some control over important extraneous variables. For example, by making their observations on clear summer days in all countries, Levine and Norenzayan controlled for effects of the weather on people’s walking speeds.  In Levine and Norenzayan’s study, measurement was relatively straightforward. They simply measured out a 60-foot distance along a city sidewalk and then used a stopwatch to time participants as they walked over that distance.

As another example, researchers Robert Kraut and Robert Johnston wanted to study bowlers’ reactions to their shots, both when they were facing the pins and then when they turned toward their companions (Kraut & Johnston, 1979) [5] . But what “reactions” should they observe? Based on previous research and their own pilot testing, Kraut and Johnston created a list of reactions that included “closed smile,” “open smile,” “laugh,” “neutral face,” “look down,” “look away,” and “face cover” (covering one’s face with one’s hands). The observers committed this list to memory and then practiced by coding the reactions of bowlers who had been videotaped. During the actual study, the observers spoke into an audio recorder, describing the reactions they observed. Among the most interesting results of this study was that bowlers rarely smiled while they still faced the pins. They were much more likely to smile after they turned toward their companions, suggesting that smiling is not purely an expression of happiness but also a form of social communication.

In yet another example (this one in a laboratory environment), Dov Cohen and his colleagues had observers rate the emotional reactions of participants who had just been deliberately bumped and insulted by a confederate after they dropped off a completed questionnaire at the end of a hallway. The confederate was posing as someone who worked in the same building and who was frustrated by having to close a file drawer twice in order to permit the participants to walk past them (first to drop off the questionnaire at the end of the hallway and once again on their way back to the room where they believed the study they signed up for was taking place). The two observers were positioned at different ends of the hallway so that they could read the participants’ body language and hear anything they might say. Interestingly, the researchers hypothesized that participants from the southern United States, which is one of several places in the world that has a “culture of honor,” would react with more aggression than participants from the northern United States, a prediction that was in fact supported by the observational data (Cohen, Nisbett, Bowdle, & Schwarz, 1996) [6] .

When the observations require a judgment on the part of the observers—as in the studies by Kraut and Johnston and Cohen and his colleagues—a process referred to as   coding is typically required . Coding generally requires clearly defining a set of target behaviors. The observers then categorize participants individually in terms of which behavior they have engaged in and the number of times they engaged in each behavior. The observers might even record the duration of each behavior. The target behaviors must be defined in such a way that guides different observers to code them in the same way. This difficulty with coding illustrates the issue of interrater reliability, as mentioned in Chapter 4. Researchers are expected to demonstrate the interrater reliability of their coding procedure by having multiple raters code the same behaviors independently and then showing that the different observers are in close agreement. Kraut and Johnston, for example, video recorded a subset of their participants’ reactions and had two observers independently code them. The two observers showed that they agreed on the reactions that were exhibited 97% of the time, indicating good interrater reliability.

One of the primary benefits of structured observation is that it is far more efficient than naturalistic and participant observation. Since the researchers are focused on specific behaviors this reduces time and expense. Also, often times the environment is structured to encourage the behaviors of interest which again means that researchers do not have to invest as much time in waiting for the behaviors of interest to naturally occur. Finally, researchers using this approach can clearly exert greater control over the environment. However, when researchers exert more control over the environment it may make the environment less natural which decreases external validity. It is less clear for instance whether structured observations made in a laboratory environment will generalize to a real world environment. Furthermore, since researchers engaged in structured observation are often not disguised there may be more concerns with reactivity.

Case Studies

A  case study   is an in-depth examination of an individual. Sometimes case studies are also completed on social units (e.g., a cult) and events (e.g., a natural disaster). Most commonly in psychology, however, case studies provide a detailed description and analysis of an individual. Often the individual has a rare or unusual condition or disorder or has damage to a specific region of the brain.

Like many observational research methods, case studies tend to be more qualitative in nature. Case study methods involve an in-depth, and often a longitudinal examination of an individual. Depending on the focus of the case study, individuals may or may not be observed in their natural setting. If the natural setting is not what is of interest, then the individual may be brought into a therapist’s office or a researcher’s lab for study. Also, the bulk of the case study report will focus on in-depth descriptions of the person rather than on statistical analyses. With that said some quantitative data may also be included in the write-up of a case study. For instance, an individual’s depression score may be compared to normative scores or their score before and after treatment may be compared. As with other qualitative methods, a variety of different methods and tools can be used to collect information on the case. For instance, interviews, naturalistic observation, structured observation, psychological testing (e.g., IQ test), and/or physiological measurements (e.g., brain scans) may be used to collect information on the individual.

HM is one of the most notorious case studies in psychology. HM suffered from intractable and very severe epilepsy. A surgeon localized HM’s epilepsy to his medial temporal lobe and in 1953 he removed large sections of his hippocampus in an attempt to stop the seizures. The treatment was a success, in that it resolved his epilepsy and his IQ and personality were unaffected. However, the doctors soon realized that HM exhibited a strange form of amnesia, called anterograde amnesia. HM was able to carry out a conversation and he could remember short strings of letters, digits, and words. Basically, his short term memory was preserved. However, HM could not commit new events to memory. He lost the ability to transfer information from his short-term memory to his long term memory, something memory researchers call consolidation. So while he could carry on a conversation with someone, he would completely forget the conversation after it ended. This was an extremely important case study for memory researchers because it suggested that there’s a dissociation between short-term memory and long-term memory, it suggested that these were two different abilities sub-served by different areas of the brain. It also suggested that the temporal lobes are particularly important for consolidating new information (i.e., for transferring information from short-term memory to long-term memory).

The history of psychology is filled with influential cases studies, such as Sigmund Freud’s description of “Anna O.” (see Note 6.1 “The Case of “Anna O.””) and John Watson and Rosalie Rayner’s description of Little Albert (Watson & Rayner, 1920) [7] , who allegedly learned to fear a white rat—along with other furry objects—when the researchers repeatedly made a loud noise every time the rat approached him.

The Case of “Anna O.”

Sigmund Freud used the case of a young woman he called “Anna O.” to illustrate many principles of his theory of psychoanalysis (Freud, 1961) [8] . (Her real name was Bertha Pappenheim, and she was an early feminist who went on to make important contributions to the field of social work.) Anna had come to Freud’s colleague Josef Breuer around 1880 with a variety of odd physical and psychological symptoms. One of them was that for several weeks she was unable to drink any fluids. According to Freud,

She would take up the glass of water that she longed for, but as soon as it touched her lips she would push it away like someone suffering from hydrophobia.…She lived only on fruit, such as melons, etc., so as to lessen her tormenting thirst. (p. 9)

But according to Freud, a breakthrough came one day while Anna was under hypnosis.

[S]he grumbled about her English “lady-companion,” whom she did not care for, and went on to describe, with every sign of disgust, how she had once gone into this lady’s room and how her little dog—horrid creature!—had drunk out of a glass there. The patient had said nothing, as she had wanted to be polite. After giving further energetic expression to the anger she had held back, she asked for something to drink, drank a large quantity of water without any difficulty, and awoke from her hypnosis with the glass at her lips; and thereupon the disturbance vanished, never to return. (p.9)

Freud’s interpretation was that Anna had repressed the memory of this incident along with the emotion that it triggered and that this was what had caused her inability to drink. Furthermore, he believed that her recollection of the incident, along with her expression of the emotion she had repressed, caused the symptom to go away.

As an illustration of Freud’s theory, the case study of Anna O. is quite effective. As evidence for the theory, however, it is essentially worthless. The description provides no way of knowing whether Anna had really repressed the memory of the dog drinking from the glass, whether this repression had caused her inability to drink, or whether recalling this “trauma” relieved the symptom. It is also unclear from this case study how typical or atypical Anna’s experience was.

Case studies are useful because they provide a level of detailed analysis not found in many other research methods and greater insights may be gained from this more detailed analysis. As a result of the case study, the researcher may gain a sharpened understanding of what might become important to look at more extensively in future more controlled research. Case studies are also often the only way to study rare conditions because it may be impossible to find a large enough sample of individuals with the condition to use quantitative methods. Although at first glance a case study of a rare individual might seem to tell us little about ourselves, they often do provide insights into normal behavior. The case of HM provided important insights into the role of the hippocampus in memory consolidation.

However, it is important to note that while case studies can provide insights into certain areas and variables to study, and can be useful in helping develop theories, they should never be used as evidence for theories. In other words, case studies can be used as inspiration to formulate theories and hypotheses, but those hypotheses and theories then need to be formally tested using more rigorous quantitative methods. The reason case studies shouldn’t be used to provide support for theories is that they suffer from problems with both internal and external validity. Case studies lack the proper controls that true experiments contain. As such, they suffer from problems with internal validity, so they cannot be used to determine causation. For instance, during HM’s surgery, the surgeon may have accidentally lesioned another area of HM’s brain (a possibility suggested by the dissection of HM’s brain following his death) and that lesion may have contributed to his inability to consolidate new information. The fact is, with case studies we cannot rule out these sorts of alternative explanations. So, as with all observational methods, case studies do not permit determination of causation. In addition, because case studies are often of a single individual, and typically an abnormal individual, researchers cannot generalize their conclusions to other individuals. Recall that with most research designs there is a trade-off between internal and external validity. With case studies, however, there are problems with both internal validity and external validity. So there are limits both to the ability to determine causation and to generalize the results. A final limitation of case studies is that ample opportunity exists for the theoretical biases of the researcher to color or bias the case description. Indeed, there have been accusations that the woman who studied HM destroyed a lot of her data that were not published and she has been called into question for destroying contradictory data that didn’t support her theory about how memories are consolidated. There is a fascinating New York Times article that describes some of the controversies that ensued after HM’s death and analysis of his brain that can be found at: https://www.nytimes.com/2016/08/07/magazine/the-brain-that-couldnt-remember.html?_r=0

Archival Research

Another approach that is often considered observational research involves analyzing archival data that have already been collected for some other purpose. An example is a study by Brett Pelham and his colleagues on “implicit egotism”—the tendency for people to prefer people, places, and things that are similar to themselves (Pelham, Carvallo, & Jones, 2005) [9] . In one study, they examined Social Security records to show that women with the names Virginia, Georgia, Louise, and Florence were especially likely to have moved to the states of Virginia, Georgia, Louisiana, and Florida, respectively.

As with naturalistic observation, measurement can be more or less straightforward when working with archival data. For example, counting the number of people named Virginia who live in various states based on Social Security records is relatively straightforward. But consider a study by Christopher Peterson and his colleagues on the relationship between optimism and health using data that had been collected many years before for a study on adult development (Peterson, Seligman, & Vaillant, 1988) [10] . In the 1940s, healthy male college students had completed an open-ended questionnaire about difficult wartime experiences. In the late 1980s, Peterson and his colleagues reviewed the men’s questionnaire responses to obtain a measure of explanatory style—their habitual ways of explaining bad events that happen to them. More pessimistic people tend to blame themselves and expect long-term negative consequences that affect many aspects of their lives, while more optimistic people tend to blame outside forces and expect limited negative consequences. To obtain a measure of explanatory style for each participant, the researchers used a procedure in which all negative events mentioned in the questionnaire responses, and any causal explanations for them were identified and written on index cards. These were given to a separate group of raters who rated each explanation in terms of three separate dimensions of optimism-pessimism. These ratings were then averaged to produce an explanatory style score for each participant. The researchers then assessed the statistical relationship between the men’s explanatory style as undergraduate students and archival measures of their health at approximately 60 years of age. The primary result was that the more optimistic the men were as undergraduate students, the healthier they were as older men. Pearson’s  r  was +.25.

This method is an example of  content analysis —a family of systematic approaches to measurement using complex archival data. Just as structured observation requires specifying the behaviors of interest and then noting them as they occur, content analysis requires specifying keywords, phrases, or ideas and then finding all occurrences of them in the data. These occurrences can then be counted, timed (e.g., the amount of time devoted to entertainment topics on the nightly news show), or analyzed in a variety of other ways.

Media Attributions

• What happens when you remove the hippocampus? – Sam Kean by TED-Ed licensed under a standard YouTube License
• Pappenheim 1882  by unknown is in the  Public Domain .
• Festinger, L., Riecken, H., & Schachter, S. (1956). When prophecy fails: A social and psychological study of a modern group that predicted the destruction of the world. University of Minnesota Press. ↵
• Rosenhan, D. L. (1973). On being sane in insane places. Science, 179 , 250–258. ↵
• Wilkins, A. (2008). “Happier than Non-Christians”: Collective emotions and symbolic boundaries among evangelical Christians. Social Psychology Quarterly, 71 , 281–301. ↵
• Levine, R. V., & Norenzayan, A. (1999). The pace of life in 31 countries. Journal of Cross-Cultural Psychology, 30 , 178–205. ↵
• Kraut, R. E., & Johnston, R. E. (1979). Social and emotional messages of smiling: An ethological approach. Journal of Personality and Social Psychology, 37 , 1539–1553. ↵
• Cohen, D., Nisbett, R. E., Bowdle, B. F., & Schwarz, N. (1996). Insult, aggression, and the southern culture of honor: An "experimental ethnography." Journal of Personality and Social Psychology, 70 (5), 945-960. ↵
• Watson, J. B., & Rayner, R. (1920). Conditioned emotional reactions. Journal of Experimental Psychology, 3 , 1–14. ↵
• Freud, S. (1961).  Five lectures on psycho-analysis . New York, NY: Norton. ↵
• Pelham, B. W., Carvallo, M., & Jones, J. T. (2005). Implicit egotism. Current Directions in Psychological Science, 14 , 106–110. ↵
• Peterson, C., Seligman, M. E. P., & Vaillant, G. E. (1988). Pessimistic explanatory style is a risk factor for physical illness: A thirty-five year longitudinal study. Journal of Personality and Social Psychology, 55 , 23–27. ↵

Research that is non-experimental because it focuses on recording systemic observations of behavior in a natural or laboratory setting without manipulating anything.

An observational method that involves observing people’s behavior in the environment in which it typically occurs.

When researchers engage in naturalistic observation by making their observations as unobtrusively as possible so that participants are not aware that they are being studied.

Where the participants are made aware of the researcher presence and monitoring of their behavior.

Refers to when a measure changes participants’ behavior.

In the case of undisguised naturalistic observation, it is a type of reactivity when people know they are being observed and studied, they may act differently than they normally would.

Researchers become active participants in the group or situation they are studying.

Researchers pretend to be members of the social group they are observing and conceal their true identity as researchers.

Researchers become a part of the group they are studying and they disclose their true identity as researchers to the group under investigation.

When a researcher makes careful observations of one or more specific behaviors in a particular setting that is more structured than the settings used in naturalistic or participant observation.

A part of structured observation whereby the observers use a clearly defined set of guidelines to "code" behaviors—assigning specific behaviors they are observing to a category—and count the number of times or the duration that the behavior occurs.

An in-depth examination of an individual.

A family of systematic approaches to measurement using qualitative methods to analyze complex archival data.

Research Methods in Psychology Copyright © 2019 by Rajiv S. Jhangiani, I-Chant A. Chiang, Carrie Cuttler, & Dana C. Leighton is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

What Are Observational Studies?

Observational studies are research studies in which researchers collect information from participants or look at data that was already collected. 

In observational studies, researchers follow groups of people over a period of time. Depending on the study, groups may include healthy people, people with cancer, or people who are at high risk for developing cancer, such as those with a family history.  

How observational studies help cancer research

Observational studies can help researchers learn more about cancer and suggest paths for future research that may lead to insights such as:

• how specific cancers form, grow, and spread
• genes that cause cancer to develop at a high rate within certain groups
• exposures or behaviors that may increase the risk of cancer
• clues to help prevent cancer
• clues that lead to new treatments
• patterns and trends of new cancer cases 
• the experiences of people who have had cancer in the past

Types of observational studies 

There are different types of observational studies. Two examples include natural history and longitudinal studies.

Natural history studies look at certain conditions in people with cancer or people who are at a high risk of developing cancer. Researchers often collect information about a person’s and their family’s medical history, as well as blood, saliva, and tumor samples that may be studied to learn more about how cancer develops or how it responds to treatment.

Longitudinal studies gather data on people over time, often to see whether those with different exposures have different cancer outcomes. Examples include those with different kinds of diets, smoking history, or other traits.

What to expect

Find Observation Studies >

View a studies that are looking for people now.

What you do as part of an observational study depends on the study. In some studies, you may be asked to fill out surveys or questionnaires. Researchers may ask about your medical history and that of your family. They might ask you to provide tissue samples, such as blood, saliva, or—if you have cancer—your tumor. Some studies may require your medical records. For some studies you might be asked to come in for an in-person visit with the research team.

Depending on the study, you might provide samples and information just once, or many times over the course of the study. 

Possible risks and benefits

There are few risks to taking part in an observational research study. One possible risk is the accidental release of information from your health records. To prevent this from happening, there are security measures in place to protect your privacy. A benefit of taking part in an observational study is knowing that you will help doctors learn more about cancer. These studies help create a foundation that can lead to further research that may help people with cancer in the future. Or they might help people in the future avoid cancer.

Your rights 

Before you join a study, the research team will make sure you understand:

• why the study is being done
• what will happen during the study
• how it may affect your daily life

Once you understand the study and decide to take part, you will be asked to sign a consent form. But even after you sign the form, you can change your mind and leave the study at any time.

Costs and expenses

Most observational studies are free to those who take part. As you think about joining an observational study, be sure to ask the study team about costs.

Observational studies: a review of study designs, challenges and strategies to reduce confounding

Affiliation.

• 1 Quality Use of Medicines and Pharmacy Research Centre, Sansom Institute, University of South Australia, SA, Australia. [email protected]
• PMID: 19392919
• DOI: 10.1111/j.1742-1241.2009.02056.x

There are several methods in which one can assess the relationship between an intervention and an outcome. Randomized controlled trials (RCTs) are considered as the gold standard for evaluating interventions. However, for many questions of clinical importance, RCTs would be impractical or unethical. Clinicians must rely on observational studies for the best available evidence when RCTs are unavailable. This article provides an overview of observational research designs to facilitate the understanding and appraising of their validity and applicability in clinical practice. Major methodological issues of observational studies including selection bias and confounding are also discussed. In addition, strategies to minimize these problems in the design and analytical phases of a study are highlighted. Knowledge of the strengths, weaknesses and recent methodological advances in observational studies can assist clinicians to make informed decisions about whether a particular observational study would provide useful information to enhance patient care.

Publication types

• Research Support, Non-U.S. Gov't
• Confounding Factors, Epidemiologic*
• Epidemiologic Methods*
• Medical Records
• Observation / methods*
• Research Design*

Experimental Studies and Observational Studies

• Reference work entry
• First Online: 01 January 2022
• pp 1748–1756
• Cite this reference work entry

• Martin Pinquart 3  

711 Accesses

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Atalay K, Barrett GF (2015) The impact of age pension eligibility age on retirement and program dependence: evidence from an Australian experiment. Rev Econ Stat 97:71–87. https://doi.org/10.1162/REST_a_00443

Article   Google Scholar  

Bergeman L, Boker SM (eds) (2016) Methodological issues in aging research. Psychology Press, Hove

Google Scholar  

Byrkes CR, Bielak AMA (under review) Evaluation of publication bias and statistical power in gerontological psychology. Manuscript submitted for publication

Campbell DT, Stanley JC (1966) Experimental and quasi-experimental designs for research. Rand-McNally, Chicago

Carpenter D (2010) Reputation and power: organizational image and pharmaceutical regulation at the FDA. Princeton University Press, Princeton

Cavanaugh JC, Blanchard-Fields F (2019) Adult development and aging, 8th edn. Cengage, Boston

Fölster M, Hess U, Hühnel I et al (2015) Age-related response bias in the decoding of sad facial expressions. Behav Sci 5:443–460. https://doi.org/10.3390/bs5040443

Freund AM, Isaacowitz DM (2013) Beyond age comparisons: a plea for the use of a modified Brunswikian approach to experimental designs in the study of adult development and aging. Hum Dev 56:351–371. https://doi.org/10.1159/000357177

Haslam C, Morton TA, Haslam A et al (2012) “When the age is in, the wit is out”: age-related self-categorization and deficit expectations reduce performance on clinical tests used in dementia assessment. Psychol Aging 27:778–784. https://doi.org/10.1037/a0027754

Institute for Social Research (2018) The health and retirement study. Aging in the 21st century: Challenges and opportunities for americans. Survey Research Center, University of Michigan

Jung J (1971) The experimenter’s dilemma. Harper & Row, New York

Leary MR (2001) Introduction to behavioral research methods, 3rd edn. Allyn & Bacon, Boston

Lindenberger U, Scherer H, Baltes PB (2001) The strong connection between sensory and cognitive performance in old age: not due to sensory acuity reductions operating during cognitive assessment. Psychol Aging 16:196–205. https://doi.org/10.1037//0882-7974.16.2.196

Löckenhoff CE, Carstensen LL (2004) Socioemotional selectivity theory, aging, and health: the increasingly delicate balance between regulating emotions and making tough choices. J Pers 72:1395–1424. https://doi.org/10.1111/j.1467-6494.2004.00301.x

Maxwell SE (2015) Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? Am Psychol 70:487–498. https://doi.org/10.1037/a0039400

Menard S (2002) Longitudinal research (2nd ed.). Sage, Thousand Oaks, CA

Mitchell SJ, Scheibye-Knudsen M, Longo DL et al (2015) Animal models of aging research: implications for human aging and age-related diseases. Ann Rev Anim Biosci 3:283–303. https://doi.org/10.1146/annurev-animal-022114-110829

Moher D (1998) CONSORT: an evolving tool to help improve the quality of reports of randomized controlled trials. JAMA 279:1489–1491. https://doi.org/10.1001/jama.279.18.1489

Oxford Centre for Evidence-Based Medicine (2011) OCEBM levels of evidence working group. The Oxford Levels of Evidence 2. Available at: https://www.cebm.net/category/ebm-resources/loe/ . Retrieved 2018-12-12

Patten ML, Newhart M (2018) Understanding research methods: an overview of the essentials, 10th edn. Routledge, New York

Piccinin AM, Muniz G, Sparks C et al (2011) An evaluation of analytical approaches for understanding change in cognition in the context of aging and health. J Geront 66B(S1):i36–i49. https://doi.org/10.1093/geronb/gbr038

Pinquart M, Silbereisen RK (2006) Socioemotional selectivity in cancer patients. Psychol Aging 21:419–423. https://doi.org/10.1037/0882-7974.21.2.419

Redman LM, Ravussin E (2011) Caloric restriction in humans: impact on physiological, psychological, and behavioral outcomes. Antioxid Redox Signal 14:275–287. https://doi.org/10.1089/ars.2010.3253

Rutter M (2007) Proceeding from observed correlation to causal inference: the use of natural experiments. Perspect Psychol Sci 2:377–395. https://doi.org/10.1111/j.1745-6916.2007.00050.x

Schaie W, Caskle CI (2005) Methodological issues in aging research. In: Teti D (ed) Handbook of research methods in developmental science. Blackwell, Malden, pp 21–39

Shadish WR, Cook TD, Campbell DT (2002) Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin, Boston

Sonnega A, Faul JD, Ofstedal MB et al (2014) Cohort profile: the health and retirement study (HRS). Int J Epidemiol 43:576–585. https://doi.org/10.1093/ije/dyu067

Weil J (2017) Research design in aging and social gerontology: quantitative, qualitative, and mixed methods. Routledge, New York

Download references

Author information

Authors and affiliations.

Psychology, Philipps University, Marburg, Germany

Martin Pinquart

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Martin Pinquart .

Editor information

Editors and affiliations.

Population Division, Department of Economics and Social Affairs, United Nations, New York, NY, USA

Department of Population Health Sciences, Department of Sociology, Duke University, Durham, NC, USA

Matthew E. Dupre

Section Editor information

Department of Sociology and Center for Population Health and Aging, Duke University, Durham, NC, USA

Kenneth C. Land

Department of Sociology, University of Kentucky, Lexington, KY, USA

Anthony R. Bardo

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this entry

Cite this entry.

Pinquart, M. (2021). Experimental Studies and Observational Studies. In: Gu, D., Dupre, M.E. (eds) Encyclopedia of Gerontology and Population Aging. Springer, Cham. https://doi.org/10.1007/978-3-030-22009-9_573

Download citation

DOI : https://doi.org/10.1007/978-3-030-22009-9_573

Published : 24 May 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-22008-2

Online ISBN : 978-3-030-22009-9

eBook Packages : Social Sciences Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 07 June 2023

Observational studies must be reformed before the next pandemic

• Emily E. Ricotta   ORCID: orcid.org/0000-0001-9928-8275 1 ,
• Annette Rid 2 ,
• I. Glenn Cohen   ORCID: orcid.org/0000-0002-0223-3305 3 &
• Nicholas G. Evans   ORCID: orcid.org/0000-0002-3330-0224 4  

Nature Medicine volume  29 ,  pages 1903–1905 ( 2023 ) Cite this article

4793 Accesses

3 Citations

59 Altmetric

Metrics details

• Epidemiology

Observational studies provide crucial information early during epidemics and pandemics, but they often suffer from methodological shortcomings, which can be resolved.

Scientific research is a necessary part of epidemic preparedness and response. Observational studies, in which the intervention and outcome(s) of interest are not under the researcher’s control, are used in epidemics to describe basic properties of a pathogen and its transmission; clinical symptoms; associations between interventions and patient outcomes; and the effectiveness of public health measures to curb disease spread.

Early importance

An example of a type of observational study that is particularly important for epidemic research is a prospective cohort study. These studies enroll populations of individuals who have a particular exposure or similar characteristics, and researchers collect data to evaluate possible outcomes associated with their exposure. For example, the Immunophenotyping Assessment in a COVID-19 Cohort (IMPACC) study was a prospective cohort study that launched in May 2020 1 . It followed newly hospitalized, SARS-CoV-2-positive individuals to understand clinical and immunological disease manifestations. The study results defined a set of clinical characteristics to assist clinicians with diagnosis and treatment; it also set the stage to evaluate individuals suffering from long COVID.

Observational study designs are ideal early in epidemics because of their speed and ease of implementation across settings, lower cost relative to other study designs, and flexibility in integrating pre-existing data sources, such as historical clinical data, census data and previous study results, to control for variables that may not be collectable. Beyond generating information to improve public health responses, rigorous observational studies can inform the design of subsequent randomized, controlled trials (RCTs) of novel interventions, ultimately reducing morbidity and saving lives 2 .

An observational study can be an ethically superior design early in epidemics because the risk–benefit tradeoff is frequently simpler than for other study designs, such as RCTs. For instance, because they do not directly provide experimental interventions, observational studies do not cause any intervention-related adverse events 3 . There are also settings in which RCTs are impossible to conduct, for example when an epidemic is emerging and outcomes are still so rare that achieving sufficient enrollment and statistical power in RCTs is infeasible 4 .

Yet despite their major potential for scientific and social value 5 , observational trials in recent epidemics and pandemics have failed to address priority research questions and suffered from important methodological shortcomings, generating false leads for investigators and policy-makers and contributing to scientific misinformation and mistrust. Targeted reforms that are neither resource nor time intensive can address these problems.

Methodological shortcomings

Some observational studies in recent epidemics generated information that did not have the potential (ex ante) to lead to significant health benefits or did not address vital research questions, such as clinical presentation, host specificity or transmissibility. For example, one meta-analysis assessing the association between ABO blood type and risk of SARS-CoV-2 infection found 314 relevant papers with data collected in 2020 6 . Although conducting a few high-quality studies of blood type and risk of infection may have been justified, in the context of a global pandemic and without a clearly actionable finding, the existence of upward of 300 studies seems to be of low social value when considering the priority research questions in a public health emergency.

Many observational studies in recent epidemics were conducted in ways that were methodologically or otherwise flawed, reducing the likelihood that the results could lead to substantial health benefits. In particular, studies variously lacked data standards such as defined units and vocabularies for the management of data across studies that examined the same intervention or outcome; did not measure or incorrectly measured confounding factors; or used incorrect design and analytic methods 7 , 8 . Infamously, poorly conducted and heavily biased observational studies of hydroxychloroquine resulted in its use as treatment for COVID-19, causing patients to receive incorrect treatment, given that it was ultimately found ineffective, and disrupting the early pandemic response 7 . Post hoc strategies to address methodologic biases are moreover limited and can result in conflicting evidence at best or compound incorrect or harmful evidence at worst 9 .

Observational studies, especially if conducted early in an epidemic, may also suffer from small samples and inconsistencies in sample selection, limiting their generalizability from the sample to the broader population (external validity) and their power to detect significant results. Geographical dispersion of events may mean that individual research teams have few cases on which to build a study, and those cases may be more reflective of the particular features of a study site than of the outbreak in question 7 , 8 . This, too, reduces the likelihood that study results can lead to significant health benefits.

Indeed, observational studies may have negative social value if their findings undermine the epidemic response. Observational studies, not least because of the speed with which they can be conducted, may spread low-quality or spurious information, thereby informing major and potentially irrevocable decisions in the early epidemic. These decisions cost lives due to the adoption of ineffective interventions and the abandonment of effective ones, divert limited resources for healthcare and research, and lead to overall poor policymaking 7 .

Even well-designed observational studies are difficult to communicate and easily misinterpreted by policy-makers, journalists and the public in the often rapidly evolving situation of an epidemic. Residual confounding, bias and study estimators (such as odds ratios) in observational studies are less clear than those in clinical trials and are more varied study to study, meaning that interpreting study results may take more time and be less straightforward to non-experts. There is a real information hazard if the results and limitations of those studies are not well communicated.

Master protocols

Given the limitations of observational studies identified above, reforms are needed to address these limitations, thereby promoting the social value of observational studies in future epidemics (Table 1 ).

Master observational study protocols should be developed to establish priority research questions during infectious disease outbreaks, helping to guarantee that study results lead to health benefits. Master protocols also help ensure appropriate participant and measurement variable selection while reducing bias, increasing the likelihood that health benefits will result. Readily available protocols will aid in prioritizing important information for outbreak response, such as basic reproduction number, symptoms, prognosis for different risk groups and effectiveness of nonpharmaceutical interventions. These protocols should be developed in consultation with a diverse segment of the research community to ensure that priorities and outcome measures are robust in advance of the next epidemic. Ongoing protocol development for multisite RCTs can inform this process 10 , 11 .

Data consistency

Open data standards should be developed and adopted to improve consistency in data collection, especially of outcomes such as case definition or intervention effectiveness 12 . Harmonized data standards enable data comparison across studies and reduce the burden of managing mountains of incompatible data, increasing the likelihood that observational studies will provide socially valuable results. Lessons can be learned from clinical data standards that help harmonize and standardize clinical data from electronic health records and claims data 13 .

Research groups should be empowered to collaborate and consolidate observational study data into larger samples to improve research quality and make it more likely that the results will improve clinical or public health practice. This goes beyond merely harmonizing and centralizing participating sites’ electronic health records. Rather, research sponsors should establish funding opportunities and large-scale collaborative research networks that create a shared sense of purpose and trust (ref. 10 ; https://www.recoverytrial.net/ ).

Research communication

Conducting observational studies across settings via a network unified in protocol and data standards would provide high-quality evidence to policymakers and prevent the controversy associated with multiple low-quality studies providing conflicting information. It would also reduce delays in effective policy action due to incomplete or incorrect results that require rolling back recommendations, while allowing scientists to focus finite resources on rigorously addressing priority research questions.

Reformed observational studies should, when published, be accompanied by appropriate explanatory text to guide their interpretation, such as the "Key Findings” section provided by some journals at the beginning of articles to provide context. Including an equally prominent, plain-language interpretation of the statistical claims and limitations of observational studies could mitigate the risk of intentional or unintentional misinterpretation. Journals could require authors to provide such lay scientific method summaries after peer review and acceptance but before publication, or work with authors and in-house staff to craft these summaries.

Patient privacy

Reforming observational studies as proposed has the potential to improve their social value, but it also poses challenges. Happily, these are relatively easy to overcome. Sharing, using and reusing data from observational studies can significantly increase their scientific and social value. However, these practices may also result in broad dissemination of participants’ protected health information. Data standards and sharing will need to align with existing ethical norms for protecting participants’ data privacy and confidentiality.

The informed consent process for observational studies is frequently less robust than that for interventional studies, creating a barrier to understanding the true meaning and extent of data sharing. Even if participants are not ultimately harmed by this practice, a lack of informed discussion before data collection can undermine public trust in health research. Fortunately, appropriate data collection, management, use and reuse efforts can promote social value without compromising participants’ rights or interests. This has been demonstrated by the incorporation of research participant preferences about data storage, management and reuse into modern studies involving data-sharing activities 14 .

Adoption of standards

A second challenge is the adoption of data and protocol standards in observational studies by the scientific community, especially absent a coordinating body. Here, professional societies, research sponsors, regulators, journal editors and journalists can play critical roles in requiring or incentivizing the adoption of these standards. For example, professional societies could champion the standards as the best practice in the field. In the USA, the National Institutes of Health (NIH) could clarify that specific standards for observational studies are required per its new data management and sharing policy, or it could spearhead the design and implementation of observational study standards for NIH-sponsored research that set an example for the field. The US Food and Drug Administration (FDA) requires the use of specific data standards for New Drug Applications and Biologics License Applications and can refuse to receive any electronic submission whose study data do not conform to those specified in the FDA Data Standards Catalog. These programs could be extended to observational trials that support regulatory approval. Prominent journals or consortia such as the Committee on Publication Ethics could require that submitted manuscripts adhere to data and protocol standards, as some journals have done by requiring that manuscripts follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Journalists could also recognize master protocol-compliant research as a standard of excellence in the field in their reporting.

Finally, lowering barriers to implementation through free access to training materials, open standards and data to facilitate research would provide an incentive to form an observational study user community. Designing new standards for observational studies and training researchers to adopt them as a part of routine scientific inquiry would also develop capacity for producing high-quality results during the next crisis.

At the regulatory level, standardizing ethical and scientific review of master protocols and data standards within and between countries would allow simultaneous collection of high-quality observational data for public health response and research use, rather than the conduct of research using data and samples collected solely for response purposes without the use of rigorous (or any) epidemiologic methods. Adoption of data standards and communications practices by journals will prepare them for the next rush of epidemic research and will prime media and policymakers to understand the statistical claims therein.

An ethical imperative

Observational studies, done properly, are a lifeline, especially in a crisis. These proposed reforms to improve the social value of epidemic observational studies require modest investment by research sponsors, professional societies, academic journals and observational trialists themselves, without raising new ethical concerns. This makes pursuing these reforms an ethical imperative, as they will save lives at low cost to the scientific and policy communities. The global pandemic that has killed more than 6 million people and fundamentally reshaped the world continues. Now that the acute phase is over, there is a critical opportunity to begin planning for the next pandemic and develop protocols and policies that can also be used in response to other global health challenges.

IMPACC Manuscript Writing Team. Sci. Immunol. 6 , eabf3733 (2021).

Paneth, N. & Joyner, M. J. Clin. Invest. 131 , e146392 (2021).

Article   PubMed   PubMed Central   Google Scholar  

Rid, A. & Emanuel, E. J. The Lancet 384 , 1896–1899 (2014).

Article   Google Scholar  

Cohen, J. Science https://doi.org/10.1126/science.aav3996 (2018).

Rid, A. Perspect. Biol. Med. 63 , 293–312 (2020).

Article   PubMed   Google Scholar  

Banchelli, F. et al. J. Clin. Med. 11 , 3029 (2022).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alexander, P. E. et al. J. Clin. Epidemiol. 123 , 120–126 (2020).

Alirol, E. et al. BMC Med. Ethics 18 , 43 (2017).

Boudesseul, J. et al. Front. Public Health 9 , 722458 (2021).

Ogburn, E. L. et al. Science 368 , 1198–1199 (2020).

Dean, N. E. et al. N. Engl. J. Med. 382 , 1366–1369 (2020).

Morton, S. C. et al. J. Clin. Epidemiol. 71 , 3–10 (2016).

Institute of Medicine. Patient Safety: Achieving a New Standard for Care; https://doi.org/10.17226/10863 (National Academies, 2004).

Omberg, L. et al. Nat. Biotechnol. 40 , 480–487 (2022).

Article   CAS   PubMed   Google Scholar  

Download references

Acknowledgements

This article was supported in part by the division of intramural research of the National Institute of Allergy and Infectious Diseases and The Clinical Center, NIH. N.G.E. received funding from the Greenwall Foundation, the Davis Educational Foundation and the US Air Force Office of Scientific Research. The views expressed here are those of the authors and do not necessarily reflect the policies of the National Institutes of Health or the US Department of Health and Human Services.

Author information

Authors and affiliations.

Epidemiology and Data Management Unit, Division of Intramural Research, National Institute of Allergy and Infectious Diseases, National Institutes of Health (NIH), Bethesda, MD, USA

Emily E. Ricotta

Department of Bioethics, Clinical Center, NIH, Bethesda, MD, USA

Annette Rid

Petrie-Flom Center for Health Law Policy, Biotechnology & Bioethics, Harvard Law School, Cambridge, MA, USA

I. Glenn Cohen

Department of Philosophy, University of Massachusetts Lowell, Lowell, MA, USA

Nicholas G. Evans

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Emily E. Ricotta .

Ethics declarations

Competing interests.

I.G.C. reports serving on the bioethics advisory board of Illumina Serves and the bioethics council of Bayer.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Ricotta, E.E., Rid, A., Cohen, I.G. et al. Observational studies must be reformed before the next pandemic. Nat Med 29 , 1903–1905 (2023). https://doi.org/10.1038/s41591-023-02375-8

Download citation

Published : 07 June 2023

Issue Date : August 2023

DOI : https://doi.org/10.1038/s41591-023-02375-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

We have a new app!

Take the Access library with you wherever you go—easy access to books, videos, images, podcasts, personalized features, and more.

Download the Access App here: iOS and Android . Learn more here!

• Remote Access
• Save figures into PowerPoint
• Download tables as PDFs

Chapter 12. Observational Study Designs

Bradley G. Hammill

• Download Chapter PDF

Disclaimer: These citations have been automatically generated based on the information we have and it may not be 100% accurate. Please consult the latest official manual style if you have any questions regarding the format accuracy.

Download citation file:

• Search Book

Jump to a Section

• Introduction
• Analytic Study Designs
• Descriptive Study Designs
• Conclusions
• Full Chapter
• Supplementary Content

Observational studies in clinical research can be classified as either analytic or descriptive ( Table 12–1 ). Analytic observational studies are similar to randomized, controlled clinical trials in that the goal is to estimate the causal effect of an exposure on an outcome. Also similar to trials, analytic observational studies always include some type of comparison group, against which the experience of the exposed group is compared. Well-designed analytic studies can generate strong evidence for or against a stated hypothesis. Descriptive studies, on the other hand, aim to describe the characteristics or experiences of a particular patient group. Even well-designed descriptive studies cannot be used to draw strong conclusions about the effect of an exposure on an outcome. Instead, these studies are often used to generate study questions that can then be tested by more rigorous methods.

Although many observational study designs are available to researchers ( 1 ), a few are most widely used and will be described below. The analytic study designs presented are the case-control study and the cohort study. The descriptive study designs presented are the ecologic study, the cross-sectional prevalence survey, and case reports or case series.

Case-Control Studies

Sign in or create a free Access profile below to access even more exclusive content.

With an Access profile, you can save and manage favorites from your personal dashboard, complete case quizzes, review Q&A, and take these feature on the go with our Access app.

Pop-up div Successfully Displayed

This div only appears when the trigger link is hovered over. Otherwise it is hidden from view.

Please Wait

• Privacy Policy

Buy Me a Coffee

Home » Observational Research – Methods and Guide

Observational Research – Methods and Guide

Table of Contents

Observational Research

Definition:

Observational research is a type of research method where the researcher observes and records the behavior of individuals or groups in their natural environment. In other words, the researcher does not intervene or manipulate any variables but simply observes and describes what is happening.

Observation

Observation is the process of collecting and recording data by observing and noting events, behaviors, or phenomena in a systematic and objective manner. It is a fundamental method used in research, scientific inquiry, and everyday life to gain an understanding of the world around us.

Types of Observational Research

Observational research can be categorized into different types based on the level of control and the degree of involvement of the researcher in the study. Some of the common types of observational research are:

Naturalistic Observation

In naturalistic observation, the researcher observes and records the behavior of individuals or groups in their natural environment without any interference or manipulation of variables.

Controlled Observation

In controlled observation, the researcher controls the environment in which the observation is taking place. This type of observation is often used in laboratory settings.

Participant Observation

In participant observation, the researcher becomes an active participant in the group or situation being observed. The researcher may interact with the individuals being observed and gather data on their behavior, attitudes, and experiences.

Structured Observation

In structured observation, the researcher defines a set of behaviors or events to be observed and records their occurrence.

Unstructured Observation

In unstructured observation, the researcher observes and records any behaviors or events that occur without predetermined categories.

Cross-Sectional Observation

In cross-sectional observation, the researcher observes and records the behavior of different individuals or groups at a single point in time.

Longitudinal Observation

In longitudinal observation, the researcher observes and records the behavior of the same individuals or groups over an extended period of time.

Data Collection Methods

Observational research uses various data collection methods to gather information about the behaviors and experiences of individuals or groups being observed. Some common data collection methods used in observational research include:

Field Notes

This method involves recording detailed notes of the observed behavior, events, and interactions. These notes are usually written in real-time during the observation process.

Audio and Video Recordings

Audio and video recordings can be used to capture the observed behavior and interactions. These recordings can be later analyzed to extract relevant information.

Surveys and Questionnaires

Surveys and questionnaires can be used to gather additional information from the individuals or groups being observed. This method can be used to validate or supplement the observational data.

Time Sampling

This method involves taking a snapshot of the observed behavior at pre-determined time intervals. This method helps to identify the frequency and duration of the observed behavior.

Event Sampling

This method involves recording specific events or behaviors that are of interest to the researcher. This method helps to provide detailed information about specific behaviors or events.

Checklists and Rating Scales

Checklists and rating scales can be used to record the occurrence and frequency of specific behaviors or events. This method helps to simplify and standardize the data collection process.

Observational Data Analysis Methods

Observational Data Analysis Methods are:

Descriptive Statistics

This method involves using statistical techniques such as frequency distributions, means, and standard deviations to summarize the observed behaviors, events, or interactions.

Qualitative Analysis

Qualitative analysis involves identifying patterns and themes in the observed behaviors or interactions. This analysis can be done manually or with the help of software tools.

Content Analysis

Content analysis involves categorizing and counting the occurrences of specific behaviors or events. This analysis can be done manually or with the help of software tools.

Time-series Analysis

Time-series analysis involves analyzing the changes in behavior or interactions over time. This analysis can help identify trends and patterns in the observed data.

Inter-observer Reliability Analysis

Inter-observer reliability analysis involves comparing the observations made by multiple observers to ensure the consistency and reliability of the data.

Multivariate Analysis

Multivariate analysis involves analyzing multiple variables simultaneously to identify the relationships between the observed behaviors, events, or interactions.

Event Coding

This method involves coding observed behaviors or events into specific categories and then analyzing the frequency and duration of each category.

Cluster Analysis

Cluster analysis involves grouping similar behaviors or events into clusters based on their characteristics or patterns.

Latent Class Analysis

Latent class analysis involves identifying subgroups of individuals or groups based on their observed behaviors or interactions.

Social network Analysis

Social network analysis involves mapping the social relationships and interactions between individuals or groups based on their observed behaviors.

The choice of data analysis method depends on the research question, the type of data collected, and the available resources. Researchers should choose the appropriate method that best fits their research question and objectives. It is also important to ensure the validity and reliability of the data analysis by using appropriate statistical tests and measures.

Applications of Observational Research

Observational research is a versatile research method that can be used in a variety of fields to explore and understand human behavior, attitudes, and preferences. Here are some common applications of observational research:

• Psychology : Observational research is commonly used in psychology to study human behavior in natural settings. This can include observing children at play to understand their social development or observing people’s reactions to stress to better understand how stress affects behavior.
• Marketing : Observational research is used in marketing to understand consumer behavior and preferences. This can include observing shoppers in stores to understand how they make purchase decisions or observing how people interact with advertisements to determine their effectiveness.
• Education : Observational research is used in education to study teaching and learning in natural settings. This can include observing classrooms to understand how teachers interact with students or observing students to understand how they learn.
• Anthropology : Observational research is commonly used in anthropology to understand cultural practices and beliefs. This can include observing people’s daily routines to understand their culture or observing rituals and ceremonies to better understand their significance.
• Healthcare : Observational research is used in healthcare to understand patient behavior and preferences. This can include observing patients in hospitals to understand how they interact with healthcare professionals or observing patients with chronic illnesses to better understand their daily routines and needs.
• Sociology : Observational research is used in sociology to understand social interactions and relationships. This can include observing people in public spaces to understand how they interact with others or observing groups to understand how they function.
• Ecology : Observational research is used in ecology to understand the behavior and interactions of animals and plants in their natural habitats. This can include observing animal behavior to understand their social structures or observing plant growth to understand their response to environmental factors.
• Criminology : Observational research is used in criminology to understand criminal behavior and the factors that contribute to it. This can include observing criminal activity in a particular area to identify patterns or observing the behavior of inmates to understand their experience in the criminal justice system.

Observational Research Examples

Here are some real-time observational research examples:

• A researcher observes and records the behaviors of a group of children on a playground to study their social interactions and play patterns.
• A researcher observes the buying behaviors of customers in a retail store to study the impact of store layout and product placement on purchase decisions.
• A researcher observes the behavior of drivers at a busy intersection to study the effectiveness of traffic signs and signals.
• A researcher observes the behavior of patients in a hospital to study the impact of staff communication and interaction on patient satisfaction and recovery.
• A researcher observes the behavior of employees in a workplace to study the impact of the work environment on productivity and job satisfaction.
• A researcher observes the behavior of shoppers in a mall to study the impact of music and lighting on consumer behavior.
• A researcher observes the behavior of animals in their natural habitat to study their social and feeding behaviors.
• A researcher observes the behavior of students in a classroom to study the effectiveness of teaching methods and student engagement.
• A researcher observes the behavior of pedestrians and cyclists on a city street to study the impact of infrastructure and traffic regulations on safety.

How to Conduct Observational Research

Here are some general steps for conducting Observational Research:

• Define the Research Question: Determine the research question and objectives to guide the observational research study. The research question should be specific, clear, and relevant to the area of study.
• Choose the appropriate observational method: Choose the appropriate observational method based on the research question, the type of data required, and the available resources.
• Plan the observation: Plan the observation by selecting the observation location, duration, and sampling technique. Identify the population or sample to be observed and the characteristics to be recorded.
• Train observers: Train the observers on the observational method, data collection tools, and techniques. Ensure that the observers understand the research question and objectives and can accurately record the observed behaviors or events.
• Conduct the observation : Conduct the observation by recording the observed behaviors or events using the data collection tools and techniques. Ensure that the observation is conducted in a consistent and unbiased manner.
• Analyze the data: Analyze the observed data using appropriate data analysis methods such as descriptive statistics, qualitative analysis, or content analysis. Validate the data by checking the inter-observer reliability and conducting statistical tests.
• Interpret the results: Interpret the results by answering the research question and objectives. Identify the patterns, trends, or relationships in the observed data and draw conclusions based on the analysis.
• Report the findings: Report the findings in a clear and concise manner, using appropriate visual aids and tables. Discuss the implications of the results and the limitations of the study.

When to use Observational Research

Here are some situations where observational research can be useful:

• Exploratory Research: Observational research can be used in exploratory studies to gain insights into new phenomena or areas of interest.
• Hypothesis Generation: Observational research can be used to generate hypotheses about the relationships between variables, which can be tested using experimental research.
• Naturalistic Settings: Observational research is useful in naturalistic settings where it is difficult or unethical to manipulate the environment or variables.
• Human Behavior: Observational research is useful in studying human behavior, such as social interactions, decision-making, and communication patterns.
• Animal Behavior: Observational research is useful in studying animal behavior in their natural habitats, such as social and feeding behaviors.
• Longitudinal Studies: Observational research can be used in longitudinal studies to observe changes in behavior over time.
• Ethical Considerations: Observational research can be used in situations where manipulating the environment or variables would be unethical or impractical.

Purpose of Observational Research

Observational research is a method of collecting and analyzing data by observing individuals or phenomena in their natural settings, without manipulating them in any way. The purpose of observational research is to gain insights into human behavior, attitudes, and preferences, as well as to identify patterns, trends, and relationships that may exist between variables.

The primary purpose of observational research is to generate hypotheses that can be tested through more rigorous experimental methods. By observing behavior and identifying patterns, researchers can develop a better understanding of the factors that influence human behavior, and use this knowledge to design experiments that test specific hypotheses.

Observational research is also used to generate descriptive data about a population or phenomenon. For example, an observational study of shoppers in a grocery store might reveal that women are more likely than men to buy organic produce. This type of information can be useful for marketers or policy-makers who want to understand consumer preferences and behavior.

In addition, observational research can be used to monitor changes over time. By observing behavior at different points in time, researchers can identify trends and changes that may be indicative of broader social or cultural shifts.

Overall, the purpose of observational research is to provide insights into human behavior and to generate hypotheses that can be tested through further research.

Advantages of Observational Research

There are several advantages to using observational research in different fields, including:

• Naturalistic observation: Observational research allows researchers to observe behavior in a naturalistic setting, which means that people are observed in their natural environment without the constraints of a laboratory. This helps to ensure that the behavior observed is more representative of the real-world situation.
• Unobtrusive : Observational research is often unobtrusive, which means that the researcher does not interfere with the behavior being observed. This can reduce the likelihood of the research being affected by the observer’s presence or the Hawthorne effect, where people modify their behavior when they know they are being observed.
• Cost-effective : Observational research can be less expensive than other research methods, such as experiments or surveys. Researchers do not need to recruit participants or pay for expensive equipment, making it a more cost-effective research method.
• Flexibility: Observational research is a flexible research method that can be used in a variety of settings and for a range of research questions. Observational research can be used to generate hypotheses, to collect data on behavior, or to monitor changes over time.
• Rich data : Observational research provides rich data that can be analyzed to identify patterns and relationships between variables. It can also provide context for behaviors, helping to explain why people behave in a certain way.
• Validity : Observational research can provide high levels of validity, meaning that the results accurately reflect the behavior being studied. This is because the behavior is being observed in a natural setting without interference from the researcher.

Disadvantages of Observational Research

While observational research has many advantages, it also has some limitations and disadvantages. Here are some of the disadvantages of observational research:

• Observer bias: Observational research is prone to observer bias, which is when the observer’s own beliefs and assumptions affect the way they interpret and record behavior. This can lead to inaccurate or unreliable data.
• Limited generalizability: The behavior observed in a specific setting may not be representative of the behavior in other settings. This can limit the generalizability of the findings from observational research.
• Difficulty in establishing causality: Observational research is often correlational, which means that it identifies relationships between variables but does not establish causality. This can make it difficult to determine if a particular behavior is causing an outcome or if the relationship is due to other factors.
• Ethical concerns: Observational research can raise ethical concerns if the participants being observed are unaware that they are being observed or if the observations invade their privacy.
• Time-consuming: Observational research can be time-consuming, especially if the behavior being observed is infrequent or occurs over a long period of time. This can make it difficult to collect enough data to draw valid conclusions.
• Difficulty in measuring internal processes: Observational research may not be effective in measuring internal processes, such as thoughts, feelings, and attitudes. This can limit the ability to understand the reasons behind behavior.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Case Study – Methods, Examples and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Survey Research – Types, Methods, Examples

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

This Notice of Funding Opportunity (NOFO) encourages Research Project Grant (R01) applicants to pursue clinical observational (CO) studies to obtain data necessary for designing future clinical studies for musculoskeletal, rheumatic, or skin diseases or conditions. Research data from observational studies can enhance clinical study design by providing essential information about disease symptoms, stages and timing of disease progression, comorbid conditions, availability of potential research participants, and outcomes that are important to patients. CO studies also can facilitate efforts to validate objective biomarkers or subjective outcome measures for use in a future clinical study. 

Applicants to this funding opportunity are encouraged to propose studies that address significant obstacles or questions in the design of a clinical study, such as determining the appropriate primary or secondary outcome measures or the stages of disease during which patients are most likely to respond to an intervention.

Only observational studies will be supported through this NOFO. 

Research Area Examples

Specific examples of research areas of interest include but are not limited to:

• Characterization of the frequency and/or severity of disease associated symptoms that would be measured as an outcome in future clinical studies.
• Association studies that compare a biochemical or imaging biomarker to other, established outcome measures to establish surrogate markers.
• Testing of clinical study recruitment strategies for rare diseases or underserved populations that would be candidates for participating in a future clinical study.
• Collection of data on adverse events from administration of treatment under a standard of care setting in an observational study protocol that could be used to design a future clinical research study.
• Collection of standard of care data in an observational study setting to be used as a historical control for a future clinical research study.

• Open access
• Published: 12 April 2024

A prospective observational cohort study of covid-19 epidemiology and vaccine seroconversion in South Western Sydney, Australia, during the 2021–2022 pandemic period.

• Daniela Potter 1 , 2 , 3 ,
• Jason Diep 3 ,
• Colleen Munro 3 ,
• Noelle Lin 3 ,
• Ramon Xu 3 ,
• Jeffrey Wong 3 ,
• Robert Porritt 4 ,
• Michael Maley 4 ,
• Hong Foo 4 &
• Angela Makris 1 , 2 , 3  

BMC Nephrology volume  25 , Article number:  131 ( 2024 ) Cite this article

Metrics details

It is known that COVID-19 disproportionally adversely affects the immunocompromised, including kidney transplant recipients (KTR), as compared to the general population. Risk factors for adverse outcomes and vaccine seroconversion patterns are not fully understood. Australia was uniquely positioned to reduce initial case numbers during the 2021–2022 pandemic period due to its relative isolation and several significant public health interventions. South-Western Sydney Local Heath District was one of the predominant regions affected.

A single centre, prospective cohort study of prevalent renal transplant recipients was conducted between 25th July 2021 and 1st May 2022. Baseline characteristics, COVID-19 vaccination status, COVID-19 diagnosis and outcomes were determined from the electronic medical record, Australian vaccination register and Australian and New Zealand Dialysis and Transplant Registry. Assessment of vaccine-induced seroconversion was assessed with ELISA in a subpopulation. Analysis was performed using SPSS v.28.

We identified 444 prevalent transplant recipients (60% male, 50% diabetic, median age 58 years (Interquartile range (IQR)21.0) and eGFR 56 ml/min/1.73m 2 (IQR 21.9). COVID-19 was identified in 32% ( n  = 142) of patients, of which 38% ( n  = 54) required hospitalisation and 7% ( n  = 10) died. At least one COVID-19 vaccination was received by 95% ( n  = 423) with 17 (4%) patients remaining unvaccinated throughout the study period. Seroconversion after 2 and 3 doses of vaccine was 22% and 48% respectively. Increased COVID-19 related deaths were associated with older age (aOR 1.1, 95% CI 1.004–1.192, p  = 0.040), smoking exposure (aOR 8.2, 05% CI 1.020-65.649, p  = 0.048) and respiratory disease (aOR 14.2, 95%CI:1.825–110.930, p  = 0.011) on multi-variable regression analysis. Receipt of three doses of vaccination was protective against acquiring COVID-19 (aOR 0.48, 95% CI 0.287–0.796, p  = 0.005) and death (aOR 0.6, 95% CI: 0.007–0.523, p  = 0.011), but not against hospitalisation ( p  = 0.32). Seroconversion was protective for acquiring COVID-19 on multi-variable regression independent of vaccination dose (aOR 0.1, 95%CI: 0.0025–0.523, p  = 0.011).

Conclusions

COVID-19 was associated with a high mortality rate. Older age, respiratory disease and prior smoking exposure may be risk factors for increased mortality. Vaccination of 3 doses is protective against acquiring COVID-19 and death, however not hospitalisation. Antibody response is protective for acquiring COVID-19, however seroconversion rates are low.

Peer Review reports

Introduction

It is known that COVID-19 disproportionally adversely affects the immunocompromised, including kidney transplant recipients (KTR), as compared to the general population. The advent of specific COVID-19 therapies and novel vaccination improved outcomes, however mortality rates for organ transplant recipients from large cohort studies remained as high as 14% into 2021 [ 1 , 2 ]. Factors predicting mortality are not fully understood but age, cardiovascular disease, diabetes, and certain immunosuppression regimens have been suggested [ 1 , 3 , 4 , 5 , 6 , 7 ]. KTRs were prioritised for vaccine administration, however, were not included in original vaccination trials [ 8 , 9 ]. Subsequent data suggests conventional 2-dose regimens are insufficient for KTRs, with 3 doses potentially ineffective against later strains such as BA.1 (Omicron) [ 10 , 11 , 12 ]. The primary course of vaccination was extended, between March 2021 and July 2022, to 5 doses in Australia, however adequate ongoing vaccination strategies are unclear [ 13 ].

COVID-19 in Australia and South Western Sydney

Australia was protected from high case numbers during the early phases of the pandemic due to its geographical isolation, strict initial international border controls and aggressive case tracking. Although Australia comprises 6 states and two territories, each have a significant degree of independence and power in health policy making. Those states with low case numbers throughout 2020–2022, such as South and Western Australia, maintained strict international and interstate border controls, but relaxed internal restrictions with almost near normal, pre-COVID, living conditions. They enacted limited, “snap lockdowns” in response to small numbers of detected cases to keep COVID-19 suppressed, until the majority of the population could be vaccinated [ 14 ]. Within New South Wales, however, several significant outbreaks occurred in 2021–2022, prompting repeated modification of public orders and prolonged periods of community lockdown and restrictions [ 15 ]. South Western Sydney Local Health District (SWSLHD) was one of the first areas in New South Wales (NSW) to be affected by COVID-19, and experienced one of the higher reported case numbers and the highest reported deaths of any Local Health District in NSW [ 16 ]. In response to the high rates of infection, SWSLHD experienced the most restrictive lockdown regulations in NSW during the pandemic period. During the second NSW wave in 2021, several local government areas within SWSLHD were classed as “areas of concern” and had additional public orders imposed, including: a stay at home order, restrictions on entering or leaving a district except for specific work exemptions (which required a permit), not allowed to travel more than 5 km for exercise, mandatory mask wearing outside and, at one point, a 9pm to 5am curfew [ 17 ]. August and September 2021 was associated with peak B.1.617.2 (Delta) wave incidence, followed by peak BA.1 (Omicron) in January 2022 [ 16 ]. COVID-19 vaccination was available for immunosuppressed individuals in Australia from 22nd March 2021 [ 18 ]. There is also a large burden of chronic kidney disease (CKD), with SWSLHD accounting for approximately 3.3% of prevalent KTRs in Australia. SWSLHD is also diverse, multiethnic population with 54% of people speaking a language other than English, predominantly Arabic or Vietnamese, and 43% of the population were born overseas, in comparison to 29% to the rest of NSW [ 19 , 20 ].

This study was conducted in the 2nd to 3rd year of the pandemic, during two dominant strain outbreaks, B.1.617.2 (Delta) and BA.1.(Omicron), after vaccination was available for all recipients [ 15 ]. Our objective was to ascertain the impact of COVID-19 on KTRS, with a focus on acquisition, hospitalisation, and mortality from COVID-19, and to perform a serology assessment of vaccine seroconversion.

Study design

A single centre (SWSLHD) prospective cohort study of prevalent kidney transplant recipients was undertaken between 25th July 2021 and 1st May 2022.

The study was commenced prospectively, coinciding with the onset of rising COVID-19 transmission and initiation of community stay at home orders. After restrictions had ended, vaccination numbers had increased, and it was clear no further public health orders were likely to be initiated, the study was terminated. All KTRs were strongly encouraged to receive vaccination throughout the study period, via national public health messaging, family practitioner support, and nephrologist advice. The Renal department at SWSLHD undertook a program at this time to encourage immunisation by developing a multi-lingual (Arabic, Vietnamese) information letter in view of the multi-ethnic population (distributed, mailed or emailed) to all KTRs and dialysis patients. A dedicated contact nephrologist was available to answer vaccination specific queries to facilitate timely immunisation.

Participants

All prevalent KTRs, aged  ≥  18, were included in the initial observational component of the study (see ethics below). Patients were identified from an existing clinical database and cross-referenced by searching the entire health district electronic coding system for renal transplantation to reduce the risk of selection bias. After the final data collection point on 1st May 2022 the cohorts of COVID-19 positive and COVID-19 negative patients were identified.

Variables and data sources

Baseline clinical and transplant characteristics, including: age, sex, body mass index, place of birth, smoking status, primary renal disease, co-morbidities, baseline eGFR, use of any blood thinner, prior dialysis modality and modality change during study, requirement for an interpreter, number of transplants, donor type, number of mismatches, transplant vintage, baseline immunosuppression regiment, dosage and levels and administration of Anti-thymocyte globulin, were determined from the electronic medical record, the Australian and New Zealand Dialysis and Transplant Registry (ANZDATA) records, and locally available Nephrologist letters. The date and brand of each COVID-19 vaccination is recorded into the Australian Immunisation Register and electronic health record prospectively. We obtained information on every dose of COVID-19 vaccination provided to patients. We assessed the impact of increasing vaccination dose, from 1 onwards. COVID-19 diagnosis, and outcomes were determined from the electronic medical record, including date of diagnosis, administration of sotrovimab or molpurinovir, hospitalisation and level of care for COVID-19, oxygen requirement, use of dexamethasone and other adjunctive agents including baricitinib, tocilizumab, remdesivir and sarilumab, length of stay and mortality from COVID-19.

The combination of both ANZDATA records, local electronic health record and Nephrologist letters was utilised to reduce missing data and increase accuracy of data imputation. The study period encompassed a period of mandatory reporting of all positive COVID-19 polymerase chain results and rapid antigen tests to the NSW Health Service. In SWSLHD each positive result was reviewed by a dedicated COVID-19 Community Health service and documented in the electronic health record, which we anticipated would reduce the impact of sampling bias and missing data.

Serology assessment

All patients were invited to participate in the post COVID-19 vaccination serology conversion assessment component of the study. At study commencement all patients received a multi-lingual text message (English, Arabic, or Vietnamese) offering participation in COVID-19 serologic conversion testing. Additional written informed consent for this component of the study was obtained from those willing to participate. Blood tests we requested to be performed at least 14 days after their 2nd and 3rd vaccine dose. These patients were planned to be analysed as a subgroup from the main cohort.

All patient serum underwent testing at NSW Health Pathology– Liverpool, using both the Roche Elecsys Anti-SARS-CoV-2 assay (“Elecsys”) and the EUROIMMUN Anti-SARS-CoV-2 QuantiVAC ELISA (“QuantiVAC”), which have different targets. The Elecsys assay is an electrochemiluminescence assay for the qualitative detection of antibodies to SARS-CoV-2 nucleocapsid protein in human serum, and is considered reflective of wild-type infection [ 21 ]. A result (cutoff index; signal to cutoff ratio) of ≥  1.0 is considered reactive. The QuantiVAC ELISA is an enzyme immunoassay, providing quantitative in vitro determination of antibodies to the immunoglobulin class IgG against the S1 antigen and receptor binding domain of SARS-CoV-2 [ 22 ]. Detection of the anti-S1 (spike) antibody is considered to indicate either a wild-type infection or a response to vaccination. A result of < 8RU/ml was considered negative, ≥ 8-<11 RU/ml borderline and ≥ 11RU/ml positive. Utilising the results from these two assays, in conjunction with the patient’s vaccination status and any noted clinical COVID-19, it was possible to determine whether the patient’s antibody response was secondary to clinical infection or to vaccination (Supp Table  1 ).

To determine vaccine-induced seroconversion in patients who undertook serial testing, we reviewed the relationship of serial serum collections, vaccination and known COVID-19. The serum sample collected closest in time to the date of vaccination, with a minimum of 14 days post-vaccination was included in the analysis. If there was evidence of seroconversion from a reactive QuantiVac ELISA, subsequent reactive samples were not included. If there was evidence of seroconversion after a subsequent incrementing vaccine dose, with no known interval COVID-19, patients were considered to have vaccine-induced seroconversion at the incrementation. If there was no evidence of seroconversion, despite additional vaccine administration, the final sample was included to reflect this. Borderline results were considered as seroconverted in the context of immunocompromise.

This study was approved by the SWSLHD Human Research and Ethics Committee (Approval Reference: 2019/STE00860) with a waiver of consent for the initial cohort analysis of all KTRs in the district and individual informed consent for the serology component if a patient elected to participate.

Statistical methods

Data was analysed using parametric and non-parametric tests for normally distributed and non-normally distributed variables respectively. Univariate analysis was performed with chi-squared, or Fishers test as appropriate, on categorical variables, and either independent t-test or Mann-Whitney U for continuous variables. Missing data was left as null with no imputation. Any variable with > 10% missing data was not included in any model Multi-variable binary logistic regression (backward stepwise conditional) was undertaken. Probability of entry for any variable was 0.05, removal 0.1. A goodness-of-fit test was undertaken ((Hosmer-Lemeshow test) was utilised to assess the goodness of fit and stability of the model. Statistical analysis was performed using SPSS v.28. P  < 0.05 (2-sided) was considered significant. Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guidelines were followed for the reporting of the results [ 26 ].

A total of 537 patients were initially identified. After excluding patients that were deceased ( n  = 37), already on dialysis ( n  = 26), moved out of area ( n  = 23), or lost to follow up ( n  = 7), a total of 444 patients remained for analysis (Fig.  1 .). A total of 84 patients elected to participate in testing for the seroconversion analysis, who were analysed as a subgroup. Baseline characteristics of the final 444 prevalent KTRs are shown in Table  1 . They were predominantly male (60%), with a median age 58 years (Interquartile range [IQR]21.0) and baseline mean estimated glomerular filtration rate (eGFR) of 57 ml/min/1.73m 2 (Standard Deviation [SD] 21.9). Patients were primarily deceased donor recipients (69%) due to glomerulonephritis (50%) or diabetes (15%), with a median transplant vintage of 69.0 months (IQR 111.0). The primary immunosuppression regimen consisted of prednisolone (93%), mycophenolate (80%), and tacrolimus (72%).

Flow chart of patient inclusion and COVID-19 diagnosis

Vaccination status

Vaccination status was acquired for 440 (99%) patients. By study end, 95% ( n  = 423) of patients had received at least 1 vaccination. The number of patients that received 1,2,3, or 4 vaccine doses was 4 (1%), 75 (17%), 239 (54%) and 105 (24%) respectively. 17 (4%) patients remained unvaccinated throughout the study period. The vaccines administered included Pfizer BioNTech BNT162b2 (70%), AstraZeneca ChAdOx1 nCoV-19 (26%) and Moderna mRNA-1273 (3%) (Supp Table  2 ).

COVID-19 outcomes

COVID-19 was reported in 142 (32%) patients, and of these 54 (38%) required admission for COVID-19 with 10 (7%) deaths due to COVID-19. 17 (4%) patients died from any cause during the study period, with COVID-19 accounting for 59% of all deaths.

COVID-19 diagnosis

Univariate factors associated with acquiring COVID-19 are shown in Table  2 . On multivariable analysis, an increased risk of acquiring COVID-19 was associated with male sex (aOR 1.7, 1.093–2.701, p  = 0.019), younger age (aOR, 0.98, 0.964–0.994, p  = 0.006) and lower eGFR (aOR 0.99, 0.978–0.998, p  = 0.020), after adjusting for significant univariate associations, body mass index (BMI) and diabetes. (Table  3 ). Receipt of 3 or more doses of vaccine was protective (aOR 0.48, 95% CI 0.287–0.796, p  = 0.005).

COVID-19 mortality

Deaths from COVID-19 occurred throughout the study period, with 3 deaths in September 2021, 1 death in January 2022, 4 deaths in February 2022 and 1 death in both April and May 2022. Univariate analyses are shown in Table  2 . On multivariate analysis, increased mortality due to COVID-19 was associated with older age (aOR1.1, 95%CI 1.004–1.192, p  = 0.04), respiratory disease (aOR 14.2, 95%CI 1.825–110.930, p  = 0.011) and current or past smoking exposure (aOR 8.2, 95% CI 1.020-65.649, p  = 0.048) after adjusting for significant univariate associations, sex, BMI, diabetes, and vaccination (3 + doses). Vaccination of 3 or more doses was protective (aOR 0.6, 95% CI 0.007–0.523, p  = 0.011) (Table  3 ).

COVID-19 hospitalisation

Of those with reported COVID-19, 62 (44%) received sotrovimab and 11 (8%) received molnupiravir (Suppl Table 6.). 54 (38%) patients required hospitalisation for COVID-19, and 16 (11%) required intensive care unit (ICU) care. 33 (23%) patients required oxygen therapy. The maximum level of oxygen required was: low flow nasal prong oxygen in 13 (9%), high flow nasal prong oxygen in 4 (3%), non-invasive ventilation in 8 (6%) and invasive ventilation in 8 (6%) patients. Sotrovimab and molnupiravir were given in the community. When provided, neither were found to be protective for hospital admission ( p  = 0.11, p  = 021 respectively). Among hospitalised patients, those who received sotrovimab had evidence of protection for ICU admission (OR 0.2, 95%CI 0.035–0.886, p  = 0.030). Median length of hospital stay was 8 days (IQR ± 13). There was an association between prior sotrovimab use and shorter length of stay (5 vs. 10 days, p  = 0.027). Vaccination with 3 doses did not impact hospital admission ( p  = 0.32), ICU admission ( p  = 0.14) or length of stay (0.54).

Immunosuppression alteration occurred frequently in hospitalised patients (85%), as compared to those who were not hospitalised (10%). Hospitalisation with COVID-19 increased the odds of a reduction of immunosuppression (OR 50.5, 95% CI 18.211-139.883, p  < 0.001), however it was not significant for those who required an ICU admission among hospitalised patients ( p  = 0.41) or mortality ( p  = 0.64). Univariate factors associated with hospitalisation for COVID-19 are shown in Table  2 .

On multivariable analysis, increased hospitalisation was associated with older age (aOR 1.0, 95% CI 1.007–1.0092, p  = 0.021), lower eGFR (aOR 0.96, 95% CI 0.994 − 0.982, p  < 0.001) and receipt of a deceased donor graft (aOR 4.1, 95% CI 1.128–14.747, p  = 0.032), after adjusting for significant univariable associations, sex, BMI and vaccination (3 doses) (Table  3 ). Vaccination was not protective.

Seroconversion

84 patients underwent serological testing, including: 71 patients who had a single test, 12 who had 2 serial tests and 1 patient who had 3 serial tests. All but one patient, had a non-reactive Eleycs assay, indicating no prior exposure to COVID-19. The single patient with a reactive Eleycs assay was not known to have had prior COVID-19, however, was transplanted overseas with limited details prior to returning to Australia before the study period. This patient had no further serological evaluation and was excluded from the seroconversion analysis, resulting in 83 patients providing 97 serological tests assessed for vaccine-induced seroconversion.

All but 2/97 tests were collected prior to documented COVID-19. These two patients participated in serial testing. Prior to known COVID-19 they were Elecsys assay and QuantiVac ELISA negative. Post COVID-19 they remained Elecsys assay negative, however seroconverted on the QuantiVac ELISA. During this interval they received additional vaccinations, incrementing from 2 to 3 doses. As it is not possible to determine if these patients seroconverted due to wild type COVID-19 infection or vaccination, the serial samples prior to known COVID-19 were analysed. Of the remaining 95 tests, 5 were excluded based on QuantiVac ELISA results: 1 patient who did not have a QuanitVac ELISA processed on initial collection 1, but undertook repeat testing which was utilised, 3 patients with serial reactive tests performed after 2 and 3 doses of vaccine with no status change, therefore sampling after the 2nd dose was included, and 1 patient with 2 serial reactive tests, both after the 4th dose of vaccine and the earlier sample was included.

This resulted in 90 analysed samples: 1, 64, 21 and 4 samples after 1,2,3 and 4 doses of vaccine respectively (Suppl Table 3). Seroconversion rates after 1, 2, 3 and 4 doses were: 0, 22%, 48%, and 75% respectively (Suppl Table 4). Overall seroconversion rate at study end was 33% (27/83).

Univariate factors associated with COVID-19 diagnosis in this subgroup are shown in Supplementary Table 5. On multivariable analysis, after adjusting for univariate associations, in addition to age and diabetes, factors associated with an increased rate of acquiring COVID-19 included Asian place of birth (aOR 9.0, 95% CI 1.803–44.888, p  = 0.007) and higher dose of prednisolone (aOR 1.5, 95% CI 1.125–1.949, p  = 0.005). Seroconversion was protective (aOR 0.1, 95% CI 0.025–0.627, p  = 0.011), independent of vaccination of 3 + doses ( p  = 0.108) (Table  3 ).

The number of hospitalised patients in this subgroup was small ( n  = 6). No hospitalised patients demonstrated evidence of seroconversion, however this did not reach statistical significance ( p  = 0.539). No patient who died underwent serology assessment.

In this large, observational study of KTRs in Australia, during a period following community stay-at-home orders and two strain outbreaks, COVID-19 resulted in significant morbidity and mortality throughout the 2021–2022 pandemic period. Over 30% of the cohort developed breakthrough COVID-19, despite 78% receiving 3 or more doses of vaccine. Early monoclonal or antiviral treatment was provided to 51% of positive patients, however 38% of patients still required hospitalisation, with death occurring in 7% [ 15 , 16 ]. Overall seroconversion rates were low, with 3 doses of vaccine achieving a seroconversion rate of 48%.

Several risk factors for mortality amongst KTRs have been suggested, including older age, sex, cardiometabolic or respiratory co-morbidities and obesity [ 1 , 3 , 4 , 5 , 6 , 7 , 11 , 23 ].. This data supports older age, respiratory disease and smoking exposure may be independent factors for mortality for COVID-19 in KTRs. On systematic review and registry data analysis, no single co-morbidity had consistently been identified as a risk factor, other than age [ 3 , 11 ].

It has been suggested certain immunosuppression regimens are associated with increased COVID-19 mortality [ 1 , 6 , 7 ]. We did not find any effect of individual immunosuppressive agent on mortality, however there were high rates of baseline steroid (93%) and anti-metabolite (89%) use. A higher dose of prednisolone was associated with increased risk of acquiring COVID-19 in the serology subset.

Prior recommendations suggested temporarily altering immunosuppressive regimens during COVID-19 infections, and we noted high rates of alteration on hospitalisation in line with this trend [ 29 ]. There was no association with ICU admission or mortality among this group, and of patients who were not hospitalised, the majority did not have drug alteration (90%). Drug alteration, therefore, is likely reflective of a response to the severity of COVID-19. Current advice, with new strain evolution, suggests immunosuppression alteration is not required, particularly in the asymptomatic or those with a mild illness, with our data supportive of this [ 30 ]. Systematic review has not supported an association between immunosuppression and mortality, and there is limited comparative data to guide reduction of immunosuppression therefore decisions should be based on individualized assessment and the risk of rejection [ 11 , 23 ].

This data covered a period until May 2022, during a predominant Omicron outbreak from January 2022, whereby most patients had received 3 or more vaccine doses [ 15 , 16 ].. The Omicron era heralded decreased virulence, however the neutralising capability after 3 doses of vaccine was suggested to be diminished [ 12 ]. In this data, vaccination of 3 or more doses was protective for death and acquiring COVID-19, with no effect on hospitalisation, ICU admission or length of stay. In the serology subset, seroconversion, independent of dose of vaccination, was protective for acquiring COVID-19. Mortality rates during periods of Omicron predominance among solid organ transplant recipients have been reported to be 3 − 4%, however hospitalisation rates have remained 24–32%, with ICU admission rates of 28–36% [ 24 , 31 ]. Ongoing hospitalisation rates remain a concern for KTRs and further data regarding vaccine schedule optimisation and seroconversion assessment, independent of vaccination dose number, is needed.

This data demonstrated protection against death, reduced rates of ICU admission and length of stay with the use of sotrovimab, with no protective effects of molnupiravir. Our study reflects a period where sotrovimab was the primary agent of choice in early COVID-19 disease (approved August 2021) as opposed to molnupirovir (approved January 2022), likely influencing our results [ 27 , 28 ].

Current Australian recommendations do not recommend either sotrovimab or molnupirovir. Tixagevimab plus cilgavimab (Evusheld) has also lost its recommendation. Nirmatrelvir plus ritonavir (Paxlovid) retains its conditional recommendation, however, its use in KTRs is challenging due to effects on calcineurin inhibitor levels [ 25 ]. Remdesivir remains recommended only for patients requiring oxygen due to symptomatic COVID-19. There remains a paucity of agents effective at treating early COVID-19 in renal transplant recipients.

This data supports concern surrounding ongoing mortality and hospitalisation risk for KTRs, in the context of low seroconversion rates despite increasing vaccination dose schedule. This reiterates vaccination of at least 3 doses, and potentially evidence of seroconversion, is protective, however, in the absence of effective early treatments, encouragement of protective behaviours, such as social distancing, mask compliance and hand hygiene should continue.

This study is limited as a single centre and results are not generalisable. As with all observational data our analyses are limited to associations. Those undertaking the serology assessment were a self-selected population, which is likely to result in unmeasured patient bias, especially with regards to protective behaviours. They were highly vaccinated, with more than 90% receiving 3 or more doses. In addition, our data spanned two predominant strain periods, Delta and Omicron, and we were not able to specify strains in individual patients. While it is likely we captured most noted infections due to mandatory government reporting, cases could have been omitted if patients did not note an infection, obtain testing, or report a positive test, resulting in potential underdiagnosis of mild and asymptomatic cases. In addition, seroconversion does not always reflect in-vivo activity of antibodies and we did not assess the effect of waning immunity over time.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Sahota A, Tien A, Yao J, Dong E, Herald J, Javaherifar S, et al. Incidence, risk factors, and outcomes of COVID-19 infection in a large cohort of solid organ transplant recipients. Transplantation. 2022;106(12):2426–34.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Hall VG, Solera JT, Al-Alahmadi G, Marinelli T, Cardinal H, Poirier C, et al. Severity of COVID-19 among solid organ transplant recipients in Canada, 2020–2021: a prospective, multicentre cohort study. CMAJ. 2022;194(33):E1155–63.

Article   PubMed   PubMed Central   Google Scholar  

Hilbrands LB, Duivenvoorden R, Vart P, Franssen CFM, Hemmelder MH, Jager KJ, et al. COVID-19-related mortality in kidney transplant and dialysis patients: results of the ERACODA collaboration. Nephrol Dial Transpl. 2020;35(11):1973–83.

Article   CAS   Google Scholar  

Caillard S, Chavarot N, Francois H, Matignon M, Greze C, Kamar N, et al. Is COVID-19 infection more severe in kidney transplant recipients? Am J Transpl. 2021;21(3):1295–303.

Udomkarnjananun S, Kerr SJ, Townamchai N, Susantitaphong P, Tulvatana W, Praditpornsilpa K, et al. Mortality risk factors of COVID-19 infection in kidney transplantation recipients: a systematic review and meta-analysis of cohorts and clinical registries. Sci Rep. 2021;11(1):20073.

Gerard AO, Barbosa S, Anglicheau D, Couzi L, Hazzan M, Thaunat O, et al. Association between Maintenance Immunosuppressive Regimens and COVID-19 mortality in kidney transplant recipients. Transplantation. 2022;106(10):2063–7.

Requião-Moura LR, Modelli de Andrade LG, de Sandes-Freitas TV, Cristelli MP, Viana LA, Nakamura MR, et al. The mycophenolate-based Immunosuppressive Regimen is Associated with increased mortality in kidney transplant patients with COVID-19. Transplantation. 2022;106(10):e441–51.

Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. 2020;383(27):2603–15.

Article   CAS   PubMed   Google Scholar  

Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021;384(5):403–16.

Sanders JF, Bemelman FJ, Messchendorp AL, Baan CC, van Baarle D, van Binnendijk R, et al. The RECOVAC Immune-response study: the immunogenicity, tolerability, and Safety of COVID-19 vaccination in patients with chronic kidney Disease, on Dialysis, or living with a kidney transplant. Transplantation. 2022;106(4):821–34.

Opsomer R, Kuypers D. COVID-19 and solid organ transplantation: finding the right balance. Transpl Rev (Orlando). 2022;36(3):100710.

Article   Google Scholar  

Kumar D, Hu Q, Samson R, Ferreira VH, Hall VG, Ierullo M, et al. Neutralization against Omicron variant in transplant recipients after three doses of mRNA vaccine. Am J Transpl. 2022;22(8):2089–93.

(ATAGI) ATAGoI. ATAGI updated recommendations for a winter dose of COVID-19 vaccine. In: Care DoHaA, editor. 2022.

Edwards BB, Rehill R, Zhong PEL, Killigrew F, Riquelme Gonazalez A, Sheard P, Zhi E. R., Philips, T. Variation in policy response to COVID-19 across Australian states and territories. Blavatinik School Of Government: University of Oxford; 2022.

Google Scholar  

Australian Bureau of Statistic. COVID-19 Mortality by wave2022. https://www.abs.gov.au/articles/covid-19-mortality-wave .

South Western Sydney Local Health District Public Health Unit. Personal Communication: Epidemiology of COVID-19 in SWSLHD. 2023.

NSW Government. Public Health (COVID-19 Additional Restrictions for Delta Outbreak) Order (No 2) Amendment Order 2021. In: Research MfHaM, editor. 2021.

Australian Goverment. Priority groups for COVID-19 Vaccination Program Phase 1B. In: Health, editor. Online2021.

Australian and New Zealand Dialysis and Transplant Registry. ANZDATA 45th Annual Report 2022 (Data to 2021). 2022.

ID (Informed Decisions). [Data Extraction from Australian Bureau of Statistic and Census]. https://home.id.com.au/ .

United States Food and Drugs Administration. Elecsys (R) Anti-SARS-CoV-2 Intruction for Use. 2020.

United States Food and Drugs Administration. Anti-SARS-CoV2 ELISA (IgG) Instructions for Use. 2022.

Nimmo A, Gardiner D, Ushiro-Lumb I, Ravanan R, Forsythe JLR. The global impact of COVID-19 on solid organ transplantation: two years into a pandemic. Transplantation. 2022;106(7):1312–29.

Anjan S, Khatri A, Viotti JB, Cheung T, Garcia LAC, Simkins J, et al. Is the Omicron variant truly less virulent in solid organ transplant recipients? Transpl Infect Dis. 2022;24(6):e13923.

National Clinical Evidence Taskforce. Australian Guideline for the clinical care of people with COVID-19. 2022;v70.1.

von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The strengthening the reporting of Observational studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med. 2007;4(10):e296.

Australian Government. TGA approves new COVID-19 treatment for use in Australia. In: Health and Aged Care, editor. 2021.

Government A. TGA Provisionally approves Merck Sharp and Dohme (Australia) Pty Ltd’s oral COVID-19 treatment, LAGEVRIO (Molnupiravir). 2022.

Maggiore U, Abramowicz D, Crespo M, Mariat C, Mjoen G, Peruzzi L, et al. How should I manage immunosuppression in a kidney transplant patient with COVID-19? An ERA-EDTA DESCARTES expert opinion. Nephrol Dial Transpl. 2020;35(6):899–904.

Gandolfini I, Crespo M, Hellemans R, Maggiore U, Mariat C, Mjoen G, et al. Issues regarding COVID-19 in kidney transplantation in the ERA of the Omicron variant: a commentary by the ERA Descartes Working Group. Nephrol Dial Transpl. 2022;37(10):1824–9.

Solera JT, Arbol BG, Alshahrani A, Bahinskaya I, Marks N, Humar A, et al. Impact of vaccination and early monoclonal antibody therapy on Coronavirus Disease 2019 outcomes in Organ Transplant recipients during the Omicron Wave. Clin Infect Dis. 2022;75(12):2193–200.

Download references

Acknowledgements

SWSLHD Renal Unit and Research Staff. NSW Health Pathology Laboratories– Liverpool. All the patients who participated in the study.

Locally (non-peer reviewed) funded (renal department trust fund).

Author information

Authors and affiliations.

University of Western Sydney, Sydney, NSW, Australia

Daniela Potter & Angela Makris

University of New South Wales, Sydney, NSW, Australia

Department of Renal Medicine, Liverpool Hospital, Liverpool, Sydney, NSW, Australia

Daniela Potter, Jason Diep, Colleen Munro, Noelle Lin, Ramon Xu, Jeffrey Wong & Angela Makris

Department of Microbiology and Infectious Diseases, NSW Health Pathology, Liverpool, NSW, Australia

Robert Porritt, Michael Maley & Hong Foo

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the work, as well as reading and approving the final manuscript. DP is the primary corresponding author and was responsible for the design of study and the primary data interpretation and drafting of work. AM was responsible for study conception, study design, data analysis and contributed to draft revisions of the work. JP, CM, NL and RX were responsible for data acquisition and analysis, as well as draft revision. JW was responsible for data interpretation and draft revisions. RP was the primary technical laboratory advisor and analyst. MM and HF were both advisors on laboratory, infectious and epidemiological components of the study and contributed to draft revisions.

Corresponding author

Correspondence to Daniela Potter .

Ethics declarations

Ethics approval and consent.

Written informed consent for this component of the study was obtained from those willing to participate. This study was approved by the SWSLHD Human Research and Ethics Committee (Approval Reference: 2019/STE00860).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Potter, D., Diep, J., Munro, C. et al. A prospective observational cohort study of covid-19 epidemiology and vaccine seroconversion in South Western Sydney, Australia, during the 2021–2022 pandemic period.. BMC Nephrol 25 , 131 (2024). https://doi.org/10.1186/s12882-024-03560-8

Download citation

Received : 16 September 2023

Accepted : 26 March 2024

Published : 12 April 2024

DOI : https://doi.org/10.1186/s12882-024-03560-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Kidney Transplantation
• COVID-19 Epidemiology
• COVID-19 Vaccination
• COVID-19 Vaccination Seroconversion

BMC Nephrology

ISSN: 1471-2369

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Reference Manager
• Simple TEXT file

People also looked at

Original research article, the impact of observers’ beliefs on the perceived contribution of a research lesson.

• 1 Experiential Learning Office–Engineering Faculty, McMaster University, Hamilton, ON, Canada
• 2 CIAE, Institute of Education, University of Chile, Santiago, Chile

Recently, Lesson Study (LS) has gained popularity in countries worldwide because of its potentially positive effects on teachers’ practices (e.g., reflection, cooperation, and pre-service development) and students’ learning. However, despite global interest in LS’s implementation, an important gap exists between Japanese LS and its implementation in other countries, which may be due to several reasons, such as differences in culture and educational systems or teachers’ beliefs. In this study, we examined the effect of teachers’ beliefs on their evaluation of LS in Chile. We administered a questionnaire to 94 teachers who participated in the Research Lesson (RL) as observers. The questionnaire assessed teachers’ beliefs and RL’s contributions to knowledge of the subject matter, instructional strategies, monitoring skills, lesson planning, and student understanding. Using a stepwise logistic regression, after controlling for sex and occupation, we found that the observers’ beliefs influenced their perceptions of RL’s contributions to monitoring, assessment, and instruction. Participants with student-focused beliefs were more likely to find that RL contributed to their monitoring skills and ability to assess students’ understanding of content. The results regarding instructional strategies were mixed. Our findings can help devise strategies to increase the effectiveness of LS implementation. For example, by designing two types of LS, one adapted to teachers with student-focused teaching beliefs and the other to teacher-focused teaching beliefs. To the best of our knowledge, this dual strategy is not part of LS implementation, at least in Chile. LS teams could easily explore this dual strategy, which could improve teachers’ professional development.

1 Introduction

Lesson Study (LS) began in 1880 in Japan with the goal of reproducing the best practices in teaching ( Isoda, 2015 ) and is a collaborative teaching improvement process designed to build strong and productive communities of teachers who share and learn from each other. Many countries have introduced this methodology ( Fujii, 2014 ). Evidence indicates that LS accelerates the production of effective lessons and, in particular, helps develop open-ended approaches with lessons that aim to develop higher-order skills ( Inprasitha, 2015 ). Additionally, Vermunt et al. (2019) concluded that the LS approach to pedagogical development fosters meaning-oriented teacher learning, which can be explained by LS’s strong focus on analyzing and understanding pupils’ learning. In particular, teachers in schools implementing LS have reported more meaning- and application-oriented learning than teachers from schools without LS experience ( Vermunt et al., 2019 ).

Enhancing the dissemination and awareness of LS’s benefits is important, as Chokshi and Fernandez (2004) found that schools allocate time for LS initiatives once teachers recognize the potential of LS. The potential benefits of LS include various pedagogical aspects and students’ learning outcomes. Researchers have studied numerous different effects of LS, including teachers’ increased self-efficacy ( Mintzes et al., 2013 ; Schipper et al., 2018 ; Vermunt et al., 2019 ), teachers’ content knowledge ( Fernandez and Robinson, 2006 ; Lewis and Perry, 2014 ; Juhler, 2016 ), teaching practices and teachers’ skills ( Fernandez and Robinson, 2006 ; Fernandez, 2010 ; Vermunt et al., 2019 ), teachers’ beliefs ( Fernandez and Robinson, 2006 ; Lewis and Perry, 2015 ; Yakar and Turgut, 2017 ) and pre-service teachers’ preparation ( Marble, 2006 , 2007 ; Suh and Fulginiti, 2012 ). In Chile, there used to be funds to implement this methodology, but due to a significant funding reduction, only a few institutions continue to work with the methodology ( Estrella et al., 2018 ).

However, a notable gap in the implementation of LS exists between Japan and other countries owing to divergent cultures and educational systems. Countries implementing LS outside of Japan have been found to not fully capture its key components ( Fujii, 2014 ). For instance, in the US, most LS teams do not share their learning with their peers, losing the opportunity to share and discuss their experiences with their educational community ( Whitney, 2020 ). Similarly, in England, Seleznyov (2020) reported that LS practices were diluted over time and identified various impediments, such as a lack of time and excessive teacher workload, a focus on demonstrating short-term impact, and a lack of teacher research skills. Additionally, Godfrey et al. (2019) found that engagement and reported learning decreased when teachers in England had to commit their own time to the LS process.

1.1 Research lessons

Research lessons (RLs) are critical to LS, which involve teachers coming together to observe and discuss lessons. Some benefits of RLs include opportunities for teachers to observe experienced teachers as a form of modeling their own learning experiences. RLs provide valuable examples of meaningful curricula and standard discussions, encouraging teachers to align their teaching with policy- and research-based recommendations ( Lumpe et al., 2014 ). Furthermore, Dudley et al. (2019) studied the implementation of RLs in mathematics within a school system in England and concluded that by focusing on student learning and recursive cycles of LS, teachers could develop the curriculum and raise standards while supporting the creation of the necessary conditions for learning. This transition to a student-centered education was also part of a two-year project in which teachers focused on students’ mathematical reasoning and sense-making rather than on other teacher actions or teaching ( Wessels, 2018 ).

Another potential benefit of RLs is teacher dialogue, consisting of more meaningful reflections. During the implementation of RLs in Ireland, teachers engaged in meaningful dialogues about pedagogy and student learning, fostering deeper levels of reflection on their understanding and practice knowledge ( McSweeney and Gardner, 2018 ). Similarly, a study in Singapore concluded that after four RLs and discourse analyses, a balance was achieved in mathematical representation while escalating the construction of meaning-making ( Fang et al., 2019 ).

Nonetheless, the field’s understanding of RLs remains limited, especially regarding methods of sharing work with others. Whitney (2020) examined cases of LS teams in the US and found that most LS teams did not distribute their learning to the field. Individuals external to the team observed the RL and engaged in subsequent post-lesson discussions.

This study has two objectives. First, we gathered the evaluations of observers outside the LS team when they attended an RL in Chile. Second, we examined how observers’ beliefs affected their evaluation of RL. Thus, our research question (RQ) was, “To what extent do teachers’ beliefs affect the perceived contributions of an RL in the context of an LS process?”

1.2 The Chilean context and teachers’ predisposition to observation

Our first study aim was to gather the evaluations of observers attending an RL in Chile. Participation as an observer in an RL may share similarities with teachers’ own evaluation processes, particularly in the context of observing pedagogical practices and sharing best practices with peers. Potential apprehension could exists among educators because of the history of teacher performance evaluations in Chile. This subsection briefly describes the historical perspective of the Chilean teacher evaluation system.

Chile’s experience with teacher performance evaluation is distinctive, particularly considering its past political developments. During Chile’s dictatorship (1973–1989), the teaching profession experienced a significant weakening, resulting in a substantial unemployment rate among teachers. Following the return to democracy in the 1990s, concerted efforts were made by the national government, in collaboration with teachers’ unions, to rectify the adverse consequences of past politics ( Avalos and Assael, 2006 ; Assaél and Cornejo, 2018 ).

One contentious aspect of these remedial efforts was the introduction of an annual grading system exclusively within public schools (municipality-dependent), which has frequently been perceived as punitive and arbitrary by educators ( Avalos and Assael, 2006 ; Taut et al., 2011 ; Assaél and Cornejo, 2018 ). Despite some modifications over the years, the prevailing sentiment among teachers was that these evaluations remained unjust and were penalizing. Moreover, these evaluations impose additional work burdens on teachers, exacerbated by the prevailing practice of unpaid overtime among educators in Chile. This evaluation is referred to as the “Teaching Evaluation” and is mandated by law.

In 2016, a significant change occurred when a second method of evaluating teachers was introduced, referred to as the “Teaching Career Evaluation.” This evaluation encompassed all teachers associated with schools that received state funding. Consequently, educators from state-funded educational institutions were included in the evaluation process ( Martínez, 2022 ; Colegio de Profesores, 2023 ). Importantly, the evaluation was not designed to supplement the existing evaluation system but introduced an additional evaluation more closely associated with the years of a teacher’s career. Consequently, teachers within the public system were subject to evaluations from two distinct systems, increasing their workload and the demands placed on them.

Both evaluations were designed to reinforce the teaching profession and contribute to improving the quality of education. The “Teaching Evaluation” comprises four key components: (1) a self-assessment rubric; (2) an interview conducted by a peer evaluator; (3) a third-party reference report; and (4) a portfolio showcasing an in-video pedagogical performance. Each component includes a rating assigned to the teacher, classifying teachers into different performance levels: outstanding, competent, basic, or unsatisfactory. The “Teaching Career Evaluation” includes (1) an evaluation of specific and pedagogical knowledge and (2) the same portfolio used in the “Teaching Evaluation.” According to Alvarado (2012) , the teachers’ performance and their students’ academic performance on math- and language-standardized national tests have been positively correlated.

Teachers who received favorable evaluations progress to higher tiers within the Teaching Career framework. The upper tiers often entail augmented salary packages and improved working conditions ( Manzi et al., 2011 ). Engagement in these evaluative assessments may be a financial necessity for some teachers due to the comparatively modest salaries that educators in Chile receive ( Assaél and Cornejo, 2018 ; Avalos-Bevan, 2018 ), rather than an opportunity to foster reflection between peers regarding their pedagogical practice and work ( Assaél and Cornejo, 2018 ; Avalos-Bevan, 2018 ).

Consequently, public school teachers in Chile have experience with being observed and engaging in discussions with others regarding their teaching practices. However, these experiences may be biased toward an emphasis on assessment and evaluation, which, in turn, translates into an economic impact.

2.1 Participants

The participants in this study were observers of at least one RL hosted by different LS teams. The survey and methodology were approved by our ethics committee. An open invitation to complete the survey was sent to the participants of three different RLs sponsored by the Center for Advanced Research in Education (CIAE). Participants answered the survey between March and May 2020. All RLs were Science, Technology, Engineering and Math (STEM)-related and planned for primary education, with an open invitation to elementary and secondary school teachers, academics, researchers, educators, future teachers, school administrators, and educational authorities. The teachers who taught the lesson were experienced teachers. Two were held in Santiago, Chile, in 2017 and 2018, and the third in Valparaíso, Chile, in 2019. The last RL was conducted within the context of the XI Regional Conference on Lesson Studies.

A total of 97 participants answered the survey, and 92 agreed to participate, resulting in a response rate of 94.84%. Of the 92 participants, 62 were women (67.39%), and 30 were men (32.61%). Regarding the participants’ occupations, 70 were in-service teachers (76.08%), five were student teachers (5.43%), and 17 (18.47%) had other roles related to education, such as academics, members of a school board, and principals.

Regarding the participants’ observation experience, the information available is limited. As explained in the previous section, teachers in Chile have experience being observed; however, we do not have evidence of the level of training that the participants received as observers before attending the RL.

2.2 Instruments

A questionnaire was developed to assess participants’ perceptions of the RL’s contributions to their skills and beliefs.

2.2.1 RL contribution to pedagogical practices, overall LS evaluation, and future participation

Participants answered five Likert-scale questions regarding the contribution level of RL to their (1) knowledge of the subject matter, (2) instructional strategies, (3) skills to monitor the level of understanding of the students, (4) lesson planning skills, and (5) abilities to assess the students’ understanding of the content. Participants evaluated the contribution of RL by choosing one of five options (from 1 =  not at all to 5 =  extremely relevant ). This type of question, which includes scale with neutral option, has been used in previous studies to evaluate the impact of LS on teachers’ professional development ( Alamri, 2020 ), participants’ perception of changes in intercultural competence for lesson study ( Sakai et al., 2022 ).

Participants also answered a Likert-scale question about their overall evaluation of RL that was rated from very poor (1) to very good (5), as well as a question regarding whether they would participate again in an RL session, which was rated as yes , maybe , or no .

2.2.2 Teachers’ beliefs about learning

The Teacher Beliefs Interview (TBI), a semi-structured interview protocol, was developed to describe and map various epistemological beliefs held by science teachers ( Luft and Roehrig, 2007 ). Based on the TBI, teachers’ beliefs range from teacher-focused to student-focused. The TBI includes seven items assessing their knowledge and learning beliefs. Depending on the teachers’ responses, their beliefs were coded into five categories: (1) t raditional , with a focus on information, transmission, structure, or sources; (2) instructive , with a focus on providing experiences, teacher focus, or teacher decisions; (3) transitional , with a focus on teacher/student relationships, subjective decisions, or affective responses; (4) responsive , with a focus on collaboration, feedback, or knowledge development; and (5) reform-based , focusing on mediating student knowledge or interactions. The traditional and instructive categories are teacher-focused, whereas the responsive and reform-based categories are student-focused. This instrument has been employed in prior studies about LS and teachers’ beliefs. In the study by Tupper (2022) , the TBI was utilized alongside other instruments to assess the relationship between teachers’ beliefs, self-efficacy, and professional noticing with multiple lesson study experiences. Similarly, Fortney (2009) incorporated the TBI, among other instruments, to investigate how teachers reconcile student-centered instruction with their preexisting traditional beliefs about teaching. Finally, Yakar and Turgut (2017) also used the TBI and found that through lesson study preservice teachers changed their beliefs toward more student-centered.

Of the seven questions on TBI, four were related to learning beliefs. The other three items were related to knowledge beliefs. Three questions regarding learning were included in the questionnaire (the fourth question was excluded because it asked about science learning). Rather than adhering to the open-ended format, we modified the approach by incorporating the three questions that asked the participants to choose between two real-life strategies using concrete examples. These three questions were related to the participants’ beliefs about learning. This adaptation led to a structured multiple-choice format, providing participants with clear examples while maintaining the integrity of the TBI’s underlying principles. Table 1 shows the questions, examples, and mapping suggested by Luft and Roehrig (2007) for assessing the participants’ beliefs.

Table 1 . Questions in the survey and the corresponding response mapping based on TBI.

The Cronbach’s alpha was calculated to measure the reliability of the instrument. The results showed that the instrument had an estimated α equal to 0.68.

2.3 Analysis

To answer the research question, a descriptive analysis and stepwise logistic regression model were performed for each dependent variable. Five independent models were tested, one for each dependent variable.

2.3.1 Dependent variable—RL contribution

The five dependent variables were the RL contributions to knowledge, instruction, monitoring, planning, and assessment. Each variable was binary, with value 1 being assigned when the participant evaluated the RL’s level of contribution as either very or extremely and a value of 0 being assigned for all other ratings.

2.3.2 Independent variables

Participants’ beliefs were considered independent variables. Thus, there were three independent variables, each corresponding to one of the three questions regarding how participants (1) maximized learning in their classrooms, coded as max_learning; (2) knew when their students understood, coded as understanding; and (3) knew when learning occurred in their classrooms, coded as learning.

As with the dependent variables, each independent variable was binary, with the variable being scored as Student Focused (SF) when participants chose strategies that were mapped as student-focused and/or transitional on the TBI ( Luft and Roehrig, 2007 ) and Not Student Focused (NSF) in any other case. In other words, if a participant chose a responsive (R) and/or reform-based (RB) strategy alongside a transitional strategy, the participant was coded as SF. In all other cases, the participant was coded as NSF. Table 2 shows the scenarios in which participants were classified as having SF. The participants were classified as having NSF for all other combinations of selected strategies.

Table 2 . Categorization of participants’ answers based on number of strategies selected.

2.3.3 Other explanatory variables

Two other independent variables are also considered. The first was sex (woman or man), while the second was the participant’s occupation ( in-service teacher , student teacher , or other ).

The stepwise logistic regression selection considered a full model with all the variables involved (max_learning, understanding, learning, sex, and occupation) and a base model with only sex as an independent variable ( Figure 1 ).

Figure 1 . Models explaining the level of contribution of the RL used in the stepwise logistic regression. (A) Full model with all the variables involved. (B) Base model with sex as the independent variable.

The notation for the full model is:

For the base model, it is:

Model selection used the R function stepAIC ( Zhang, 2016 ), which iteratively adds and removes predictors to the base model. Thus, the best model was selected based on its lower AIC value ( Akaike, 1974 ). This process was performed independently for each dependent variable: the level of contribution of RL to (1) knowledge, (2) instruction, (3) monitoring, (4) planning, and (5) assessment. Finally, when a variable was significant, the odds ratio (OR) was reported as a measure of the association between exposure and outcome ( Szumilas, 2010 ).

3.1 LS evaluation and future participation

Overall, the LS received a positive evaluation: 76.09% of teachers evaluated RL as either good or very good ( N  = 70). When asked if they would participate again in RL, 86.95% reported that they would. The distribution of the answers is shown in Figure 2 .

Figure 2 . Number of answers of the participants’ overall evaluation of RL and future participation.

3.2 Participants’ beliefs

Table 3 summarizes the distribution of teachers’ beliefs after categorizing them as either SF or NSF.

Table 3 . Distribution of the observers’ beliefs for each independent variable, based on the TBI categories.

3.3 Reported RL impact on pedagogical aspects

Table 4 shows the level of contribution of RL to pedagogical practices as reported by the participants. The aspects with the highest levels of impact were planning and assessment, while the aspects with the lowest impact on teachers’ pedagogical practices were monitoring and instruction.

Table 4 . Evaluation of the contribution of RL to different aspects.

3.4 Impact of participants’ beliefs on RL contribution

Table 5 summarizes the distribution of the level of contribution by belief for the three independent variables. Results in Table 5 suggest that participants’ beliefs might influence the perception of the RL contribution. The data imply a consistent trend where observers with SF beliefs tend to value the RL contribution more than those with NSF beliefs. Moreover, the differences are more or less pronounced depending on how those underlying beliefs are measured (i.e., max learning, learning, or understanding).

Table 5 . Distribution of evaluation of the contribution of the RL based on the category of observers’ beliefs.

Differences in the RL’s contribution based on the observers’ beliefs were observed. Regarding monitoring skills, 72.34% of those with SF beliefs (measured with the variable max_learning) valued the contribution of RL as very or extremely , while less than half of observers with NSF beliefs shared the same valuation.

In terms of assessing skills, 76.60% of participants with SF beliefs (measured with the variable max_learning) rated the contribution as very or extremely , compared to 55.56% of those who were NSF.

The results for the instructional skills varied. When considering the observers’ beliefs measured by the variable max_learning, 74.47% of the observers with SF beliefs valued the contribution of RL positively, in contrast to 48.89% of those with NSF. In contrast, when measuring beliefs by the variable learning, 71.05% of observers with NSF beliefs valued the contribution as positive, whereas 55.56% of observers with student-focused SF beliefs shared the same view.

3.4.1 Knowledge

The summary of the logistic regression model is shown in Table 6 .

Table 6 . Logistic regression summary for the selected model illustrating the contribution of RL to knowledge.

The selected model was the base model (AIC = 121.87), with sex as the only independent variable. However, sex was not a significant predictor of the probability of evaluating contribution to RL ( p  = 0.14; Table 6 ). Thus, the participants’ beliefs did not add information when explaining the contribution of RL to their knowledge skills.

3.4.2 Monitoring

The summary of the logistic regression model is shown in Table 7 .

Table 7 . Logistic regression summary for the selected model showing the contribution of RL to monitoring.

The model that best explained the contribution of RL to monitoring skills was the independent variable max_learning after controlling for sex and occupation (AIC = 121.44). The variable max_learning had a significant positive estimated value ( p  = 0.035), whereas the other variables (i.e., sex and occupation) were not significant. In other words, when holding the variables of sex and occupation constant, participants with SF beliefs (measured through the variable max_learning) were more likely to rate the contribution of RL on their monitoring skills as very or extremely (OR = 1.99, 95%CI [1.06, 3.85]).

3.4.3 Planning

The selected model considered max_learning and understanding as independent variables, controlling for sex (AIC = 108.29). Table 8 summarizes the corresponding logistic regression results.

Table 8 . Logistic regression summary for the selected model showing RL’s contribution to planning.

In the selected model, the three independent variables had nonsignificant estimated values. The variables max_learning and sex had positive values, while understanding had an estimated negative value.

3.4.4 Assessing

The model that best explains the contribution to assessment is the independent variable max_learning, after controlling for sex and occupation (AIC = 117.66). Table 9 summarizes the corresponding logistic regression results.

Table 9 . Logistic regression summary for the selected model showing RL’s contribution to assessing.

In the selected model, max_learning had a positive and significant estimated value ( p  = 0.039). Thus, when holding the variables of sex and occupation constant, observers with SF beliefs, measured through the variable max_learning, were more likely to value the contribution on assessing as very or extremely , in contrast to the observers with NSF beliefs (OR = 2.01, 95%CI [1.05, 3.98]).

3.4.5 Instruction

The model that best explains the contribution to instruction had the independent variables max_learning and learning controlled for by sex (AIC = 114.68). Table 10 summarizes the corresponding logistic regression results.

Table 10 . Logistic regression summary for the selected model showing RL’s contribution to instruction.

In this model, the variable max_learning had a significant, positive estimated value ( p  = 0.002), whereas the variable learning had a significant, negative estimated value ( p  = 0.007). Thus, when holding the variable of sex constant, observers with beliefs which were SF, measured through the variable max_learning, were more likely to value the contribution on assessing as very or extremely , in contrast to the observers with NSF beliefs. Further when controlling for the variable of sex, observers with beliefs that were NSF, measured through the variable learning, were more likely to value the contribution on instruction as very or extremely .

4 Discussion and conclusion

This study has two aims. First, we aimed to collect evaluations of observers outside the LS team when attending the LS Research Lesson stage. Second, we examined how the observers’ beliefs affected their evaluations of RL. Thus, the research question (RQ) was, “To what extent do teachers’ beliefs affect the perceived contribution of a Research Lesson in the context of a lesson-study process?”

Evidence indicates that LS teams do not share their learning with their peers, thereby losing opportunities to share and discuss their experiences with the educational community. An RL provides teachers new opportunities to observe experienced teachers to model their own learning experiences and discuss curricula and standards, helping persuade teachers to align their teaching with policy and evidence-based recommendations. However, little is known about the RS stage, particularly regarding how to share work with others. Whitney (2020) studied cases of LS study teams in the US and found that most LS teams did not distribute their learning to the field, and instances of people outside the team watching the research lesson and participating in post-lesson discussions existed.

Nonetheless, a relationship has been observed between beliefs and classroom practices ( Skott, 2001 ; Stipek et al., 2001 ; Schoenfeld, 2011 ). Moreover, these beliefs could affect teachers’ changes in the context of professional development programs ( Maass, 2011 ). de Vries et al. (2013) found a relationship between teachers’ participation in Continuous Professional Development programs and their beliefs.

This paper aims to reduce the existing gap between the implementation of LS in Japan and in other countries. When considering the implementation of LS in Chile, it is critical to look at teachers’ predisposition to observation. Historically, teachers of public schools have experience in being observed; however, class observation has been associated with assessment and evaluation. Although this research does not specifically inquire about teachers’ beliefs regarding observation, it is an important element to consider in the teaching culture of Chile. For example, by conducting RL observation, Chilean teachers could focus more on evaluating aspects that are important in their own teaching evaluation, such as class planning and student assessment.

The results show that, in the Chilean context, when explaining the perceived contribution of the RL, observers’ beliefs improved the models’ performance, contributing to instruction, monitoring, planning, and evaluation. The current findings are beneficial, as they provide insights into how different participants value their contributions to RL. Furthermore, LS teams should consider this when issuing open invitations to external individuals to observe RL. Understanding the influence of participants’ beliefs on their evaluation of a situation is valuable. In addition, these findings can help develop strategies for more effective LS implementation, such as designing two types of LS, one adapted to teachers with beliefs about teaching focused on students and the other adapted to teaching beliefs focused on teachers. To the best of our knowledge, this dual strategy is currently not a part of LS implementation, at least in Chile. LS teams could easily explore this dual strategy, which could improve teachers’ professional development. In terms of practical implications, we believe that by adapting different types of LS, the whole group collaboration within each RL could lead to a richer discussion on aspects of the lesson. It would bring to the forefront those aspects that might not initially appear to contribute significantly. As a result, it would promote reflection among observers based on their own beliefs. Moreover, this dual strategy could help emphasize the benefits of the methodology and strengthen the overarching goals of the LS by highlighting those aspects that are less appreciated within each group.

LS encompasses multiple stages and requires dedication from the LS team. This paper provides initial insights into a single stage of LS based on a limited sample. Nonetheless, future research should focus on the whole process, considering a more holistic perspective of the LS implementation. Similarly, future research could explore the beliefs of the LS team, rather than solely focusing on the observers of the RL. Additionally, considering other contextual factors such as school culture, previous observation experiences, and administrative resources to participate in Professional Development activities. For instance, it would greatly enrich the analysis of future research to know the level of expertise participants have in conducting observations. This would allow us to understand more deeply which aspects teachers highlight depending on their experience in observation. Finally, the students’ behavior, which was outside of the scope of this work, may be affected by changes in their routine, such as the physical space allowing for a large number of observers or different teachers than those they are used to. On the other hand, we used the TBI ( Luft and Roehrig, 2007 ) in a novel manner, which was developed to be used as an interview. We used the open-ended TBI questions and provided the participants with concrete examples based on the framework developed by Luft and Roehrig (2007) , thereby framing the questions in a multiple-choice format instead of an open-ended approach. While this method proved helpful in explaining the differences in the perceived contribution, future research should study alternative instruments for assessing participants’ beliefs. In particular, the questionnaire instrument has a Cronbach’s alpha value of 0.68, which is close to the threshold of reliability; the reliability of this instrument might impact the measurement properties for this study, and the results should be taken with caution. Furthermore, a qualitative approach could provide insights into how and why those beliefs influence the participants’ perception of RL’s contributions.

Finally, in the Chilean context, opportunities for collaboration and peer observation that extend beyond evaluation purposes are important. The current study is an example of how people outside the LS team can benefit from different pedagogical aspects of this methodology. Future research should assess changes in teachers’ beliefs to determine whether the reported value of RL affects their daily practices.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Comité de Ética de la Investigación (CEI) de la Facultad de Ciencias Sociales de la Universidad de Chile https://facso.uchile.cl/facultad/comites/comite-de-etica-de-la-investigacion . The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

DC: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Writing – original draft, Writing – review & editing. EM-D: Formal analysis, Writing – original draft, Writing – review & editing, Investigation. RA: Funding acquisition, Supervision, Writing – original draft, Writing – review & editing.

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This research was funded by ANID Postdoctoral Fellowship Nb. 3180590 and ANID/PIA/Basal Funds for Centers of Excellence FB0003.

Acknowledgments

Authors gratefully acknowledge the support from ANID Postdoctoral Fellowship Nb. 3180590 and ANID/PIA/Basal Funds for Centers of Excellence FB0003.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Autom. Control 19, 716–723. doi: 10.1109/TAC.1974.1100705

Crossref Full Text | Google Scholar

Alamri, N. M. (2020). The implementation of the lesson study strategy in teaching mathematics: teachers’ perspectives. Educ. Res. Int. 2020, 1–8. doi: 10.1155/2020/1683758

Alvarado, M. (2012). “La Evaluación Docente y Sus Instrumentos: Discriminación Del Desempeño Docente y Asociación Con Los Resultados de Los Estudiantes.”

Google Scholar

Assaél, J., and Cornejo, R. (2018). “Work regulations and teacher subjectivity in a context of standardization and accountability policies in Chile.” In R. Normand, M. Liu, L. Carvalho, D. Oliveira, and L. LeVasseur (eds.), Education policies and the restructuring of the educational profession: global and comparative perspectives , Singapore: Springer, 245–257.

Avalos, B., and Assael, J. (2006). Moving from resistance to agreement: the case of the Chilean teacher performance evaluation. Int. J. Educ. Res. 45, 254–266. doi: 10.1016/j.ijer.2007.02.004

Avalos-Bevan, B. (2018). Teacher evaluation in Chile: highlights and complexities in 13 years of experience. Teach. Teach. 24, 297–311. doi: 10.1080/13540602.2017.1388228

Chokshi, S., and Fernandez, C. (2004). Challenges to importing Japanese lesson study: concerns, misconceptions, and nuances. Phi Delta Kappan 85, 520–525. doi: 10.1177/003172170408500710

Colegio de Profesores. (2023). “Evaluación Docente.” Available at: https://www.colegiodeprofesores.cl/wp-content/uploads/2023/05/EVALUACION-DOCENTE-001.pdf .

de Vries, S., Jansen, E. P. W. A., and van de Grift, W. J. C. M. (2013). Profiling teachers’ continuing professional development and the relation with their beliefs about learning and teaching. Teach. Teach. Educ. 33, 78–89. doi: 10.1016/j.tate.2013.02.006

Dudley, P., Haiyan, X., Vermunt, J. D., and Lang, J. (2019). Empirical evidence of the impact of lesson study on students’ achievement, teachers’ professional learning and on institutional and system evolution. Eur. J. Educ. 54, 202–217. doi: 10.1111/ejed.12337

Estrella, S., Mena-Lorca, A., and Olfos, R. (2018). “Lesson study in Chile: a very promising but still uncertain path” in Mathematics lesson study around the world: theoretical and methodological issues . eds. M. Quaresma, C. Winsløw, S. Clivaz, J. da Ponte, A. Ní Shúilleabháin, and A. Takahashi (Cham: Springer), 105–122.

Fang, Y., Wang, X., and Kim-Eng, C. L. (2019). “Representing instructional improvement in lesson study through principled analysis of research lessons in Singapore: a case of equivalent fractions” in Theory and practice of lesson study in mathematics: an international perspective . eds. R. Huang, A. Takahashi, and J. P. da Ponte (Switzerland: Springer), 393–418.

Fernandez, M. L. (2010). Investigating how and what prospective teachers learn through microteaching lesson study. Teach. Teach. Educ. 26, 351–362. doi: 10.1016/j.tate.2009.09.012

Fernandez, M. L., and Robinson, M. (2006). Prospective teachers’ perspectives on microteaching lesson study. Education 127:203+.

Fortney, B. S. (2009). “The impact of Japanese lesson study on preservice teacher belief structures about teaching and learning science” (Order No. 3360299). Available from Pro Quest Dissertations & Theses Global. (305005930).

Fujii, T. (2014). Implementing Japanese lesson study in foreign countries: misconceptions revealed. Math. Teach. Educ. Dev. 16:n1

Godfrey, D., Seleznyov, S., Anders, J., Wollaston, N., and Barrera-Pedemonte, F. (2019). A developmental evaluation approach to lesson study: exploring the impact of lesson study in London schools. Prof. Dev. Educ. 45, 325–340. doi: 10.1080/19415257.2018.1474488

Inprasitha, M. (2015). “Prospective teacher education in mathematics through lesson study.” In Series on mathematics education, vol. 3 , by M. Inprasitha, M. Isoda, P. Wang-Iverson, and B.-H. Yeap, eds., 185–196. Singapore: World Scientific.

Isoda, M. (2015). “The science of lesson study in the problem solving approach.” In Series on mathematics education, vol. 3 , by M. Inprasitha, M. Isoda, P. Wang-Iverson, and B.-H. Yeap, eds., 81–108. Singapore: World Scientific.

Juhler, M. V. (2016). The use of lesson study combined with content representation in the planning of physics lessons during field practice to develop pedagogical content knowledge. J. Sci. Teach. Educ. 27, 533–553. doi: 10.1007/s10972-016-9473-4

Lewis, C., and Perry, R. (2014). Lesson study with mathematical resources: a sustainable model for locally-led teacher professional learning. Math. Teach. Educ. Dev. 16, 22–42.

Lewis, C. C., and Perry, R. R. (2015). “A randomized trial of lesson study with mathematical resource kits: analysis of impact on teachers’ beliefs and learning community” in Large-scale studies in mathematics education . eds. J. Middleton, J. Cai, and S. Hwang (Cham: Springer), 133–158.

Luft, J. A., and Roehrig, G. H. (2007). Capturing science teachers’ epistemological beliefs: the development of the teacher beliefs interview. Electron. J. Res. Sci. Math. Educ. 11, 38–63.

Lumpe, A., Vaughn, A., Henrikson, R., and Bishop, D. (2014). “Teacher professional development and self-efficacy beliefs” in The role of science teachers’ beliefs in international classrooms (Brill), 49–63. Rotterdam: Sense Publishers.

Maass, K. (2011). How can teachers’ beliefs affect their professional development? ZDM 43, 573–586. doi: 10.1007/s11858-011-0319-4

Manzi, J., González, R., and Sun, Y.. (2011). La Evaluación Docente En Chile.

Marble, S. T. (2006). Learning to teach through lesson study. Action Teach. Educ. 28, 86–96. doi: 10.1080/01626620.2006.10463422

Marble, S. (2007). Inquiring into teaching: lesson study in elementary science methods. J. Sci. Teach. Educ. 18, 935–953. doi: 10.1007/s10972-007-9071-6

Martínez, D. (2022). “Fin a La Doble Evaluación a Docentes. [Master’s Thesis, University of Chile].” Available at: https://repositorio.uchile.cl/handle/2250/194670 .

McSweeney, K., and Gardner, J.. (2018). “Lesson study matters in Ireland.” In Rural Environment. Education. Personality (REEP 2018) 2018 (Vol. 11, pp. 304–313). Latvia University of Life Sciences and Technologies. doi: 10.22616/REEP.2018.037

Mintzes, J. J., Marcum, B., Messerschmidt-Yates, C., and Mark, A. (2013). Enhancing self-efficacy in elementary science teaching with professional learning communities. J. Sci. Teach. Educ. 24, 1201–1218. doi: 10.1007/s10972-012-9320-1

Sakai, T., Akai, H., Ishizaka, H., Tamura, K., Lee, Y.-J., Choy, B. H., et al. (2022). Changes in qualities and abilities of Japanese teachers through participation in global lesson study on mathematics. Int. J. Lesson Learn. Stud. 11, 290–304. doi: 10.1108/IJLLS-04-2022-0058

Schipper, T., Goei, S. L., De Vries, S., and Van Veen, K. (2018). Developing teachers’ self-efficacy and adaptive teaching behaviour through lesson study. Int. J. Educ. Res. 88, 109–120. doi: 10.1016/j.ijer.2018.01.011

Schoenfeld, A. H. (2011). Toward professional development for teachers grounded in a theory of decision making. ZDM 43, 457–469. doi: 10.1007/s11858-011-0307-8

Seleznyov, S. (2020). Lesson study: exploring implementation challenges in England. Int. J. Lesson Learn. Stud. 9, 179–192. doi: 10.1108/IJLLS-08-2019-0059

Skott, J. (2001). The emerging practices of a novice teacher: the roles of his school mathematics images. J. Math. Teach. Educ. 4, 3–28. doi: 10.1023/A:1009978831627

Stipek, D. J., Givvin, K. B., Salmon, J. M., and MacGyvers, V. L. (2001). Teachers’ beliefs and practices related to mathematics instruction. Teach. Teach. Educ. 17, 213–226. doi: 10.1016/S0742-051X(00)00052-4

Suh, J. M., and Fulginiti, K. (2012). ‘Situating the learning’ of teaching: implementing lesson study at a professional development school. School Univ. Partnerships 5, 24–37.

Szumilas, M. (2010). Explaining odds ratios. J. Can. Acad. Child Adolesc. Psychiatry 19, 227–229.

PubMed Abstract | Google Scholar

Taut, S., Santelices, M. V., Araya, C., and Manzi, J. (2011). Perceived effects and uses of the national teacher evaluation system in Chilean elementary schools. Stud. Educ. Eval. 37, 218–229. doi: 10.1016/j.stueduc.2011.08.002

Tupper, D. (2022). Elementary science lesson study: Relationships among efficacy, beliefs, and professional noticing (Order No. 29993352). Available from Pro Quest Dissertations & Theses Global. (2741309611).

Vermunt, J. D., Vrikki, M., Van Halem, N., Warwick, P., and Mercer, N. (2019). The impact of lesson study professional development on the quality of teacher learning. Teach. Teach. Educ. 81, 61–73. doi: 10.1016/j.tate.2019.02.009

Wessels, H. (2018). Noticing in pre-service teacher education: research lessons as a context for reflection on learners’ mathematical reasoning and sense-making . In: Kaiser, G., Forgasz, H., Graven, M., Kuzniak, A., Simmt, E., Xu, B. (eds) Invited Lectures from the 13th International Congress on Mathematical Education. ICME-13 Monographs. Cham: Springer. 731–748.

Whitney, S. R. (2020). Are lesson study participants sharing their professional knowledge? Int. J. Lesson Learn. Stud. 9, 57–66. doi: 10.1108/IJLLS-11-2018-0090

Yakar, Z., and Turgut, D. (2017). Effectiveness of lesson study approach on preservice science teachers’ beliefs. Int. Educ. Stud. 10, 36–43. doi: 10.5539/ies.v10n6p36

Zhang, Z. (2016). Variable selection with stepwise and best subset approaches. Ann. Transl. Med. 4:136. doi: 10.21037/atm.2016.03.35

PubMed Abstract | Crossref Full Text | Google Scholar

Keywords: lesson study, research lesson, peer observation, teacher professional development, teachers’ beliefs

Citation: Caballero D, Mella-Defranchi E and Araya R (2024) The impact of observers’ beliefs on the perceived contribution of a Research Lesson. Front. Educ . 9:1331293. doi: 10.3389/feduc.2024.1331293

Received: 31 October 2023; Accepted: 21 March 2024; Published: 10 April 2024.

Reviewed by:

Copyright © 2024 Caballero, Mella-Defranchi and Araya. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Daniela Caballero, [email protected]

This article is part of the Research Topic

Global Lesson Study Policy, Practice, and Research for Advancing Teacher and Student Learning in STEM

• Today's news
• Reviews and deals
• Climate change
• 2024 election
• Fall allergies
• Health news
• Mental health
• Sexual health
• Family health
• So mini ways
• Unapologetically
• Buying guides

Entertainment

• How to Watch
• My watchlist
• Stock market
• Biden economy
• Personal finance
• Stocks: most active
• Stocks: gainers
• Stocks: losers
• Trending tickers
• World indices
• US Treasury bonds
• Top mutual funds
• Highest open interest
• Highest implied volatility
• Currency converter
• Basic materials
• Communication services
• Consumer cyclical
• Consumer defensive
• Financial services
• Industrials
• Real estate
• Mutual funds
• Credit cards
• Credit card rates
• Balance transfer credit cards
• Business credit cards
• Cash back credit cards
• Rewards credit cards
• Travel credit cards
• Checking accounts
• Online checking accounts
• High-yield savings accounts
• Money market accounts
• Personal loans
• Student loans
• Car insurance
• Home buying
• Options pit
• Investment ideas
• Research reports
• Fantasy football
• Pro Pick 'Em
• College Pick 'Em
• Fantasy baseball
• Fantasy hockey
• Fantasy basketball
• Download the app
• Daily fantasy
• Scores and schedules
• GameChannel
• World Baseball Classic
• Premier League
• CONCACAF League
• Champions League
• Motorsports
• Horse racing
• Newsletters

New on Yahoo

• Privacy Dashboard

Drinking Coffee Might Reduce Risk of Colorectal Cancer Recurrence, Study Finds

Guido Mieth / Getty Images

Fact checked by Nick Blackmer

Key Takeaways

New observational study linked coffee with lower risk of colorectal cancer recurrence.

Coffee might reduce oxidative stress and protect cells from damage.

More research is needed to confirm the findings so experts recommend adjusting other lifestyle factors and getting colorectal cancer screenings.

If you drink multiple cups of coffee a day, there’s new research to support your habit.

Drinking three to five cups of coffee each day may reduce the risk of colorectal cancer recurrence and death, according to an observational study in the Netherlands.

This observational study relied on data from 1,719 people in the Netherlands with stage 1–3 colorectal cancer. Those who drank more than four cups of coffee each day had a 32% lower risk of recurrence compared to people who drank less than two cups per day.

“It is unclear whether this is generalizable to other populations in other parts of the world,” David A. Greenwald, MD , director of Clinical Gastroenterology and Endoscopy at Mount Sinai Hospital, who was not involved in the study, told Verywell in an email. “There are certainly other factors that affect risk for recurrence of colorectal cancer like the precise stage of the tumor.”

Around 20%–30% of people with stage 1–3 colorectal cancer experience recurrence.

“Patients with stage 2 to 3 colorectal cancer, however, could consider the findings in the study as they choose diet and lifestyle plans,” Greenwald added.

Related: Colon Cancer Recurrence Statistics

Can Coffee Reduce Colorectal Cancer Risk?

A randomized controlled trial in 2015 also found that drinking at least four cups of coffee a day significantly reduced the risk of recurrence and death for people with stage 3 colon cancer.

A 2016 study concluded that drinking over 2.5 servings of coffee each day reduced colorectal cancer risk by 54%, while another 2020 research associated coffee consumption with increased survival in people with advanced or metastatic colorectal cancer.

However, studies on coffee and colorectal cancer are inconsistent. A 2018 meta-analysis from Japan did not find enough evidence to show that coffee impacts colorectal cancer risk.

“The most important thing that people can do to reduce the risk of colorectal cancer recurrence is to form a relationship with an oncologist and follow their recommendations. Regular monitoring of blood levels, including carcinoembryonic antigen (CEA), and for many, regular imaging studies to detect any recurrence early is indicated,” Greenwald said.

Related: Why Gut Health Matters for Colorectal Cancer Risk, Especially for Young People

Although there isn’t conclusive evidence, certain properties in coffee—such as caffeine, flavonoids, and polyphenols—could keep cancer from spreading or protect cells from damage.

“Drinking coffee may help reduce oxidative stress—the condition where there are too many unstable molecules in the body, resulting in tissue damage,” said Misagh Karimi, MD, a medical oncologist at the City of Hope Orange County Lennar Foundation Cancer Center in Irvine, California. “Another possibility is that coffee may affect microorganisms in a way that promotes the effects of chemotherapy on the cancerous cells, preventing a recurrence. And drinking coffee may improve the liver function in people with colorectal cancer.”

Karimi said these possibilities presented in the new study still need to be confirmed by additional research.

It’s unclear exactly how much coffee the study participants drank since the researchers used self-reported data. Standard coffee cups are generally smaller in Europe than in the United States, so it is likely that the optimal reported range of three to five cups per day would translate to fewer cups of coffee in other parts of the world.

The researchers also didn’t specify if participants drank caffeinated or decaf coffee, although they noted that most people in the Netherlands drink caffeinated coffee.

Related: Study: Drinking Coffee Might Help Colorectal Cancer Patients Live Longer

What Can You Do to Reduce Colorectal Cancer Risk?

Age, genetics, family history, and having inflammatory bowel disease can all increase your risk of colorectal cancer—one of the fastest-growing early-onset cancers in the United States.

There’s not enough evidence for experts to recommend drinking coffee to reduce your risk of developing colorectal cancer or preventing recurrence after a diagnosis.

But exercising regularly, maintaining a healthy body weight, avoiding alcohol, quitting smoking, and eating a diet high in fruits, vegetables, and whole grains and low in red meat and processed foods may help reduce colorectal cancer risk.

You should also get a screening starting at the age of 45 , although you could get screened earlier if you have a family history of colorectal cancer.

“The most important things people can do to reduce their risk of colorectal cancer is to maintain healthy lifestyle choices and to get colon cancer screening as their physician recommends and in keeping with their risk profile,” Karimi said.

Related: 4 Early Symptoms of Colon Cancer Young Adults Should Know

What This Means For You

There is not enough evidence to prove that drinking coffee reduces colorectal cancer risk or recurrence. You can consult with your healthcare provider about your risk. Most people should start getting screened at age 45.

Read the original article on Verywell Health .

Recommended Stories

I went on 6 flights in 10 days, and these $17 slippers kept my feet comfy and supported — even in the airport.

These cozy kicks with over 8,900 five-star fans were my overseas travel must-have.

Arizona reinstated a 160-year-old abortion ban this week. What’s happening in other states?

This week Arizona reinstated a 160-year-old law that bans nearly all abortions in the state, joining dozens of other states where the procedure is banned or restricted. Here's what else you may have missed in abortion news this week.

2024 NBA Awards: All-NBA, All-Defensive and All-Rookie selections

Here's one voter's awards ballot with All-NBA, All-Defensive and All-Rookie selections.

Epic wants to blow the Google Play Store wide open

After winning an antitrust case against Google, Epic Games has revealed what it wants as a result. In short, it's looking to blow the Play Store wide open.

The best travel tech for your next getaway, according to flight crew members — Apple, Anker, Google and more

Take a hint from the pros and pick up one of these problem-solving gadgets.

Two Fed officials urge patience before any interest rate cuts

Kansas City Fed president Jeff Schmid and San Francisco Fed president Mary Daly said they would employ patience and caution before approving any rate cuts.

O.J. Simpson's getaway car, the white Bronco, is a museum piece in Tennessee

When O.J. Simpson attempted to elude police in a now-infamous white Ford Bronco, he likely wasn’t thinking that it would someday end up in a Tennessee museum that was designed to look like Alcatraz.

2025 Toyota 4Runner is here, Chevy Corvette ZR1 teased | Autoblog Podcast #827

Toyota unveiled its new 4Runner, Chevy teased the Corvette ZR1, BMW confirmed the M5 Touring, Alfa Romeo revealed the Milano EV and Tesla made some moves.

The Fellow Aiden makes coffee good enough for snobs with a single button

After years of customer requests, Fellow is finally announcing its first coffee machine in the Aiden, which delivers a high-quality pour-over experience with a single touch.

Pet threatening your home office? Protect your cables with this $8 cable sleeve — no more chewing

Over 48,000 customers agree that these covers will help you experience the life-changing magic of contained and covered cords.

Roku says 576,000 user accounts hacked after second security incident

Streaming giant Roku has confirmed a second security incident in as many months, with hackers this time able to compromise more than half a million Roku user accounts. In a statement Friday, the company said about 576,000 user accounts were accessed using a technique known as credential stuffing, where malicious hackers use usernames and passwords stolen from other data breaches and reuse the logins on other sites. Roku said in fewer than 400 account breaches, the malicious hackers made fraudulent purchases of Roku hardware and streaming subscriptions using the payment data stored in those users' accounts.

8BitDo's Ultimate Controller with charging dock drops to $56 on Amazon

8BitDo's Ultimate Controller with charging dock is currently on sale for $56. That's 20 percent off and a solid deal on one of the best third-party gamepads around.

NBA Playoff Picture: Everything at stake entering Friday with 17 teams still playing for seeding

Here are the latest updates on the landscape, including magic numbers, relevant tiebreakers and the stakes for Friday's slate of games.

Clean up with this $98 Bissell deal — the lowest price we've seen — plus other sweet savings we found today

Also on our cheat sheet: A sturdy $25 Lands' End tote and a KitchenAid stand mixer for $170 off.

The Morning After: Our verdict on the Humane AI Pin

The biggest news stories this morning: Repair your iPhones with used components, The best laptops, Taylor Swift’s music is back on TikTok.

Mortgage rates today, April 12, 2024: 30-year rates increase for the 3rd straight week

These are today's mortgage rates. Rates probably won't plummet this home-buying season. Lock in your rate now if you're otherwise financially ready.

Meta trials its AI chatbot across WhatsApp, Instagram and Messenger in India and Africa

Meta has confirmed to TechCrunch that it is testing Meta AI, its large language model-powered chatbot, with WhatsApp, Instagram and Messenger users in India and parts of Africa. The move signals how Meta plans to tap massive user bases across its various apps to scale its AI offerings. Meta announced plans to build and experiment with chatbots and other AI tools in February 2023.

'Sasquatch Sunset' is so relentlessly gross that people are walking out of screenings. Star Jesse Eisenberg says the film was a ‘labor of love.’

“There are so many movies made for people who like typical things. This is not that," the film's star told Yahoo Entertainment.

O.J. Simpson dead at 76 after battle with cancer, family announces

O.J. Simpson, one of the most infamous figures in U.S. history, had been battling cancer.

O.J. Simpson timeline: From football and acting stardom to murder trial and death

Here are some of the key moments in the life of the NFL star-turned-actor acquitted of murdering his wife and her friend in the so-called trial of the century.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Dtsch Arztebl Int
• v.117(7); 2020 Feb

Methods for Evaluating Causality in Observational Studies

Emilio a.l.gianicolo.

1 Institute for Medical Biostatistics, Epidemiology and Informatics (IMBEI), University Medical Center of the Johannes Gutenberg University of Mainz

2 Institute of Clinical Physiology of the Italian National Research Council, Lecce, Italy

Martin Eichler

3 Technical University Dresden, University Hospital Carl Gustav Carus, Medical Clinic 1, Dresden

Oliver Muensterer

4 Department of Pediatric Surgery, Faculty of Medicine, Johannes Gutenberg University of Mainz

Konstantin Strauch

5 Institute of Genetic Epidemiology, Helmholtz Zentrum München—German Research Center for Environmental Health, Neuherberg; Chair of Genetic Epidemiology, Institute for Medical Information Processing, Biometry, and Epidemiology, Ludwig-Maximilians-Universität, München

Maria Blettner

In clinical medical research, causality is demonstrated by randomized controlled trials (RCTs). Often, however, an RCT cannot be conducted for ethical reasons, and sometimes for practical reasons as well. In such cases, knowledge can be derived from an observational study instead. In this article, we present two methods that have not been widely used in medical research to date.

The methods of assessing causal inferences in observational studies are described on the basis of publications retrieved by a selective literature search.

Two relatively new approaches—regression-discontinuity methods and interrupted time series—can be used to demonstrate a causal relationship under certain circumstances. The regression-discontinuity design is a quasi-experimental approach that can be applied if a continuous assignment variable is used with a threshold value. Patients are assigned to different treatment schemes on the basis of the threshold value. For assignment variables that are subject to random measurement error, it is assumed that, in a small interval around a threshold value, e.g., cholesterol values of 160 mg/dL, subjects are assigned essentially at random to one of two treatment groups. If patients with a value above the threshold are given a certain treatment, those with values below the threshold can serve as control group. Interrupted time series are a special type of regression-discontinuity design in which time is the assignment variable, and the threshold is a cutoff point. This is often an external event, such as the imposition of a smoking ban. A before-and-after comparison can be used to determine the effect of the intervention (e.g., the smoking ban) on health parameters such as the frequency of cardiovascular disease.

The approaches described here can be used to derive causal inferences from observational studies. They should only be applied after the prerequisites for their use have been carefully checked.

The fact that correlation does not imply causality was frequently mentioned in 2019 in the public debate on the effects of diesel emission exposure ( 1 , 2 ). This truism is well known and generally acknowledged. A more difficult question is how causality can be unambiguously defined and demonstrated ( box 1 ) . According to the eighteenth-century philosopher David Hume, causality is present when two conditions are satisfied: 1) B always follows A—in which case, A is called a “sufficient cause” of B; 2) if A does not occur, then B does not occur—in which case, A is called a “necessary cause” of B ( 3 ). These strict logical criteria are only rarely met in the medical field. In the context of exposure to diesel emissions, they would be met only if fine-particle exposure always led to lung cancer, and lung cancer never occurred without prior fine-particle exposure. Of course, neither of these is true. So what is biological, medical, or epidemiological causality? In medicine, causality is generally expressed in probabilistic terms, i.e. exposure to a risk factor such as cigarette smoking or diesel emissions increases the probability of a disease, e.g., lung cancer. The same understanding of causality applies to the effects of treatment: for instance, a certain type of chemotherapy increases the likelihood of survival in patients with a diagnosis of cancer, but does not guarantee it.

Causality in epidemiological observational studies (modified from Parascondola and Weed [34])

• ausality as production: A produces B. Causality is to be distinguished from mere temporal sequence. It does not suffice to note that A is always followed by B; rather, A must in some way produce, lead to, or create B. However, it remains unclear what ’producing’, ‘leading to’, or ‘creating’ exactly means. On a practical level, the notion of production is what is illustrated in the diagrams of cause-and-effect relationships that are commonly seen in medical publications.
• Sufficient and necessary causes: A is a sufficient cause of B if B always happens when A has happened. A is a necessary cause of B if B only happens when A has happened. Although these relationships are logically clear and seemingly simple, this type of deterministic causality is hardly ever found in real-life scientific research. Thus, smoking is neither a sufficient nor a necessary cause of lung cancer. Smoking is not always followed by lung cancer (not a sufficient cause), and lung cancer can occur in the absence of tobacco exposure (not a necessary cause, either).
• Sufficient component cause: This notion was developed in response to the definitions of sufficient and necessary causes. In this approach, it is assumed that multiple causes act together to produce an effect where no single one of them could do so alone. There can also be different combinations of causes that produce the same effect.
• Probabilistic causality: In this scenario, the cause (A) increases the probability (P) that the effect (B) will occur: in symbols, P (B | A) > (B | not A). Sufficient and necessary causes, as defined above in ( 2 ), are only those extreme cases in which P (B | A) = 1 and P (B | not A) = 0, respectively. When these probabilities take on values that are neither 0 nor 1, causality is no longer deterministic, but rather probabilistic (stochastic). There is no assumption that a cause must be followed by an effect. This viewpoint corresponds to the method of proceeding in statistically oriented scientific disciplines.
• Causal inference: This is the determination that a causal relationship exists between two types of event. Causal inferences are made by analyzing the changes in the effect that arise when there are changes in the cause. Causal inference goes beyond the mere assertion of an association and is connected to a number of specific concepts: some that have been widely discussed recently are counterfactuals, potential outcomes, causal diagrams, and structural equation models ( 36 , 37 ).
• Triangulation: Not all questions can be answered with an experiment or a randomized controlled trial. Alternatively, methodological pluralism is needed, or, as it is now sometimes called, triangulation: confidence in a finding increases when the same finding is arrived at from multiple data sets, multiple scientific disciplines, multiple theories, and/or multiple methods ( 35 ).
• The criterion of consequentiality: The claim that a causal relationship exists has consequences on a societal level (taking action or not taking action). Olsen has called for the formulation of a criterion to determine when action should be taken and when not ( 7 ).

In many scientific disciplines, causality must be demonstrated by an experiment. In clinical medical research, this purpose is achieved with a randomized controlled trial (RCT) ( 4 ). An RCT, however, often cannot be conducted for either ethical or practical reasons. If a risk factor such as exposure to diesel emissions is to be studied, persons cannot be randomly allocated to exposure or non-exposure. Nor is any randomization possible if the research question is whether or not an accident associated with an exposure, such as the Chernobyl nuclear reactor disaster, increased the frequency of illness or death. The same applies when a new law or regulation, e.g., a smoking ban, is introduced.

When no experiment can be conducted, observational studies need to be performed. The object under study—i.e., the possible cause—cannot be varied in a targeted and controlled way; instead, the effect this factor has on a target variable, such as a particular illness, is observed and documented.

Several publications in epidemiology have dealt with the ways in which causality can be inferred in the absence of an experiment, starting with the classic work of Bradford Hill and the nine aspects of causality (viewpoints) that he proposed ( box 2 ) ( 5 ) and continuing up to the present ( 6 , 7 ).

The Bradford Hill criteria for causality (modified from [5])

• Strength: the stronger the observed association between two variables, the less likely it is due to chance.
• Consistency: the association has been observed in multiple studies, populations at risk, places, and times, and by different researchers.
• Specificity: it is a strong argument for causality when a specific population suffers from a specific disease.
• Temporality: the effect must be temporally subsequent to the cause.
• Biological gradient: the association displays a dose–response effect, e.g., the incidence of lung cancer is greater when more cigarettes are smoked per day.
• Plausibility: a plausible mechanism linking the cause to the effect is helpful, but not absolutely required. What is biologically plausible depends upon the state-of-the-art knowledge of the time.
• Coherence: the causal interpretation of the data should not conflict with biological knowledge about the disease.
• Experiment: experimental evidence should be adduced in support, if possible.
• Analogy: an association speaks for causality if similar causes are already known to have similar effects.

Aside from the statistical uncertainty that always arises when only a sample of an affected population is studied, rather than its entirety ( 8 ), the main obstacle to the study of putative causal relationships comes from confounding variables (“confounders”). These are so named because they can, depending on the circumstances, either obscure a true effect or simulate an effect that is, in fact, not present ( 9 ). Age, for example, is a confounder in the study of the association between occupational radiation exposure and cataract ( 10 ), because both cumulative radiation exposure and the risk of cataract rise with increasing age.

The various statistical methods of dealing with known confounders in the analysis of epidemiological data have already been presented in other articles in this series ( 9 , 11 , 12 ). In the current article, we discuss two new approaches that have not been widely applied in medical and epidemiological research to date.

Methods of evaluating causal inferences in observational studies

The main advantage of an RCT is randomization, i.e., the random allocation of the units of observation (patients) to treatment groups. Potential confounders, whether known or unknown, are thereby distributed to the treatment groups at random as well, although differences between groups may arise through sample variance. Whenever randomization is not possible, the effect of confounders must be taken into account in the planning of the study and in data analysis, as well as in the interpretation of study findings.

Classic methods of dealing with confounders in study planning are stratification and matching ( 13 , 14 ), as well as so-called propensity score matching (PSM) ( 11 ).

The best-known and most commonly used method of data analysis is regression analysis, e.g., linear, logistic, or Cox regression ( 15 ). This method is based on a mathematical model created in order to explain the probability that any particular outcome will arise as the combined result of the known confounders and the effect under study.

Regression analyses are used in the analysis of clinical or epidemiological data and are found in all commonly used statistical software packages. However, they are often used inappropriately because the prerequisites for their correct application have not been checked. They should not be used, for example, if the sample is too small, if the number of variables is too large, or if a correlation between the model variables makes the results uninterpretable ( 16 ).

Regression-discontinuity methods

Regression-discontinuity methods have been little used in medical research to date, but they can be helpful in the study of cause-and-effect relationships from observational data ( 17 ). Regression-discontinuity design is a quasi-experimental approach ( box 3 ) that was developed in educational psychology in the 1960s ( 18 ). It can be used when a threshold value of a continuous variable (the “assignment variable”) determines the treatment regimen to which each patient in the study is assigned ( box 4 ) .

Terms used to characterize experiments ( 18 )

• Experiment/trial A study in which an intervention is deliberately introduced in order to observe an effect.
• Randomized experiment/trial An experiment in which persons, patients, or other units of observation are randomly assigned to one of two or more treatment groups (or intervention groups).
• Quasi-experiment An experiment in which the units of observation are not randomly assigned to the treatment/intervention groups.
• Natural experiment A study in which a natural event (e.g., an earthquake) is compared with a comparison scenario.
• Non-experimental observational study A study in which the size and direction of the association between two variables is determined.

In the simplest case, that of a linear regression, the parameters in the following model are to be estimated:

y i = ß 0 + ß 1 z i + ß 2 (x i - x c ) + e i,

i from 1 to N represents the statistical units

y is the outcome

ß 0 is the y-intercept

z is a dichotomous variable (0, ) indicating whether the patient was treated ( 1 ) or not treated (0)

x is the assignment variable

x c is the threshold

ß 1 is the effect of treatment

ß 2 is the regression coefficient of the assignment variable

e is the random error

A possible assignment variable could be, for example, the serum cholesterol level: consider a study in which patients with a cholesterol level of 160 mg/dL or above are assigned to receive a therapy. Since the cholesterol level (the assignment variable) is subject to random measurement error, it can be assumed that patients whose level of cholesterol is close to the threshold (160 mg/dL) are randomly assigned to the different treatment regimens. Thus, in a small interval around the threshold value, the assignment of patients to treatment groups can effectively be considered random ( 18 ). This sample of patients with near-threshold measurements can thus be used for the analysis of treatment efficacy. For this line of argument to be valid, it must truly be the case that the value being measured is subject to measuring error, and that there is practically no difference between persons with measured values slightly below or slightly above threshold. Treatment allocation in this narrow range can be considered quasi-random.

This method can be applied if the following prerequisites are met:

• The assignment variable is a continuous variable that is measured before the treatment is provided. If the assignment variable is totally independent of the outcome and has no biological, medical, or epidemiological significance, the method is theoretically equivalent to an RCT ( 19 ).
• The treatment must not affect the assignment variable ( 18 ).
• The patients in the two treatment groups with near-threshold values of the assignment variable must be shown to be similar in their baseline properties, i.e., covariables, including possible confounders. This can be demonstrated either with statistical techniques or graphically ( 20 ).
• The range of the assignment variable in the vicinity of the threshold must be optimally set: it must be large enough to yield samples of adequate size in the treatment groups, yet small enough that the effect of the assignment variable itself does not alter the outcome being studied. Methods of choosing this range appropriately are available in the literature ( 21 , 22 ).
• The treatment can be decided upon solely on the basis of the assignment variable (deterministic regression-discontinuity methods), or on the basis of other clinical factors (fuzzy regression-discontinuity methods).

Example 1: The one-year mortality of neonates as a function of the intensity of medical and nursing care was to be studied, where the intensity of care was determined by a birth-weight threshold: infants with very low birth weight (<1500 g) (group A) were cared for more intensively than heavier infants (group B) ( 23 ). The question to be answered was whether the greater intensity of care in group A led to a difference in mortality between the two groups. It was assumed that children with birth weight near the threshold are identical in all other respects, and that their assignment to group A or group B is quasi-random, because the measured value (birth weight) is subject to a relatively small error. Thus, for example, one might compare children weighing 1450–1500 g to those weighing 1501–1550 g at birth to study whether, and how, a greater intensity of care affects mortality.

In this example, it is assumed that the variable “birth weight” has a random measuring error, and thus that neonates whose (true) weight is near the threshold will be randomly allocated to one or the other category. But birth weight itself is an important factor affecting infant mortality, with lower birth weight associated with higher mortality ( 23 ); thus, the interval taken around the threshold for the purpose of this study had to be kept narrow. The study, in fact, showed that the children treated more intensively because their birth weight was just below threshold had a lower mortality than those treated less intensively because their birth weight was just above threshold.

Example 2: A regression-discontinuity design was used to evaluate the effect of a measure taken by the Canadian government: the introduction of a minimum age of 19 years for alcohol consumption. The researchers compared the number of alcohol-related disorders and of violent attacks, accidents, and suicides under the influence of alcohol in the months leading up to (group A) and subsequent to (group B) the 19 th birthday of the persons involved. It was found that persons in group B had a greater number of alcohol-related inpatient treatments and emergency hospitalizations than persons in group A. With the aid of this quasi-experimental approach, the researchers were able to demonstrate the success of the measure ( 24 ). It may be assumed that the two groups differed only with respect to age, and not with respect to any other property affecting alcohol consumption.

Interrupted time series

Interrupted time series are a special type of regression-discontinuity design in which time is the assignment variable. The cutoff point is often an external event that is unambiguously identifiable as having occurred at a certain point in time, e.g., an industrial accident or a change in the law. A before-and-after comparison is made in which the analysis must still take adequate account of any relevant secular trends and seasonal fluctuations ( box 5 ) .

In the simplest case of a study involving an interrupted time series, the temporal sequence is analyzed with a piecewise regression. The following model is used to study both a shift in slope and a shift in the level of an outcome before and after an intervention, e.g., the introduction of a law banning smoking ( figure 2 ):

y = ß 0 + ß 1 × time + ß 2 × intervention + ß 3 × time × intervention + e,

y is the outcome, e.g., cardiovascular diseases

intervention is a dummy variable for the time before (0) and after (1) the intervention (e.g., smoking ban)

time is the time since the beginning of the study

ß 0 is the baseline incidence of cardiovascular diseases

ß 1 is the slope in the incidence of cardiovascular diseases over time before the introduction of the smoking ban

ß 2 is the change in the incidence level of cardiovascular diseases after the introduction of the smoking ban (level effect)

ß 3 is the change in the slope over time (cf. ß 1 ) after the introduction of the smoking ban (slope effect)

The prerequisites for the use of this method must be met ( 18 , 25 ):

• Interrupted time series are valid only if a single intervention took place in the period of the study.
• The time before the intervention must be clearly distinguishable from the time after the intervention.
• There is no required minimum number of data points, but studies with only a small number of data points or small effect sizes must be interpreted with caution. The power of a study is greatest when the number of data points before the intervention equals the number after the intervention ( 26 ).
• Although the equation in Box 5 has a linear specification, polynomial and other nonlinear regression models can be used as well. Meticulous study of the temporal sequence is very important when a nonlinear model is used.
• If an observation at time t —e.g., the monthly incidence of cardiovascular diseases—is correlated with previous observations (autoregression), then the appropriate statistical techniques must be used (autoregressive integrated moving average [ARIMA] models).

Example 1: In one study, the rates of acute hospitalization for cardiovascular diseases before and after the temporary closure of Heathrow Airport because of volcanic ash were determined to investigate the putative effect of aircraft noise ( 27 ). The intervention (airport closure) took place from 15 to 20 April 2010. The hospitalization rate was found to have decreased among persons living in the urban area with the most aircraft noise. The number of observation points was too low, however, to show a causal link conclusively.

Example 2: In another study, the rates of hospitalization before and after the implementation of a smoking ban (the intervention) in public areas in Italy were determined ( 28 ). The intervention occurred in January 2004 (the cutoff time). The number of hospitalizations for acute coronary events was measured from January 2002 to November 2006 ( figure 1 ) . The analysis took account of seasonal dependence, and an effect modification for two age groups—persons under age 70 and persons aged 70 and up—was determined as well. The hospitalization rate declined in the former group, but not the latter.

Age-standardized hospitalization rates for acute coronary events (ACE) in persons under age 70 before and after the implementation of a smoking ban in public places in Italy, studied with the corresponding methods ( 30 ). The observed and predicted rates are shown (circles and solid lines, respectively). The dashed lines show the seasonally adjusted trend in ACE before and after the introduction of the nationwide smoking ban.

The necessary distinction between causality and correlation is often emphasized in scientific discussions, yet it is often not applied strictly enough. Furthermore, causality in medicine and epidemiology is mostly probabilistic in nature, i.e., an intervention alters the probability that the event under study will take place. A good illustration of this principle is offered by research on the effects of radiation, in which a strict distinction is maintained between deterministic radiation damage on the one hand, and probabilistic (stochastic) radiation damage on the other ( 29 ). Deterministic radiation damage—radiation-induced burns or death—arises with certainty whenever a subject receives a certain radiation dose (usually a high one). On the other hand, the risk of cancer-related mortality after radiation exposure is a stochastic matter. Epidemiological observations and biological experiments should be evaluated in tandem to strengthen conclusions about probabilistic causality ( box 1 ) .

While RCTs still retain their importance as the gold standard of clinical research, they cannot always be carried out. Some indispensable knowledge can only be obtained from observational studies. Confounding factors must be eliminated, or at least accounted for, early on when such studies are planned. Moreover, the data that are obtained must be carefully analyzed. And, finally, a single observational study hardly ever suffices to establish a causal relationship.

In this article, we have presented two newer methods that are relatively simple and which, therefore, could easily be used more widely in medical and epidemiological research ( 30 ). Either one should be used only after the prerequisites for its applicability have been meticulously checked. In regression-discontinuity methods, the assumption of continuity must be verified: in other words, it must be checked whether other properties of the treatment and control groups are the same, or at least equally balanced. The rules of group assignment and the role played by the continuous assignment variable must be known as well. Regression-discontinuity methods can generate causal conclusions, but any such conclusion will not be generalizable if the treatment effects are heterogeneous over the range of the assignment variable. The estimate of effect size is applicable only in a small, predefined interval around the threshold value. It must also be checked whether the outcome and the assignment variable are in a linear relationship, and whether there is any interaction between the treatment and assignment variables that needs to be considered.

In the analysis of interrupted time series, the assumption of continuity must be tested as well. Furthermore, the method is valid only if the occurrence of any other intervention at the same time point as the one under study can be ruled out ( 20 ). Finally, the type of temporal sequence must be considered, and more complex statistical methods must be applied, as needed, to take such phenomena as autoregression into account.

Observational studies often suggest causal relationships that will then be either supported or rejected after further studies and experiments. Knowledge of the effects of radiation exposure was derived, at first, mainly from observations on victims of the Hiroshima and Nagasaki atomic bomb explosions ( 31 ). These findings were reinforced by further epidemiological studies on other populations exposed to radiation (e.g., through medical procedures or as an occupational hazard), by physical considerations, and by biological experiments ( 32 ). A classic example from the mid-19 th century is the observational study by Snow ( 33 ): until then, the biological cause of cholera was unknown. Snow found that there had to be a causal relationship between the contamination of a well and a subsequent outbreak of cholera. This new understanding led to improved hygienic measures, which did, indeed, prevent infection with the cholera pathogen. Cases such as these prove that it is sometimes reasonable to take action on the basis of an observational study alone ( 6 ). They also demonstrate, however, that further studies are necessary for the definitive establishment of a causal relationship.

The effect of a smoking ban on the incidence of cardiovascular diseases

Key messages

• Causal inferences can be drawn from observational studies, as long as certain conditions are met.
• Confounding variables are a major impediment to the demonstration of causal links, as they can either obscure or mimic such a link.
• Random assignment leads to the even distribution of known and unknown confounders among the intervention groups that are being compared in the study.
• In the regression-discontinuity method, it is assumed that the assignment of patients to treatment groups is random with, in a small range of the assignment variable around the threshold, with the result that the confounders are randomly distributed as well.
• The interrupted time series is a variant of the regression-discontinuity method in which a given point in time splits the subjects into a before group and an after group, with random distribution of confounders to the two groups.

Acknowledgments

Translated from the original German by Ethan Taub, M.D.

Conflict of interest statement The authors state that they have no conflict of interest.