critical thinking observation checklist

Work Life is Atlassian’s flagship publication dedicated to unleashing the potential of every team through real-life advice, inspiring stories, and thoughtful perspectives from leaders around the world.

Contributing Writer

Work Futurist

Senior Quantitative Researcher, People Insights

Principal Writer

How to build critical thinking skills for better decision-making

It’s simple in theory, but tougher in practice – here are five tips to get you started.

Get stories like this in your inbox

Have you heard the riddle about two coins that equal thirty cents, but one of them is not a nickel? What about the one where a surgeon says they can’t operate on their own son?

Those brain teasers tap into your critical thinking skills. But your ability to think critically isn’t just helpful for solving those random puzzles – it plays a big role in your career. 

An impressive 81% of employers say critical thinking carries a lot of weight when they’re evaluating job candidates. It ranks as the top competency companies consider when hiring recent graduates (even ahead of communication ). Plus, once you’re hired, several studies show that critical thinking skills are highly correlated with better job performance.

So what exactly are critical thinking skills? And even more importantly, how do you build and improve them? 

What is critical thinking?

Critical thinking is the ability to evaluate facts and information, remain objective, and make a sound decision about how to move forward.

Does that sound like how you approach every decision or problem? Not so fast. Critical thinking seems simple in theory but is much tougher in practice, which helps explain why 65% of employers say their organization has a need for more critical thinking. 

In reality, critical thinking doesn’t come naturally to a lot of us. In order to do it well, you need to:

• Remain open-minded and inquisitive, rather than relying on assumptions or jumping to conclusions
• Ask questions and dig deep, rather than accepting information at face value
• Keep your own biases and perceptions in check to stay as objective as possible
• Rely on your emotional intelligence to fill in the blanks and gain a more well-rounded understanding of a situation

So, critical thinking isn’t just being intelligent or analytical. In many ways, it requires you to step outside of yourself, let go of your own preconceived notions, and approach a problem or situation with curiosity and fairness.

It’s a challenge, but it’s well worth it. Critical thinking skills will help you connect ideas, make reasonable decisions, and solve complex problems.

7 critical thinking skills to help you dig deeper

Critical thinking is often labeled as a skill itself (you’ll see it bulleted as a desired trait in a variety of job descriptions). But it’s better to think of critical thinking less as a distinct skill and more as a collection or category of skills. 

To think critically, you’ll need to tap into a bunch of your other soft skills. Here are seven of the most important. 

Open-mindedness

It’s important to kick off the critical thinking process with the idea that anything is possible. The more you’re able to set aside your own suspicions, beliefs, and agenda, the better prepared you are to approach the situation with the level of inquisitiveness you need. 

That means not closing yourself off to any possibilities and allowing yourself the space to pull on every thread – yes, even the ones that seem totally implausible.

As Christopher Dwyer, Ph.D. writes in a piece for Psychology Today , “Even if an idea appears foolish, sometimes its consideration can lead to an intelligent, critically considered conclusion.” He goes on to compare the critical thinking process to brainstorming . Sometimes the “bad” ideas are what lay the foundation for the good ones. 

Open-mindedness is challenging because it requires more effort and mental bandwidth than sticking with your own perceptions. Approaching problems or situations with true impartiality often means:

• Practicing self-regulation : Giving yourself a pause between when you feel something and when you actually react or take action.
• Challenging your own biases: Acknowledging your biases and seeking feedback are two powerful ways to get a broader understanding. 

Critical thinking example

In a team meeting, your boss mentioned that your company newsletter signups have been decreasing and she wants to figure out why.

At first, you feel offended and defensive – it feels like she’s blaming you for the dip in subscribers. You recognize and rationalize that emotion before thinking about potential causes. You have a hunch about what’s happening, but you will explore all possibilities and contributions from your team members.

Observation

Observation is, of course, your ability to notice and process the details all around you (even the subtle or seemingly inconsequential ones). Critical thinking demands that you’re flexible and willing to go beyond surface-level information, and solid observation skills help you do that.

Your observations help you pick up on clues from a variety of sources and experiences, all of which help you draw a final conclusion. After all, sometimes it’s the most minuscule realization that leads you to the strongest conclusion.

Over the next week or so, you keep a close eye on your company’s website and newsletter analytics to see if numbers are in fact declining or if your boss’s concerns were just a fluke. 

Critical thinking hinges on objectivity. And, to be objective, you need to base your judgments on the facts – which you collect through research. You’ll lean on your research skills to gather as much information as possible that’s relevant to your problem or situation. 

Keep in mind that this isn’t just about the quantity of information – quality matters too. You want to find data and details from a variety of trusted sources to drill past the surface and build a deeper understanding of what’s happening. 

You dig into your email and website analytics to identify trends in bounce rates, time on page, conversions, and more. You also review recent newsletters and email promotions to understand what customers have received, look through current customer feedback, and connect with your customer support team to learn what they’re hearing in their conversations with customers.

The critical thinking process is sort of like a treasure hunt – you’ll find some nuggets that are fundamental for your final conclusion and some that might be interesting but aren’t pertinent to the problem at hand.

That’s why you need analytical skills. They’re what help you separate the wheat from the chaff, prioritize information, identify trends or themes, and draw conclusions based on the most relevant and influential facts. 

It’s easy to confuse analytical thinking with critical thinking itself, and it’s true there is a lot of overlap between the two. But analytical thinking is just a piece of critical thinking. It focuses strictly on the facts and data, while critical thinking incorporates other factors like emotions, opinions, and experiences. 

As you analyze your research, you notice that one specific webpage has contributed to a significant decline in newsletter signups. While all of the other sources have stayed fairly steady with regard to conversions, that one has sharply decreased.

You decide to move on from your other hypotheses about newsletter quality and dig deeper into the analytics. 

One of the traps of critical thinking is that it’s easy to feel like you’re never done. There’s always more information you could collect and more rabbit holes you could fall down.

But at some point, you need to accept that you’ve done your due diligence and make a decision about how to move forward. That’s where inference comes in. It’s your ability to look at the evidence and facts available to you and draw an informed conclusion based on those. 

When you’re so focused on staying objective and pursuing all possibilities, inference can feel like the antithesis of critical thinking. But ultimately, it’s your inference skills that allow you to move out of the thinking process and onto the action steps. 

You dig deeper into the analytics for the page that hasn’t been converting and notice that the sharp drop-off happened around the same time you switched email providers.

After looking more into the backend, you realize that the signup form on that page isn’t correctly connected to your newsletter platform. It seems like anybody who has signed up on that page hasn’t been fed to your email list. 

Communication

3 ways to improve your communication skills at work

If and when you identify a solution or answer, you can’t keep it close to the vest. You’ll need to use your communication skills to share your findings with the relevant stakeholders – like your boss, team members, or anybody who needs to be involved in the next steps.

Your analysis skills will come in handy here too, as they’ll help you determine what information other people need to know so you can avoid bogging them down with unnecessary details. 

In your next team meeting, you pull up the analytics and show your team the sharp drop-off as well as the missing connection between that page and your email platform. You ask the web team to reinstall and double-check that connection and you also ask a member of the marketing team to draft an apology email to the subscribers who were missed. 

Problem-solving

Critical thinking and problem-solving are two more terms that are frequently confused. After all, when you think critically, you’re often doing so with the objective of solving a problem.

The best way to understand how problem-solving and critical thinking differ is to think of problem-solving as much more narrow. You’re focused on finding a solution.

In contrast, you can use critical thinking for a variety of use cases beyond solving a problem – like answering questions or identifying opportunities for improvement. Even so, within the critical thinking process, you’ll flex your problem-solving skills when it comes time to take action. 

Once the fix is implemented, you monitor the analytics to see if subscribers continue to increase. If not (or if they increase at a slower rate than you anticipated), you’ll roll out some other tests like changing the CTA language or the placement of the subscribe form on the page.

5 ways to improve your critical thinking skills

Beyond the buzzwords: Why interpersonal skills matter at work

Think critically about critical thinking and you’ll quickly realize that it’s not as instinctive as you’d like it to be. Fortunately, your critical thinking skills are learned competencies and not inherent gifts – and that means you can improve them. Here’s how:

• Practice active listening: Active listening helps you process and understand what other people share. That’s crucial as you aim to be open-minded and inquisitive.
• Ask open-ended questions: If your critical thinking process involves collecting feedback and opinions from others, ask open-ended questions (meaning, questions that can’t be answered with “yes” or “no”). Doing so will give you more valuable information and also prevent your own biases from influencing people’s input.
• Scrutinize your sources: Figuring out what to trust and prioritize is crucial for critical thinking. Boosting your media literacy and asking more questions will help you be more discerning about what to factor in. It’s hard to strike a balance between skepticism and open-mindedness, but approaching information with questions (rather than unquestioning trust) will help you draw better conclusions. 
• Play a game: Remember those riddles we mentioned at the beginning? As trivial as they might seem, games and exercises like those can help you boost your critical thinking skills. There are plenty of critical thinking exercises you can do individually or as a team . 
• Give yourself time: Research shows that rushed decisions are often regrettable ones. That’s likely because critical thinking takes time – you can’t do it under the wire. So, for big decisions or hairy problems, give yourself enough time and breathing room to work through the process. It’s hard enough to think critically without a countdown ticking in your brain. 

Critical thinking really is critical

The ability to think critically is important, but it doesn’t come naturally to most of us. It’s just easier to stick with biases, assumptions, and surface-level information. 

But that route often leads you to rash judgments, shaky conclusions, and disappointing decisions. So here’s a conclusion we can draw without any more noodling: Even if it is more demanding on your mental resources, critical thinking is well worth the effort.

Advice, stories, and expertise about work life today.

• No categories to show. Sorry!

Mind Tools Store

About Mind Tools Content

Discover something new today

Recommended for you.

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

A Short Guide to Building Your Team’s Critical Thinking Skills

• Matt Plummer

Critical thinking isn’t an innate skill. It can be learned.

Most employers lack an effective way to objectively assess critical thinking skills and most managers don’t know how to provide specific instruction to team members in need of becoming better thinkers. Instead, most managers employ a sink-or-swim approach, ultimately creating work-arounds to keep those who can’t figure out how to “swim” from making important decisions. But it doesn’t have to be this way. To demystify what critical thinking is and how it is developed, the author’s team turned to three research-backed models: The Halpern Critical Thinking Assessment, Pearson’s RED Critical Thinking Model, and Bloom’s Taxonomy. Using these models, they developed the Critical Thinking Roadmap, a framework that breaks critical thinking down into four measurable phases: the ability to execute, synthesize, recommend, and generate.

With critical thinking ranking among the most in-demand skills for job candidates , you would think that educational institutions would prepare candidates well to be exceptional thinkers, and employers would be adept at developing such skills in existing employees. Unfortunately, both are largely untrue.

• Matt Plummer (@mtplummer) is the founder of Zarvana, which offers online programs and coaching services to help working professionals become more productive by developing time-saving habits. Before starting Zarvana, Matt spent six years at Bain & Company spin-out, The Bridgespan Group, a strategy and management consulting firm for nonprofits, foundations, and philanthropists.  

Partner Center

.css-s5s6ko{margin-right:42px;color:#F5F4F3;}@media (max-width: 1120px){.css-s5s6ko{margin-right:12px;}} Join us: Learn how to build a trusted AI strategy to support your company's intelligent transformation, featuring Forrester .css-1ixh9fn{display:inline-block;}@media (max-width: 480px){.css-1ixh9fn{display:block;margin-top:12px;}} .css-1uaoevr-heading-6{font-size:14px;line-height:24px;font-weight:500;-webkit-text-decoration:underline;text-decoration:underline;color:#F5F4F3;}.css-1uaoevr-heading-6:hover{color:#F5F4F3;} .css-ora5nu-heading-6{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:start;-ms-flex-pack:start;-webkit-justify-content:flex-start;justify-content:flex-start;color:#0D0E10;-webkit-transition:all 0.3s;transition:all 0.3s;position:relative;font-size:16px;line-height:28px;padding:0;font-size:14px;line-height:24px;font-weight:500;-webkit-text-decoration:underline;text-decoration:underline;color:#F5F4F3;}.css-ora5nu-heading-6:hover{border-bottom:0;color:#CD4848;}.css-ora5nu-heading-6:hover path{fill:#CD4848;}.css-ora5nu-heading-6:hover div{border-color:#CD4848;}.css-ora5nu-heading-6:hover div:before{border-left-color:#CD4848;}.css-ora5nu-heading-6:active{border-bottom:0;background-color:#EBE8E8;color:#0D0E10;}.css-ora5nu-heading-6:active path{fill:#0D0E10;}.css-ora5nu-heading-6:active div{border-color:#0D0E10;}.css-ora5nu-heading-6:active div:before{border-left-color:#0D0E10;}.css-ora5nu-heading-6:hover{color:#F5F4F3;} Register now .css-1k6cidy{width:11px;height:11px;margin-left:8px;}.css-1k6cidy path{fill:currentColor;}

• Collaboration |
• How to build your critical thinking ski ...

How to build your critical thinking skills in 7 steps (with examples)

Critical thinking is, well, critical. By building these skills, you improve your ability to analyze information and come to the best decision possible. In this article, we cover the basics of critical thinking, as well as the seven steps you can use to implement the full critical thinking process. 

Critical thinking comes from asking the right questions to come to the best conclusion possible. Strong critical thinkers analyze information from a variety of viewpoints in order to identify the best course of action.

Don’t worry if you don’t think you have strong critical thinking abilities. In this article, we’ll help you build a foundation for critical thinking so you can absorb, analyze, and make informed decisions. 

What is critical thinking? 

Critical thinking is the ability to collect and analyze information to come to a conclusion. Being able to think critically is important in virtually every industry and applicable across a wide range of positions. That’s because critical thinking isn’t subject-specific—rather, it’s your ability to parse through information, data, statistics, and other details in order to identify a satisfactory solution. 

Decision-making tools for agile businesses

In this ebook, learn how to equip employees to make better decisions—so your business can pivot, adapt, and tackle challenges more effectively than your competition.

Top 8 critical thinking skills

Like most soft skills, critical thinking isn’t something you can take a class to learn. Rather, this skill consists of a variety of interpersonal and analytical skills. Developing critical thinking is more about learning to embrace open-mindedness and bringing analytical thinking to your problem framing process. 

In no particular order, the eight most important critical thinking skills are:

Analytical thinking: Part of critical thinking is evaluating data from multiple sources in order to come to the best conclusions. Analytical thinking allows people to reject bias and strive to gather and consume information to come to the best conclusion. 

Open-mindedness: This critical thinking skill helps you analyze and process information to come to an unbiased conclusion. Part of the critical thinking process is letting your personal biases go and coming to a conclusion based on all of the information. 

Problem solving : Because critical thinking emphasizes coming to the best conclusion based on all of the available information, it’s a key part of problem solving. When used correctly, critical thinking helps you solve any problem—from a workplace challenge to difficulties in everyday life. 

Self-regulation: Self-regulation refers to the ability to regulate your thoughts and set aside any personal biases to come to the best conclusion. In order to be an effective critical thinker, you need to question the information you have and the decisions you favor—only then can you come to the best conclusion. 

Observation: Observation skills help critical thinkers look for things beyond face value. To be a critical thinker you need to embrace multiple points of view, and you can use observation skills to identify potential problems.

Interpretation: Not all data is made equal—and critical thinkers know this. In addition to gathering information, it’s important to evaluate which information is important and relevant to your situation. That way, you can draw the best conclusions from the data you’ve collected. 

Evaluation: When you attempt to answer a hard question, there is rarely an obvious answer. Even though critical thinking emphasizes putting your biases aside, you need to be able to confidently make a decision based on the data you have available. 

Communication: Once a decision has been made, you also need to share this decision with other stakeholders. Effective workplace communication includes presenting evidence and supporting your conclusion—especially if there are a variety of different possible solutions. 

7 steps to critical thinking

Critical thinking is a skill that you can build by following these seven steps. The seven steps to critical thinking help you ensure you’re approaching a problem from the right angle, considering every alternative, and coming to an unbiased conclusion.

 First things first: When to use the 7 step critical thinking process

There’s a lot that goes into the full critical thinking process, and not every decision needs to be this thought out. Sometimes, it’s enough to put aside bias and approach a process logically. In other, more complex cases, the best way to identify the ideal outcome is to go through the entire critical thinking process. 

The seven-step critical thinking process is useful for complex decisions in areas you are less familiar with. Alternatively, the seven critical thinking steps can help you look at a problem you’re familiar with from a different angle, without any bias. 

If you need to make a less complex decision, consider another problem solving strategy instead. Decision matrices are a great way to identify the best option between different choices. Check out our article on 7 steps to creating a decision matrix .

1. Identify the problem

Before you put those critical thinking skills to work, you first need to identify the problem you’re solving. This step includes taking a look at the problem from a few different perspectives and asking questions like: 

What’s happening? 

Why is this happening? 

What assumptions am I making? 

At first glance, how do I think we can solve this problem? 

A big part of developing your critical thinking skills is learning how to come to unbiased conclusions. In order to do that, you first need to acknowledge the biases that you currently have. Does someone on your team think they know the answer? Are you making assumptions that aren’t necessarily true? Identifying these details helps you later on in the process. 

2. Research

At this point, you likely have a general idea of the problem—but in order to come up with the best solution, you need to dig deeper. 

During the research process, collect information relating to the problem, including data, statistics, historical project information, team input, and more. Make sure you gather information from a variety of sources, especially if those sources go against your personal ideas about what the problem is or how to solve it.

Gathering varied information is essential for your ability to apply the critical thinking process. If you don’t get enough information, your ability to make a final decision will be skewed. Remember that critical thinking is about helping you identify the objective best conclusion. You aren’t going with your gut—you’re doing research to find the best option

3. Determine data relevance

Just as it’s important to gather a variety of information, it is also important to determine how relevant the different information sources are. After all, just because there is data doesn’t mean it’s relevant. 

Once you’ve gathered all of the information, sift through the noise and identify what information is relevant and what information isn’t. Synthesizing all of this information and establishing significance helps you weigh different data sources and come to the best conclusion later on in the critical thinking process. 

To determine data relevance, ask yourself:

How reliable is this information? 

How significant is this information? 

Is this information outdated? Is it specialized in a specific field? 

4. Ask questions

One of the most useful parts of the critical thinking process is coming to a decision without bias. In order to do so, you need to take a step back from the process and challenge the assumptions you’re making. 

We all have bias—and that isn’t necessarily a bad thing. Unconscious biases (also known as cognitive biases) often serve as mental shortcuts to simplify problem solving and aid decision making. But even when biases aren’t inherently bad, you must be aware of your biases in order to put them aside when necessary. 

Before coming to a solution, ask yourself:

Am I making any assumptions about this information? 

Are there additional variables I haven’t considered? 

Have I evaluated the information from every perspective? 

Are there any viewpoints I missed? 

5. Identify the best solution

Finally, you’re ready to come to a conclusion. To identify the best solution, draw connections between causes and effects. Use the facts you’ve gathered to evaluate the most objective conclusion. 

Keep in mind that there may be more than one solution. Often, the problems you’re facing are complex and intricate. The critical thinking process doesn’t necessarily lead to a cut-and-dry solution—instead, the process helps you understand the different variables at play so you can make an informed decision. 

6. Present your solution

Communication is a key skill for critical thinkers. It isn’t enough to think for yourself—you also need to share your conclusion with other project stakeholders. If there are multiple solutions, present them all. There may be a case where you implement one solution, then test to see if it works before implementing another solution. 

7. Analyze your decision

The seven-step critical thinking process yields a result—and you then need to put that solution into place. After you’ve implemented your decision, evaluate whether or not it was effective. Did it solve the initial problem? What lessons—whether positive or negative—can you learn from this experience to improve your critical thinking for next time? 

Depending on how your team shares information, consider documenting lessons learned in a central source of truth. That way, team members that are making similar or related decisions in the future can understand why you made the decision you made and what the outcome was. 

Example of critical thinking in the workplace

Imagine you work in user experience design (UX). Your team is focused on pricing and packaging and ensuring customers have a clear understanding of the different services your company offers. Here’s how to apply the critical thinking process in the workplace in seven steps: 

Start by identifying the problem

Your current pricing page isn’t performing as well as you want. You’ve heard from customers that your services aren’t clear, and that the page doesn’t answer the questions they have. This page is really important for your company, since it’s where your customers sign up for your service. You and your team have a few theories about why your current page isn’t performing well, but you decide to apply the critical thinking process to ensure you come to the best decision for the page. 

Gather information about how the problem started

Part of identifying the problem includes understanding how the problem started. The pricing and packaging page is important—so when your team initially designed the page, they certainly put a lot of thought into it. Before you begin researching how to improve the page, ask yourself: 

Why did you design the pricing page the way you did? 

Which stakeholders need to be involved in the decision making process? 

Where are users getting stuck on the page?

Are any features currently working?

Then, you research

In addition to understanding the history of the pricing and packaging page, it’s important to understand what works well. Part of this research means taking a look at what your competitor’s pricing pages look like. 

Ask yourself: 

How have our competitors set up their pricing pages?

Are there any pricing page best practices? 

How does color, positioning, and animation impact navigation? 

Are there any standard page layouts customers expect to see? 

Organize and analyze information

You’ve gathered all of the information you need—now you need to organize and analyze it. What trends, if any, are you noticing? Is there any particularly relevant or important information that you have to consider? 

Ask open-ended questions to reduce bias

In the case of critical thinking, it’s important to address and set bias aside as much as possible. Ask yourself: 

Is there anything I’m missing? 

Have I connected with the right stakeholders? 

Are there any other viewpoints I should consider? 

Determine the best solution for your team

You now have all of the information you need to design the best pricing page. Depending on the complexity of the design, you may want to design a few options to present to a small group of customers or A/B test on the live website.

Present your solution to stakeholders

Critical thinking can help you in every element of your life, but in the workplace, you must also involve key project stakeholders . Stakeholders help you determine next steps, like whether you’ll A/B test the page first. Depending on the complexity of the issue, consider hosting a meeting or sharing a status report to get everyone on the same page. 

Analyze the results

No process is complete without evaluating the results. Once the new page has been live for some time, evaluate whether it did better than the previous page. What worked? What didn’t? This also helps you make better critical decisions later on.

Critically successful 

Critical thinking takes time to build, but with effort and patience you can apply an unbiased, analytical mind to any situation. Critical thinking makes up one of many soft skills that makes you an effective team member, manager, and worker. If you’re looking to hone your skills further, read our article on the 25 project management skills you need to succeed . 

Related resources

Unmanaged business goals don’t work. Here’s what does.

How Asana uses work management to drive product development

How Asana uses work management to streamline project intake processes

How Asana uses work management for smoother creative production

• Table of Contents
• Random Entry
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Advanced Tools
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Entry Contents

Bibliography

Academic tools.

• Friends PDF Preview
• Author and Citation Info
• Back to Top

Critical Thinking

Critical thinking is a widely accepted educational goal. Its definition is contested, but the competing definitions can be understood as differing conceptions of the same basic concept: careful thinking directed to a goal. Conceptions differ with respect to the scope of such thinking, the type of goal, the criteria and norms for thinking carefully, and the thinking components on which they focus. Its adoption as an educational goal has been recommended on the basis of respect for students’ autonomy and preparing students for success in life and for democratic citizenship. “Critical thinkers” have the dispositions and abilities that lead them to think critically when appropriate. The abilities can be identified directly; the dispositions indirectly, by considering what factors contribute to or impede exercise of the abilities. Standardized tests have been developed to assess the degree to which a person possesses such dispositions and abilities. Educational intervention has been shown experimentally to improve them, particularly when it includes dialogue, anchored instruction, and mentoring. Controversies have arisen over the generalizability of critical thinking across domains, over alleged bias in critical thinking theories and instruction, and over the relationship of critical thinking to other types of thinking.

2.1 Dewey’s Three Main Examples

2.2 dewey’s other examples, 2.3 further examples, 2.4 non-examples, 3. the definition of critical thinking, 4. its value, 5. the process of thinking critically, 6. components of the process, 7. contributory dispositions and abilities, 8.1 initiating dispositions, 8.2 internal dispositions, 9. critical thinking abilities, 10. required knowledge, 11. educational methods, 12.1 the generalizability of critical thinking, 12.2 bias in critical thinking theory and pedagogy, 12.3 relationship of critical thinking to other types of thinking, other internet resources, related entries.

Use of the term ‘critical thinking’ to describe an educational goal goes back to the American philosopher John Dewey (1910), who more commonly called it ‘reflective thinking’. He defined it as

active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it, and the further conclusions to which it tends. (Dewey 1910: 6; 1933: 9)

and identified a habit of such consideration with a scientific attitude of mind. His lengthy quotations of Francis Bacon, John Locke, and John Stuart Mill indicate that he was not the first person to propose development of a scientific attitude of mind as an educational goal.

In the 1930s, many of the schools that participated in the Eight-Year Study of the Progressive Education Association (Aikin 1942) adopted critical thinking as an educational goal, for whose achievement the study’s Evaluation Staff developed tests (Smith, Tyler, & Evaluation Staff 1942). Glaser (1941) showed experimentally that it was possible to improve the critical thinking of high school students. Bloom’s influential taxonomy of cognitive educational objectives (Bloom et al. 1956) incorporated critical thinking abilities. Ennis (1962) proposed 12 aspects of critical thinking as a basis for research on the teaching and evaluation of critical thinking ability.

Since 1980, an annual international conference in California on critical thinking and educational reform has attracted tens of thousands of educators from all levels of education and from many parts of the world. Also since 1980, the state university system in California has required all undergraduate students to take a critical thinking course. Since 1983, the Association for Informal Logic and Critical Thinking has sponsored sessions in conjunction with the divisional meetings of the American Philosophical Association (APA). In 1987, the APA’s Committee on Pre-College Philosophy commissioned a consensus statement on critical thinking for purposes of educational assessment and instruction (Facione 1990a). Researchers have developed standardized tests of critical thinking abilities and dispositions; for details, see the Supplement on Assessment . Educational jurisdictions around the world now include critical thinking in guidelines for curriculum and assessment.

For details on this history, see the Supplement on History .

2. Examples and Non-Examples

Before considering the definition of critical thinking, it will be helpful to have in mind some examples of critical thinking, as well as some examples of kinds of thinking that would apparently not count as critical thinking.

Dewey (1910: 68–71; 1933: 91–94) takes as paradigms of reflective thinking three class papers of students in which they describe their thinking. The examples range from the everyday to the scientific.

Transit : “The other day, when I was down town on 16th Street, a clock caught my eye. I saw that the hands pointed to 12:20. This suggested that I had an engagement at 124th Street, at one o’clock. I reasoned that as it had taken me an hour to come down on a surface car, I should probably be twenty minutes late if I returned the same way. I might save twenty minutes by a subway express. But was there a station near? If not, I might lose more than twenty minutes in looking for one. Then I thought of the elevated, and I saw there was such a line within two blocks. But where was the station? If it were several blocks above or below the street I was on, I should lose time instead of gaining it. My mind went back to the subway express as quicker than the elevated; furthermore, I remembered that it went nearer than the elevated to the part of 124th Street I wished to reach, so that time would be saved at the end of the journey. I concluded in favor of the subway, and reached my destination by one o’clock.” (Dewey 1910: 68–69; 1933: 91–92)

Ferryboat : “Projecting nearly horizontally from the upper deck of the ferryboat on which I daily cross the river is a long white pole, having a gilded ball at its tip. It suggested a flagpole when I first saw it; its color, shape, and gilded ball agreed with this idea, and these reasons seemed to justify me in this belief. But soon difficulties presented themselves. The pole was nearly horizontal, an unusual position for a flagpole; in the next place, there was no pulley, ring, or cord by which to attach a flag; finally, there were elsewhere on the boat two vertical staffs from which flags were occasionally flown. It seemed probable that the pole was not there for flag-flying.

“I then tried to imagine all possible purposes of the pole, and to consider for which of these it was best suited: (a) Possibly it was an ornament. But as all the ferryboats and even the tugboats carried poles, this hypothesis was rejected. (b) Possibly it was the terminal of a wireless telegraph. But the same considerations made this improbable. Besides, the more natural place for such a terminal would be the highest part of the boat, on top of the pilot house. (c) Its purpose might be to point out the direction in which the boat is moving.

“In support of this conclusion, I discovered that the pole was lower than the pilot house, so that the steersman could easily see it. Moreover, the tip was enough higher than the base, so that, from the pilot’s position, it must appear to project far out in front of the boat. Moreover, the pilot being near the front of the boat, he would need some such guide as to its direction. Tugboats would also need poles for such a purpose. This hypothesis was so much more probable than the others that I accepted it. I formed the conclusion that the pole was set up for the purpose of showing the pilot the direction in which the boat pointed, to enable him to steer correctly.” (Dewey 1910: 69–70; 1933: 92–93)

Bubbles : “In washing tumblers in hot soapsuds and placing them mouth downward on a plate, bubbles appeared on the outside of the mouth of the tumblers and then went inside. Why? The presence of bubbles suggests air, which I note must come from inside the tumbler. I see that the soapy water on the plate prevents escape of the air save as it may be caught in bubbles. But why should air leave the tumbler? There was no substance entering to force it out. It must have expanded. It expands by increase of heat, or by decrease of pressure, or both. Could the air have become heated after the tumbler was taken from the hot suds? Clearly not the air that was already entangled in the water. If heated air was the cause, cold air must have entered in transferring the tumblers from the suds to the plate. I test to see if this supposition is true by taking several more tumblers out. Some I shake so as to make sure of entrapping cold air in them. Some I take out holding mouth downward in order to prevent cold air from entering. Bubbles appear on the outside of every one of the former and on none of the latter. I must be right in my inference. Air from the outside must have been expanded by the heat of the tumbler, which explains the appearance of the bubbles on the outside. But why do they then go inside? Cold contracts. The tumbler cooled and also the air inside it. Tension was removed, and hence bubbles appeared inside. To be sure of this, I test by placing a cup of ice on the tumbler while the bubbles are still forming outside. They soon reverse” (Dewey 1910: 70–71; 1933: 93–94).

Dewey (1910, 1933) sprinkles his book with other examples of critical thinking. We will refer to the following.

Weather : A man on a walk notices that it has suddenly become cool, thinks that it is probably going to rain, looks up and sees a dark cloud obscuring the sun, and quickens his steps (1910: 6–10; 1933: 9–13).

Disorder : A man finds his rooms on his return to them in disorder with his belongings thrown about, thinks at first of burglary as an explanation, then thinks of mischievous children as being an alternative explanation, then looks to see whether valuables are missing, and discovers that they are (1910: 82–83; 1933: 166–168).

Typhoid : A physician diagnosing a patient whose conspicuous symptoms suggest typhoid avoids drawing a conclusion until more data are gathered by questioning the patient and by making tests (1910: 85–86; 1933: 170).

Blur : A moving blur catches our eye in the distance, we ask ourselves whether it is a cloud of whirling dust or a tree moving its branches or a man signaling to us, we think of other traits that should be found on each of those possibilities, and we look and see if those traits are found (1910: 102, 108; 1933: 121, 133).

Suction pump : In thinking about the suction pump, the scientist first notes that it will draw water only to a maximum height of 33 feet at sea level and to a lesser maximum height at higher elevations, selects for attention the differing atmospheric pressure at these elevations, sets up experiments in which the air is removed from a vessel containing water (when suction no longer works) and in which the weight of air at various levels is calculated, compares the results of reasoning about the height to which a given weight of air will allow a suction pump to raise water with the observed maximum height at different elevations, and finally assimilates the suction pump to such apparently different phenomena as the siphon and the rising of a balloon (1910: 150–153; 1933: 195–198).

Diamond : A passenger in a car driving in a diamond lane reserved for vehicles with at least one passenger notices that the diamond marks on the pavement are far apart in some places and close together in others. Why? The driver suggests that the reason may be that the diamond marks are not needed where there is a solid double line separating the diamond lane from the adjoining lane, but are needed when there is a dotted single line permitting crossing into the diamond lane. Further observation confirms that the diamonds are close together when a dotted line separates the diamond lane from its neighbour, but otherwise far apart.

Rash : A woman suddenly develops a very itchy red rash on her throat and upper chest. She recently noticed a mark on the back of her right hand, but was not sure whether the mark was a rash or a scrape. She lies down in bed and thinks about what might be causing the rash and what to do about it. About two weeks before, she began taking blood pressure medication that contained a sulfa drug, and the pharmacist had warned her, in view of a previous allergic reaction to a medication containing a sulfa drug, to be on the alert for an allergic reaction; however, she had been taking the medication for two weeks with no such effect. The day before, she began using a new cream on her neck and upper chest; against the new cream as the cause was mark on the back of her hand, which had not been exposed to the cream. She began taking probiotics about a month before. She also recently started new eye drops, but she supposed that manufacturers of eye drops would be careful not to include allergy-causing components in the medication. The rash might be a heat rash, since she recently was sweating profusely from her upper body. Since she is about to go away on a short vacation, where she would not have access to her usual physician, she decides to keep taking the probiotics and using the new eye drops but to discontinue the blood pressure medication and to switch back to the old cream for her neck and upper chest. She forms a plan to consult her regular physician on her return about the blood pressure medication.

Candidate : Although Dewey included no examples of thinking directed at appraising the arguments of others, such thinking has come to be considered a kind of critical thinking. We find an example of such thinking in the performance task on the Collegiate Learning Assessment (CLA+), which its sponsoring organization describes as

a performance-based assessment that provides a measure of an institution’s contribution to the development of critical-thinking and written communication skills of its students. (Council for Aid to Education 2017)

A sample task posted on its website requires the test-taker to write a report for public distribution evaluating a fictional candidate’s policy proposals and their supporting arguments, using supplied background documents, with a recommendation on whether to endorse the candidate.

Immediate acceptance of an idea that suggests itself as a solution to a problem (e.g., a possible explanation of an event or phenomenon, an action that seems likely to produce a desired result) is “uncritical thinking, the minimum of reflection” (Dewey 1910: 13). On-going suspension of judgment in the light of doubt about a possible solution is not critical thinking (Dewey 1910: 108). Critique driven by a dogmatically held political or religious ideology is not critical thinking; thus Paulo Freire (1968 [1970]) is using the term (e.g., at 1970: 71, 81, 100, 146) in a more politically freighted sense that includes not only reflection but also revolutionary action against oppression. Derivation of a conclusion from given data using an algorithm is not critical thinking.

What is critical thinking? There are many definitions. Ennis (2016) lists 14 philosophically oriented scholarly definitions and three dictionary definitions. Following Rawls (1971), who distinguished his conception of justice from a utilitarian conception but regarded them as rival conceptions of the same concept, Ennis maintains that the 17 definitions are different conceptions of the same concept. Rawls articulated the shared concept of justice as

a characteristic set of principles for assigning basic rights and duties and for determining… the proper distribution of the benefits and burdens of social cooperation. (Rawls 1971: 5)

Bailin et al. (1999b) claim that, if one considers what sorts of thinking an educator would take not to be critical thinking and what sorts to be critical thinking, one can conclude that educators typically understand critical thinking to have at least three features.

• It is done for the purpose of making up one’s mind about what to believe or do.
• The person engaging in the thinking is trying to fulfill standards of adequacy and accuracy appropriate to the thinking.
• The thinking fulfills the relevant standards to some threshold level.

One could sum up the core concept that involves these three features by saying that critical thinking is careful goal-directed thinking. This core concept seems to apply to all the examples of critical thinking described in the previous section. As for the non-examples, their exclusion depends on construing careful thinking as excluding jumping immediately to conclusions, suspending judgment no matter how strong the evidence, reasoning from an unquestioned ideological or religious perspective, and routinely using an algorithm to answer a question.

If the core of critical thinking is careful goal-directed thinking, conceptions of it can vary according to its presumed scope, its presumed goal, one’s criteria and threshold for being careful, and the thinking component on which one focuses. As to its scope, some conceptions (e.g., Dewey 1910, 1933) restrict it to constructive thinking on the basis of one’s own observations and experiments, others (e.g., Ennis 1962; Fisher & Scriven 1997; Johnson 1992) to appraisal of the products of such thinking. Ennis (1991) and Bailin et al. (1999b) take it to cover both construction and appraisal. As to its goal, some conceptions restrict it to forming a judgment (Dewey 1910, 1933; Lipman 1987; Facione 1990a). Others allow for actions as well as beliefs as the end point of a process of critical thinking (Ennis 1991; Bailin et al. 1999b). As to the criteria and threshold for being careful, definitions vary in the term used to indicate that critical thinking satisfies certain norms: “intellectually disciplined” (Scriven & Paul 1987), “reasonable” (Ennis 1991), “skillful” (Lipman 1987), “skilled” (Fisher & Scriven 1997), “careful” (Bailin & Battersby 2009). Some definitions specify these norms, referring variously to “consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it tends” (Dewey 1910, 1933); “the methods of logical inquiry and reasoning” (Glaser 1941); “conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication” (Scriven & Paul 1987); the requirement that “it is sensitive to context, relies on criteria, and is self-correcting” (Lipman 1987); “evidential, conceptual, methodological, criteriological, or contextual considerations” (Facione 1990a); and “plus-minus considerations of the product in terms of appropriate standards (or criteria)” (Johnson 1992). Stanovich and Stanovich (2010) propose to ground the concept of critical thinking in the concept of rationality, which they understand as combining epistemic rationality (fitting one’s beliefs to the world) and instrumental rationality (optimizing goal fulfillment); a critical thinker, in their view, is someone with “a propensity to override suboptimal responses from the autonomous mind” (2010: 227). These variant specifications of norms for critical thinking are not necessarily incompatible with one another, and in any case presuppose the core notion of thinking carefully. As to the thinking component singled out, some definitions focus on suspension of judgment during the thinking (Dewey 1910; McPeck 1981), others on inquiry while judgment is suspended (Bailin & Battersby 2009, 2021), others on the resulting judgment (Facione 1990a), and still others on responsiveness to reasons (Siegel 1988). Kuhn (2019) takes critical thinking to be more a dialogic practice of advancing and responding to arguments than an individual ability.

In educational contexts, a definition of critical thinking is a “programmatic definition” (Scheffler 1960: 19). It expresses a practical program for achieving an educational goal. For this purpose, a one-sentence formulaic definition is much less useful than articulation of a critical thinking process, with criteria and standards for the kinds of thinking that the process may involve. The real educational goal is recognition, adoption and implementation by students of those criteria and standards. That adoption and implementation in turn consists in acquiring the knowledge, abilities and dispositions of a critical thinker.

Conceptions of critical thinking generally do not include moral integrity as part of the concept. Dewey, for example, took critical thinking to be the ultimate intellectual goal of education, but distinguished it from the development of social cooperation among school children, which he took to be the central moral goal. Ennis (1996, 2011) added to his previous list of critical thinking dispositions a group of dispositions to care about the dignity and worth of every person, which he described as a “correlative” (1996) disposition without which critical thinking would be less valuable and perhaps harmful. An educational program that aimed at developing critical thinking but not the correlative disposition to care about the dignity and worth of every person, he asserted, “would be deficient and perhaps dangerous” (Ennis 1996: 172).

Dewey thought that education for reflective thinking would be of value to both the individual and society; recognition in educational practice of the kinship to the scientific attitude of children’s native curiosity, fertile imagination and love of experimental inquiry “would make for individual happiness and the reduction of social waste” (Dewey 1910: iii). Schools participating in the Eight-Year Study took development of the habit of reflective thinking and skill in solving problems as a means to leading young people to understand, appreciate and live the democratic way of life characteristic of the United States (Aikin 1942: 17–18, 81). Harvey Siegel (1988: 55–61) has offered four considerations in support of adopting critical thinking as an educational ideal. (1) Respect for persons requires that schools and teachers honour students’ demands for reasons and explanations, deal with students honestly, and recognize the need to confront students’ independent judgment; these requirements concern the manner in which teachers treat students. (2) Education has the task of preparing children to be successful adults, a task that requires development of their self-sufficiency. (3) Education should initiate children into the rational traditions in such fields as history, science and mathematics. (4) Education should prepare children to become democratic citizens, which requires reasoned procedures and critical talents and attitudes. To supplement these considerations, Siegel (1988: 62–90) responds to two objections: the ideology objection that adoption of any educational ideal requires a prior ideological commitment and the indoctrination objection that cultivation of critical thinking cannot escape being a form of indoctrination.

Despite the diversity of our 11 examples, one can recognize a common pattern. Dewey analyzed it as consisting of five phases:

• suggestions , in which the mind leaps forward to a possible solution;
• an intellectualization of the difficulty or perplexity into a problem to be solved, a question for which the answer must be sought;
• the use of one suggestion after another as a leading idea, or hypothesis , to initiate and guide observation and other operations in collection of factual material;
• the mental elaboration of the idea or supposition as an idea or supposition ( reasoning , in the sense on which reasoning is a part, not the whole, of inference); and
• testing the hypothesis by overt or imaginative action. (Dewey 1933: 106–107; italics in original)

The process of reflective thinking consisting of these phases would be preceded by a perplexed, troubled or confused situation and followed by a cleared-up, unified, resolved situation (Dewey 1933: 106). The term ‘phases’ replaced the term ‘steps’ (Dewey 1910: 72), thus removing the earlier suggestion of an invariant sequence. Variants of the above analysis appeared in (Dewey 1916: 177) and (Dewey 1938: 101–119).

The variant formulations indicate the difficulty of giving a single logical analysis of such a varied process. The process of critical thinking may have a spiral pattern, with the problem being redefined in the light of obstacles to solving it as originally formulated. For example, the person in Transit might have concluded that getting to the appointment at the scheduled time was impossible and have reformulated the problem as that of rescheduling the appointment for a mutually convenient time. Further, defining a problem does not always follow after or lead immediately to an idea of a suggested solution. Nor should it do so, as Dewey himself recognized in describing the physician in Typhoid as avoiding any strong preference for this or that conclusion before getting further information (Dewey 1910: 85; 1933: 170). People with a hypothesis in mind, even one to which they have a very weak commitment, have a so-called “confirmation bias” (Nickerson 1998): they are likely to pay attention to evidence that confirms the hypothesis and to ignore evidence that counts against it or for some competing hypothesis. Detectives, intelligence agencies, and investigators of airplane accidents are well advised to gather relevant evidence systematically and to postpone even tentative adoption of an explanatory hypothesis until the collected evidence rules out with the appropriate degree of certainty all but one explanation. Dewey’s analysis of the critical thinking process can be faulted as well for requiring acceptance or rejection of a possible solution to a defined problem, with no allowance for deciding in the light of the available evidence to suspend judgment. Further, given the great variety of kinds of problems for which reflection is appropriate, there is likely to be variation in its component events. Perhaps the best way to conceptualize the critical thinking process is as a checklist whose component events can occur in a variety of orders, selectively, and more than once. These component events might include (1) noticing a difficulty, (2) defining the problem, (3) dividing the problem into manageable sub-problems, (4) formulating a variety of possible solutions to the problem or sub-problem, (5) determining what evidence is relevant to deciding among possible solutions to the problem or sub-problem, (6) devising a plan of systematic observation or experiment that will uncover the relevant evidence, (7) carrying out the plan of systematic observation or experimentation, (8) noting the results of the systematic observation or experiment, (9) gathering relevant testimony and information from others, (10) judging the credibility of testimony and information gathered from others, (11) drawing conclusions from gathered evidence and accepted testimony, and (12) accepting a solution that the evidence adequately supports (cf. Hitchcock 2017: 485).

Checklist conceptions of the process of critical thinking are open to the objection that they are too mechanical and procedural to fit the multi-dimensional and emotionally charged issues for which critical thinking is urgently needed (Paul 1984). For such issues, a more dialectical process is advocated, in which competing relevant world views are identified, their implications explored, and some sort of creative synthesis attempted.

If one considers the critical thinking process illustrated by the 11 examples, one can identify distinct kinds of mental acts and mental states that form part of it. To distinguish, label and briefly characterize these components is a useful preliminary to identifying abilities, skills, dispositions, attitudes, habits and the like that contribute causally to thinking critically. Identifying such abilities and habits is in turn a useful preliminary to setting educational goals. Setting the goals is in its turn a useful preliminary to designing strategies for helping learners to achieve the goals and to designing ways of measuring the extent to which learners have done so. Such measures provide both feedback to learners on their achievement and a basis for experimental research on the effectiveness of various strategies for educating people to think critically. Let us begin, then, by distinguishing the kinds of mental acts and mental events that can occur in a critical thinking process.

• Observing : One notices something in one’s immediate environment (sudden cooling of temperature in Weather , bubbles forming outside a glass and then going inside in Bubbles , a moving blur in the distance in Blur , a rash in Rash ). Or one notes the results of an experiment or systematic observation (valuables missing in Disorder , no suction without air pressure in Suction pump )
• Feeling : One feels puzzled or uncertain about something (how to get to an appointment on time in Transit , why the diamonds vary in spacing in Diamond ). One wants to resolve this perplexity. One feels satisfaction once one has worked out an answer (to take the subway express in Transit , diamonds closer when needed as a warning in Diamond ).
• Wondering : One formulates a question to be addressed (why bubbles form outside a tumbler taken from hot water in Bubbles , how suction pumps work in Suction pump , what caused the rash in Rash ).
• Imagining : One thinks of possible answers (bus or subway or elevated in Transit , flagpole or ornament or wireless communication aid or direction indicator in Ferryboat , allergic reaction or heat rash in Rash ).
• Inferring : One works out what would be the case if a possible answer were assumed (valuables missing if there has been a burglary in Disorder , earlier start to the rash if it is an allergic reaction to a sulfa drug in Rash ). Or one draws a conclusion once sufficient relevant evidence is gathered (take the subway in Transit , burglary in Disorder , discontinue blood pressure medication and new cream in Rash ).
• Knowledge : One uses stored knowledge of the subject-matter to generate possible answers or to infer what would be expected on the assumption of a particular answer (knowledge of a city’s public transit system in Transit , of the requirements for a flagpole in Ferryboat , of Boyle’s law in Bubbles , of allergic reactions in Rash ).
• Experimenting : One designs and carries out an experiment or a systematic observation to find out whether the results deduced from a possible answer will occur (looking at the location of the flagpole in relation to the pilot’s position in Ferryboat , putting an ice cube on top of a tumbler taken from hot water in Bubbles , measuring the height to which a suction pump will draw water at different elevations in Suction pump , noticing the spacing of diamonds when movement to or from a diamond lane is allowed in Diamond ).
• Consulting : One finds a source of information, gets the information from the source, and makes a judgment on whether to accept it. None of our 11 examples include searching for sources of information. In this respect they are unrepresentative, since most people nowadays have almost instant access to information relevant to answering any question, including many of those illustrated by the examples. However, Candidate includes the activities of extracting information from sources and evaluating its credibility.
• Identifying and analyzing arguments : One notices an argument and works out its structure and content as a preliminary to evaluating its strength. This activity is central to Candidate . It is an important part of a critical thinking process in which one surveys arguments for various positions on an issue.
• Judging : One makes a judgment on the basis of accumulated evidence and reasoning, such as the judgment in Ferryboat that the purpose of the pole is to provide direction to the pilot.
• Deciding : One makes a decision on what to do or on what policy to adopt, as in the decision in Transit to take the subway.

By definition, a person who does something voluntarily is both willing and able to do that thing at that time. Both the willingness and the ability contribute causally to the person’s action, in the sense that the voluntary action would not occur if either (or both) of these were lacking. For example, suppose that one is standing with one’s arms at one’s sides and one voluntarily lifts one’s right arm to an extended horizontal position. One would not do so if one were unable to lift one’s arm, if for example one’s right side was paralyzed as the result of a stroke. Nor would one do so if one were unwilling to lift one’s arm, if for example one were participating in a street demonstration at which a white supremacist was urging the crowd to lift their right arm in a Nazi salute and one were unwilling to express support in this way for the racist Nazi ideology. The same analysis applies to a voluntary mental process of thinking critically. It requires both willingness and ability to think critically, including willingness and ability to perform each of the mental acts that compose the process and to coordinate those acts in a sequence that is directed at resolving the initiating perplexity.

Consider willingness first. We can identify causal contributors to willingness to think critically by considering factors that would cause a person who was able to think critically about an issue nevertheless not to do so (Hamby 2014). For each factor, the opposite condition thus contributes causally to willingness to think critically on a particular occasion. For example, people who habitually jump to conclusions without considering alternatives will not think critically about issues that arise, even if they have the required abilities. The contrary condition of willingness to suspend judgment is thus a causal contributor to thinking critically.

Now consider ability. In contrast to the ability to move one’s arm, which can be completely absent because a stroke has left the arm paralyzed, the ability to think critically is a developed ability, whose absence is not a complete absence of ability to think but absence of ability to think well. We can identify the ability to think well directly, in terms of the norms and standards for good thinking. In general, to be able do well the thinking activities that can be components of a critical thinking process, one needs to know the concepts and principles that characterize their good performance, to recognize in particular cases that the concepts and principles apply, and to apply them. The knowledge, recognition and application may be procedural rather than declarative. It may be domain-specific rather than widely applicable, and in either case may need subject-matter knowledge, sometimes of a deep kind.

Reflections of the sort illustrated by the previous two paragraphs have led scholars to identify the knowledge, abilities and dispositions of a “critical thinker”, i.e., someone who thinks critically whenever it is appropriate to do so. We turn now to these three types of causal contributors to thinking critically. We start with dispositions, since arguably these are the most powerful contributors to being a critical thinker, can be fostered at an early stage of a child’s development, and are susceptible to general improvement (Glaser 1941: 175)

8. Critical Thinking Dispositions

Educational researchers use the term ‘dispositions’ broadly for the habits of mind and attitudes that contribute causally to being a critical thinker. Some writers (e.g., Paul & Elder 2006; Hamby 2014; Bailin & Battersby 2016a) propose to use the term ‘virtues’ for this dimension of a critical thinker. The virtues in question, although they are virtues of character, concern the person’s ways of thinking rather than the person’s ways of behaving towards others. They are not moral virtues but intellectual virtues, of the sort articulated by Zagzebski (1996) and discussed by Turri, Alfano, and Greco (2017).

On a realistic conception, thinking dispositions or intellectual virtues are real properties of thinkers. They are general tendencies, propensities, or inclinations to think in particular ways in particular circumstances, and can be genuinely explanatory (Siegel 1999). Sceptics argue that there is no evidence for a specific mental basis for the habits of mind that contribute to thinking critically, and that it is pedagogically misleading to posit such a basis (Bailin et al. 1999a). Whatever their status, critical thinking dispositions need motivation for their initial formation in a child—motivation that may be external or internal. As children develop, the force of habit will gradually become important in sustaining the disposition (Nieto & Valenzuela 2012). Mere force of habit, however, is unlikely to sustain critical thinking dispositions. Critical thinkers must value and enjoy using their knowledge and abilities to think things through for themselves. They must be committed to, and lovers of, inquiry.

A person may have a critical thinking disposition with respect to only some kinds of issues. For example, one could be open-minded about scientific issues but not about religious issues. Similarly, one could be confident in one’s ability to reason about the theological implications of the existence of evil in the world but not in one’s ability to reason about the best design for a guided ballistic missile.

Facione (1990a: 25) divides “affective dispositions” of critical thinking into approaches to life and living in general and approaches to specific issues, questions or problems. Adapting this distinction, one can usefully divide critical thinking dispositions into initiating dispositions (those that contribute causally to starting to think critically about an issue) and internal dispositions (those that contribute causally to doing a good job of thinking critically once one has started). The two categories are not mutually exclusive. For example, open-mindedness, in the sense of willingness to consider alternative points of view to one’s own, is both an initiating and an internal disposition.

Using the strategy of considering factors that would block people with the ability to think critically from doing so, we can identify as initiating dispositions for thinking critically attentiveness, a habit of inquiry, self-confidence, courage, open-mindedness, willingness to suspend judgment, trust in reason, wanting evidence for one’s beliefs, and seeking the truth. We consider briefly what each of these dispositions amounts to, in each case citing sources that acknowledge them.

• Attentiveness : One will not think critically if one fails to recognize an issue that needs to be thought through. For example, the pedestrian in Weather would not have looked up if he had not noticed that the air was suddenly cooler. To be a critical thinker, then, one needs to be habitually attentive to one’s surroundings, noticing not only what one senses but also sources of perplexity in messages received and in one’s own beliefs and attitudes (Facione 1990a: 25; Facione, Facione, & Giancarlo 2001).
• Habit of inquiry : Inquiry is effortful, and one needs an internal push to engage in it. For example, the student in Bubbles could easily have stopped at idle wondering about the cause of the bubbles rather than reasoning to a hypothesis, then designing and executing an experiment to test it. Thus willingness to think critically needs mental energy and initiative. What can supply that energy? Love of inquiry, or perhaps just a habit of inquiry. Hamby (2015) has argued that willingness to inquire is the central critical thinking virtue, one that encompasses all the others. It is recognized as a critical thinking disposition by Dewey (1910: 29; 1933: 35), Glaser (1941: 5), Ennis (1987: 12; 1991: 8), Facione (1990a: 25), Bailin et al. (1999b: 294), Halpern (1998: 452), and Facione, Facione, & Giancarlo (2001).
• Self-confidence : Lack of confidence in one’s abilities can block critical thinking. For example, if the woman in Rash lacked confidence in her ability to figure things out for herself, she might just have assumed that the rash on her chest was the allergic reaction to her medication against which the pharmacist had warned her. Thus willingness to think critically requires confidence in one’s ability to inquire (Facione 1990a: 25; Facione, Facione, & Giancarlo 2001).
• Courage : Fear of thinking for oneself can stop one from doing it. Thus willingness to think critically requires intellectual courage (Paul & Elder 2006: 16).
• Open-mindedness : A dogmatic attitude will impede thinking critically. For example, a person who adheres rigidly to a “pro-choice” position on the issue of the legal status of induced abortion is likely to be unwilling to consider seriously the issue of when in its development an unborn child acquires a moral right to life. Thus willingness to think critically requires open-mindedness, in the sense of a willingness to examine questions to which one already accepts an answer but which further evidence or reasoning might cause one to answer differently (Dewey 1933; Facione 1990a; Ennis 1991; Bailin et al. 1999b; Halpern 1998, Facione, Facione, & Giancarlo 2001). Paul (1981) emphasizes open-mindedness about alternative world-views, and recommends a dialectical approach to integrating such views as central to what he calls “strong sense” critical thinking. In three studies, Haran, Ritov, & Mellers (2013) found that actively open-minded thinking, including “the tendency to weigh new evidence against a favored belief, to spend sufficient time on a problem before giving up, and to consider carefully the opinions of others in forming one’s own”, led study participants to acquire information and thus to make accurate estimations.
• Willingness to suspend judgment : Premature closure on an initial solution will block critical thinking. Thus willingness to think critically requires a willingness to suspend judgment while alternatives are explored (Facione 1990a; Ennis 1991; Halpern 1998).
• Trust in reason : Since distrust in the processes of reasoned inquiry will dissuade one from engaging in it, trust in them is an initiating critical thinking disposition (Facione 1990a, 25; Bailin et al. 1999b: 294; Facione, Facione, & Giancarlo 2001; Paul & Elder 2006). In reaction to an allegedly exclusive emphasis on reason in critical thinking theory and pedagogy, Thayer-Bacon (2000) argues that intuition, imagination, and emotion have important roles to play in an adequate conception of critical thinking that she calls “constructive thinking”. From her point of view, critical thinking requires trust not only in reason but also in intuition, imagination, and emotion.
• Seeking the truth : If one does not care about the truth but is content to stick with one’s initial bias on an issue, then one will not think critically about it. Seeking the truth is thus an initiating critical thinking disposition (Bailin et al. 1999b: 294; Facione, Facione, & Giancarlo 2001). A disposition to seek the truth is implicit in more specific critical thinking dispositions, such as trying to be well-informed, considering seriously points of view other than one’s own, looking for alternatives, suspending judgment when the evidence is insufficient, and adopting a position when the evidence supporting it is sufficient.

Some of the initiating dispositions, such as open-mindedness and willingness to suspend judgment, are also internal critical thinking dispositions, in the sense of mental habits or attitudes that contribute causally to doing a good job of critical thinking once one starts the process. But there are many other internal critical thinking dispositions. Some of them are parasitic on one’s conception of good thinking. For example, it is constitutive of good thinking about an issue to formulate the issue clearly and to maintain focus on it. For this purpose, one needs not only the corresponding ability but also the corresponding disposition. Ennis (1991: 8) describes it as the disposition “to determine and maintain focus on the conclusion or question”, Facione (1990a: 25) as “clarity in stating the question or concern”. Other internal dispositions are motivators to continue or adjust the critical thinking process, such as willingness to persist in a complex task and willingness to abandon nonproductive strategies in an attempt to self-correct (Halpern 1998: 452). For a list of identified internal critical thinking dispositions, see the Supplement on Internal Critical Thinking Dispositions .

Some theorists postulate skills, i.e., acquired abilities, as operative in critical thinking. It is not obvious, however, that a good mental act is the exercise of a generic acquired skill. Inferring an expected time of arrival, as in Transit , has some generic components but also uses non-generic subject-matter knowledge. Bailin et al. (1999a) argue against viewing critical thinking skills as generic and discrete, on the ground that skilled performance at a critical thinking task cannot be separated from knowledge of concepts and from domain-specific principles of good thinking. Talk of skills, they concede, is unproblematic if it means merely that a person with critical thinking skills is capable of intelligent performance.

Despite such scepticism, theorists of critical thinking have listed as general contributors to critical thinking what they variously call abilities (Glaser 1941; Ennis 1962, 1991), skills (Facione 1990a; Halpern 1998) or competencies (Fisher & Scriven 1997). Amalgamating these lists would produce a confusing and chaotic cornucopia of more than 50 possible educational objectives, with only partial overlap among them. It makes sense instead to try to understand the reasons for the multiplicity and diversity, and to make a selection according to one’s own reasons for singling out abilities to be developed in a critical thinking curriculum. Two reasons for diversity among lists of critical thinking abilities are the underlying conception of critical thinking and the envisaged educational level. Appraisal-only conceptions, for example, involve a different suite of abilities than constructive-only conceptions. Some lists, such as those in (Glaser 1941), are put forward as educational objectives for secondary school students, whereas others are proposed as objectives for college students (e.g., Facione 1990a).

The abilities described in the remaining paragraphs of this section emerge from reflection on the general abilities needed to do well the thinking activities identified in section 6 as components of the critical thinking process described in section 5 . The derivation of each collection of abilities is accompanied by citation of sources that list such abilities and of standardized tests that claim to test them.

Observational abilities : Careful and accurate observation sometimes requires specialist expertise and practice, as in the case of observing birds and observing accident scenes. However, there are general abilities of noticing what one’s senses are picking up from one’s environment and of being able to articulate clearly and accurately to oneself and others what one has observed. It helps in exercising them to be able to recognize and take into account factors that make one’s observation less trustworthy, such as prior framing of the situation, inadequate time, deficient senses, poor observation conditions, and the like. It helps as well to be skilled at taking steps to make one’s observation more trustworthy, such as moving closer to get a better look, measuring something three times and taking the average, and checking what one thinks one is observing with someone else who is in a good position to observe it. It also helps to be skilled at recognizing respects in which one’s report of one’s observation involves inference rather than direct observation, so that one can then consider whether the inference is justified. These abilities come into play as well when one thinks about whether and with what degree of confidence to accept an observation report, for example in the study of history or in a criminal investigation or in assessing news reports. Observational abilities show up in some lists of critical thinking abilities (Ennis 1962: 90; Facione 1990a: 16; Ennis 1991: 9). There are items testing a person’s ability to judge the credibility of observation reports in the Cornell Critical Thinking Tests, Levels X and Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005). Norris and King (1983, 1985, 1990a, 1990b) is a test of ability to appraise observation reports.

Emotional abilities : The emotions that drive a critical thinking process are perplexity or puzzlement, a wish to resolve it, and satisfaction at achieving the desired resolution. Children experience these emotions at an early age, without being trained to do so. Education that takes critical thinking as a goal needs only to channel these emotions and to make sure not to stifle them. Collaborative critical thinking benefits from ability to recognize one’s own and others’ emotional commitments and reactions.

Questioning abilities : A critical thinking process needs transformation of an inchoate sense of perplexity into a clear question. Formulating a question well requires not building in questionable assumptions, not prejudging the issue, and using language that in context is unambiguous and precise enough (Ennis 1962: 97; 1991: 9).

Imaginative abilities : Thinking directed at finding the correct causal explanation of a general phenomenon or particular event requires an ability to imagine possible explanations. Thinking about what policy or plan of action to adopt requires generation of options and consideration of possible consequences of each option. Domain knowledge is required for such creative activity, but a general ability to imagine alternatives is helpful and can be nurtured so as to become easier, quicker, more extensive, and deeper (Dewey 1910: 34–39; 1933: 40–47). Facione (1990a) and Halpern (1998) include the ability to imagine alternatives as a critical thinking ability.

Inferential abilities : The ability to draw conclusions from given information, and to recognize with what degree of certainty one’s own or others’ conclusions follow, is universally recognized as a general critical thinking ability. All 11 examples in section 2 of this article include inferences, some from hypotheses or options (as in Transit , Ferryboat and Disorder ), others from something observed (as in Weather and Rash ). None of these inferences is formally valid. Rather, they are licensed by general, sometimes qualified substantive rules of inference (Toulmin 1958) that rest on domain knowledge—that a bus trip takes about the same time in each direction, that the terminal of a wireless telegraph would be located on the highest possible place, that sudden cooling is often followed by rain, that an allergic reaction to a sulfa drug generally shows up soon after one starts taking it. It is a matter of controversy to what extent the specialized ability to deduce conclusions from premisses using formal rules of inference is needed for critical thinking. Dewey (1933) locates logical forms in setting out the products of reflection rather than in the process of reflection. Ennis (1981a), on the other hand, maintains that a liberally-educated person should have the following abilities: to translate natural-language statements into statements using the standard logical operators, to use appropriately the language of necessary and sufficient conditions, to deal with argument forms and arguments containing symbols, to determine whether in virtue of an argument’s form its conclusion follows necessarily from its premisses, to reason with logically complex propositions, and to apply the rules and procedures of deductive logic. Inferential abilities are recognized as critical thinking abilities by Glaser (1941: 6), Facione (1990a: 9), Ennis (1991: 9), Fisher & Scriven (1997: 99, 111), and Halpern (1998: 452). Items testing inferential abilities constitute two of the five subtests of the Watson Glaser Critical Thinking Appraisal (Watson & Glaser 1980a, 1980b, 1994), two of the four sections in the Cornell Critical Thinking Test Level X (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), three of the seven sections in the Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), 11 of the 34 items on Forms A and B of the California Critical Thinking Skills Test (Facione 1990b, 1992), and a high but variable proportion of the 25 selected-response questions in the Collegiate Learning Assessment (Council for Aid to Education 2017).

Experimenting abilities : Knowing how to design and execute an experiment is important not just in scientific research but also in everyday life, as in Rash . Dewey devoted a whole chapter of his How We Think (1910: 145–156; 1933: 190–202) to the superiority of experimentation over observation in advancing knowledge. Experimenting abilities come into play at one remove in appraising reports of scientific studies. Skill in designing and executing experiments includes the acknowledged abilities to appraise evidence (Glaser 1941: 6), to carry out experiments and to apply appropriate statistical inference techniques (Facione 1990a: 9), to judge inductions to an explanatory hypothesis (Ennis 1991: 9), and to recognize the need for an adequately large sample size (Halpern 1998). The Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005) includes four items (out of 52) on experimental design. The Collegiate Learning Assessment (Council for Aid to Education 2017) makes room for appraisal of study design in both its performance task and its selected-response questions.

Consulting abilities : Skill at consulting sources of information comes into play when one seeks information to help resolve a problem, as in Candidate . Ability to find and appraise information includes ability to gather and marshal pertinent information (Glaser 1941: 6), to judge whether a statement made by an alleged authority is acceptable (Ennis 1962: 84), to plan a search for desired information (Facione 1990a: 9), and to judge the credibility of a source (Ennis 1991: 9). Ability to judge the credibility of statements is tested by 24 items (out of 76) in the Cornell Critical Thinking Test Level X (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005) and by four items (out of 52) in the Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005). The College Learning Assessment’s performance task requires evaluation of whether information in documents is credible or unreliable (Council for Aid to Education 2017).

Argument analysis abilities : The ability to identify and analyze arguments contributes to the process of surveying arguments on an issue in order to form one’s own reasoned judgment, as in Candidate . The ability to detect and analyze arguments is recognized as a critical thinking skill by Facione (1990a: 7–8), Ennis (1991: 9) and Halpern (1998). Five items (out of 34) on the California Critical Thinking Skills Test (Facione 1990b, 1992) test skill at argument analysis. The College Learning Assessment (Council for Aid to Education 2017) incorporates argument analysis in its selected-response tests of critical reading and evaluation and of critiquing an argument.

Judging skills and deciding skills : Skill at judging and deciding is skill at recognizing what judgment or decision the available evidence and argument supports, and with what degree of confidence. It is thus a component of the inferential skills already discussed.

Lists and tests of critical thinking abilities often include two more abilities: identifying assumptions and constructing and evaluating definitions.

In addition to dispositions and abilities, critical thinking needs knowledge: of critical thinking concepts, of critical thinking principles, and of the subject-matter of the thinking.

We can derive a short list of concepts whose understanding contributes to critical thinking from the critical thinking abilities described in the preceding section. Observational abilities require an understanding of the difference between observation and inference. Questioning abilities require an understanding of the concepts of ambiguity and vagueness. Inferential abilities require an understanding of the difference between conclusive and defeasible inference (traditionally, between deduction and induction), as well as of the difference between necessary and sufficient conditions. Experimenting abilities require an understanding of the concepts of hypothesis, null hypothesis, assumption and prediction, as well as of the concept of statistical significance and of its difference from importance. They also require an understanding of the difference between an experiment and an observational study, and in particular of the difference between a randomized controlled trial, a prospective correlational study and a retrospective (case-control) study. Argument analysis abilities require an understanding of the concepts of argument, premiss, assumption, conclusion and counter-consideration. Additional critical thinking concepts are proposed by Bailin et al. (1999b: 293), Fisher & Scriven (1997: 105–106), Black (2012), and Blair (2021).

According to Glaser (1941: 25), ability to think critically requires knowledge of the methods of logical inquiry and reasoning. If we review the list of abilities in the preceding section, however, we can see that some of them can be acquired and exercised merely through practice, possibly guided in an educational setting, followed by feedback. Searching intelligently for a causal explanation of some phenomenon or event requires that one consider a full range of possible causal contributors, but it seems more important that one implements this principle in one’s practice than that one is able to articulate it. What is important is “operational knowledge” of the standards and principles of good thinking (Bailin et al. 1999b: 291–293). But the development of such critical thinking abilities as designing an experiment or constructing an operational definition can benefit from learning their underlying theory. Further, explicit knowledge of quirks of human thinking seems useful as a cautionary guide. Human memory is not just fallible about details, as people learn from their own experiences of misremembering, but is so malleable that a detailed, clear and vivid recollection of an event can be a total fabrication (Loftus 2017). People seek or interpret evidence in ways that are partial to their existing beliefs and expectations, often unconscious of their “confirmation bias” (Nickerson 1998). Not only are people subject to this and other cognitive biases (Kahneman 2011), of which they are typically unaware, but it may be counter-productive for one to make oneself aware of them and try consciously to counteract them or to counteract social biases such as racial or sexual stereotypes (Kenyon & Beaulac 2014). It is helpful to be aware of these facts and of the superior effectiveness of blocking the operation of biases—for example, by making an immediate record of one’s observations, refraining from forming a preliminary explanatory hypothesis, blind refereeing, double-blind randomized trials, and blind grading of students’ work. It is also helpful to be aware of the prevalence of “noise” (unwanted unsystematic variability of judgments), of how to detect noise (through a noise audit), and of how to reduce noise: make accuracy the goal, think statistically, break a process of arriving at a judgment into independent tasks, resist premature intuitions, in a group get independent judgments first, favour comparative judgments and scales (Kahneman, Sibony, & Sunstein 2021). It is helpful as well to be aware of the concept of “bounded rationality” in decision-making and of the related distinction between “satisficing” and optimizing (Simon 1956; Gigerenzer 2001).

Critical thinking about an issue requires substantive knowledge of the domain to which the issue belongs. Critical thinking abilities are not a magic elixir that can be applied to any issue whatever by somebody who has no knowledge of the facts relevant to exploring that issue. For example, the student in Bubbles needed to know that gases do not penetrate solid objects like a glass, that air expands when heated, that the volume of an enclosed gas varies directly with its temperature and inversely with its pressure, and that hot objects will spontaneously cool down to the ambient temperature of their surroundings unless kept hot by insulation or a source of heat. Critical thinkers thus need a rich fund of subject-matter knowledge relevant to the variety of situations they encounter. This fact is recognized in the inclusion among critical thinking dispositions of a concern to become and remain generally well informed.

Experimental educational interventions, with control groups, have shown that education can improve critical thinking skills and dispositions, as measured by standardized tests. For information about these tests, see the Supplement on Assessment .

What educational methods are most effective at developing the dispositions, abilities and knowledge of a critical thinker? In a comprehensive meta-analysis of experimental and quasi-experimental studies of strategies for teaching students to think critically, Abrami et al. (2015) found that dialogue, anchored instruction, and mentoring each increased the effectiveness of the educational intervention, and that they were most effective when combined. They also found that in these studies a combination of separate instruction in critical thinking with subject-matter instruction in which students are encouraged to think critically was more effective than either by itself. However, the difference was not statistically significant; that is, it might have arisen by chance.

Most of these studies lack the longitudinal follow-up required to determine whether the observed differential improvements in critical thinking abilities or dispositions continue over time, for example until high school or college graduation. For details on studies of methods of developing critical thinking skills and dispositions, see the Supplement on Educational Methods .

12. Controversies

Scholars have denied the generalizability of critical thinking abilities across subject domains, have alleged bias in critical thinking theory and pedagogy, and have investigated the relationship of critical thinking to other kinds of thinking.

McPeck (1981) attacked the thinking skills movement of the 1970s, including the critical thinking movement. He argued that there are no general thinking skills, since thinking is always thinking about some subject-matter. It is futile, he claimed, for schools and colleges to teach thinking as if it were a separate subject. Rather, teachers should lead their pupils to become autonomous thinkers by teaching school subjects in a way that brings out their cognitive structure and that encourages and rewards discussion and argument. As some of his critics (e.g., Paul 1985; Siegel 1985) pointed out, McPeck’s central argument needs elaboration, since it has obvious counter-examples in writing and speaking, for which (up to a certain level of complexity) there are teachable general abilities even though they are always about some subject-matter. To make his argument convincing, McPeck needs to explain how thinking differs from writing and speaking in a way that does not permit useful abstraction of its components from the subject-matters with which it deals. He has not done so. Nevertheless, his position that the dispositions and abilities of a critical thinker are best developed in the context of subject-matter instruction is shared by many theorists of critical thinking, including Dewey (1910, 1933), Glaser (1941), Passmore (1980), Weinstein (1990), Bailin et al. (1999b), and Willingham (2019).

McPeck’s challenge prompted reflection on the extent to which critical thinking is subject-specific. McPeck argued for a strong subject-specificity thesis, according to which it is a conceptual truth that all critical thinking abilities are specific to a subject. (He did not however extend his subject-specificity thesis to critical thinking dispositions. In particular, he took the disposition to suspend judgment in situations of cognitive dissonance to be a general disposition.) Conceptual subject-specificity is subject to obvious counter-examples, such as the general ability to recognize confusion of necessary and sufficient conditions. A more modest thesis, also endorsed by McPeck, is epistemological subject-specificity, according to which the norms of good thinking vary from one field to another. Epistemological subject-specificity clearly holds to a certain extent; for example, the principles in accordance with which one solves a differential equation are quite different from the principles in accordance with which one determines whether a painting is a genuine Picasso. But the thesis suffers, as Ennis (1989) points out, from vagueness of the concept of a field or subject and from the obvious existence of inter-field principles, however broadly the concept of a field is construed. For example, the principles of hypothetico-deductive reasoning hold for all the varied fields in which such reasoning occurs. A third kind of subject-specificity is empirical subject-specificity, according to which as a matter of empirically observable fact a person with the abilities and dispositions of a critical thinker in one area of investigation will not necessarily have them in another area of investigation.

The thesis of empirical subject-specificity raises the general problem of transfer. If critical thinking abilities and dispositions have to be developed independently in each school subject, how are they of any use in dealing with the problems of everyday life and the political and social issues of contemporary society, most of which do not fit into the framework of a traditional school subject? Proponents of empirical subject-specificity tend to argue that transfer is more likely to occur if there is critical thinking instruction in a variety of domains, with explicit attention to dispositions and abilities that cut across domains. But evidence for this claim is scanty. There is a need for well-designed empirical studies that investigate the conditions that make transfer more likely.

It is common ground in debates about the generality or subject-specificity of critical thinking dispositions and abilities that critical thinking about any topic requires background knowledge about the topic. For example, the most sophisticated understanding of the principles of hypothetico-deductive reasoning is of no help unless accompanied by some knowledge of what might be plausible explanations of some phenomenon under investigation.

Critics have objected to bias in the theory, pedagogy and practice of critical thinking. Commentators (e.g., Alston 1995; Ennis 1998) have noted that anyone who takes a position has a bias in the neutral sense of being inclined in one direction rather than others. The critics, however, are objecting to bias in the pejorative sense of an unjustified favoring of certain ways of knowing over others, frequently alleging that the unjustly favoured ways are those of a dominant sex or culture (Bailin 1995). These ways favour:

• reinforcement of egocentric and sociocentric biases over dialectical engagement with opposing world-views (Paul 1981, 1984; Warren 1998)
• distancing from the object of inquiry over closeness to it (Martin 1992; Thayer-Bacon 1992)
• indifference to the situation of others over care for them (Martin 1992)
• orientation to thought over orientation to action (Martin 1992)
• being reasonable over caring to understand people’s ideas (Thayer-Bacon 1993)
• being neutral and objective over being embodied and situated (Thayer-Bacon 1995a)
• doubting over believing (Thayer-Bacon 1995b)
• reason over emotion, imagination and intuition (Thayer-Bacon 2000)
• solitary thinking over collaborative thinking (Thayer-Bacon 2000)
• written and spoken assignments over other forms of expression (Alston 2001)
• attention to written and spoken communications over attention to human problems (Alston 2001)
• winning debates in the public sphere over making and understanding meaning (Alston 2001)

A common thread in this smorgasbord of accusations is dissatisfaction with focusing on the logical analysis and evaluation of reasoning and arguments. While these authors acknowledge that such analysis and evaluation is part of critical thinking and should be part of its conceptualization and pedagogy, they insist that it is only a part. Paul (1981), for example, bemoans the tendency of atomistic teaching of methods of analyzing and evaluating arguments to turn students into more able sophists, adept at finding fault with positions and arguments with which they disagree but even more entrenched in the egocentric and sociocentric biases with which they began. Martin (1992) and Thayer-Bacon (1992) cite with approval the self-reported intimacy with their subject-matter of leading researchers in biology and medicine, an intimacy that conflicts with the distancing allegedly recommended in standard conceptions and pedagogy of critical thinking. Thayer-Bacon (2000) contrasts the embodied and socially embedded learning of her elementary school students in a Montessori school, who used their imagination, intuition and emotions as well as their reason, with conceptions of critical thinking as

thinking that is used to critique arguments, offer justifications, and make judgments about what are the good reasons, or the right answers. (Thayer-Bacon 2000: 127–128)

Alston (2001) reports that her students in a women’s studies class were able to see the flaws in the Cinderella myth that pervades much romantic fiction but in their own romantic relationships still acted as if all failures were the woman’s fault and still accepted the notions of love at first sight and living happily ever after. Students, she writes, should

be able to connect their intellectual critique to a more affective, somatic, and ethical account of making risky choices that have sexist, racist, classist, familial, sexual, or other consequences for themselves and those both near and far… critical thinking that reads arguments, texts, or practices merely on the surface without connections to feeling/desiring/doing or action lacks an ethical depth that should infuse the difference between mere cognitive activity and something we want to call critical thinking. (Alston 2001: 34)

Some critics portray such biases as unfair to women. Thayer-Bacon (1992), for example, has charged modern critical thinking theory with being sexist, on the ground that it separates the self from the object and causes one to lose touch with one’s inner voice, and thus stigmatizes women, who (she asserts) link self to object and listen to their inner voice. Her charge does not imply that women as a group are on average less able than men to analyze and evaluate arguments. Facione (1990c) found no difference by sex in performance on his California Critical Thinking Skills Test. Kuhn (1991: 280–281) found no difference by sex in either the disposition or the competence to engage in argumentative thinking.

The critics propose a variety of remedies for the biases that they allege. In general, they do not propose to eliminate or downplay critical thinking as an educational goal. Rather, they propose to conceptualize critical thinking differently and to change its pedagogy accordingly. Their pedagogical proposals arise logically from their objections. They can be summarized as follows:

• Focus on argument networks with dialectical exchanges reflecting contesting points of view rather than on atomic arguments, so as to develop “strong sense” critical thinking that transcends egocentric and sociocentric biases (Paul 1981, 1984).
• Foster closeness to the subject-matter and feeling connected to others in order to inform a humane democracy (Martin 1992).
• Develop “constructive thinking” as a social activity in a community of physically embodied and socially embedded inquirers with personal voices who value not only reason but also imagination, intuition and emotion (Thayer-Bacon 2000).
• In developing critical thinking in school subjects, treat as important neither skills nor dispositions but opening worlds of meaning (Alston 2001).
• Attend to the development of critical thinking dispositions as well as skills, and adopt the “critical pedagogy” practised and advocated by Freire (1968 [1970]) and hooks (1994) (Dalgleish, Girard, & Davies 2017).

A common thread in these proposals is treatment of critical thinking as a social, interactive, personally engaged activity like that of a quilting bee or a barn-raising (Thayer-Bacon 2000) rather than as an individual, solitary, distanced activity symbolized by Rodin’s The Thinker . One can get a vivid description of education with the former type of goal from the writings of bell hooks (1994, 2010). Critical thinking for her is open-minded dialectical exchange across opposing standpoints and from multiple perspectives, a conception similar to Paul’s “strong sense” critical thinking (Paul 1981). She abandons the structure of domination in the traditional classroom. In an introductory course on black women writers, for example, she assigns students to write an autobiographical paragraph about an early racial memory, then to read it aloud as the others listen, thus affirming the uniqueness and value of each voice and creating a communal awareness of the diversity of the group’s experiences (hooks 1994: 84). Her “engaged pedagogy” is thus similar to the “freedom under guidance” implemented in John Dewey’s Laboratory School of Chicago in the late 1890s and early 1900s. It incorporates the dialogue, anchored instruction, and mentoring that Abrami (2015) found to be most effective in improving critical thinking skills and dispositions.

What is the relationship of critical thinking to problem solving, decision-making, higher-order thinking, creative thinking, and other recognized types of thinking? One’s answer to this question obviously depends on how one defines the terms used in the question. If critical thinking is conceived broadly to cover any careful thinking about any topic for any purpose, then problem solving and decision making will be kinds of critical thinking, if they are done carefully. Historically, ‘critical thinking’ and ‘problem solving’ were two names for the same thing. If critical thinking is conceived more narrowly as consisting solely of appraisal of intellectual products, then it will be disjoint with problem solving and decision making, which are constructive.

Bloom’s taxonomy of educational objectives used the phrase “intellectual abilities and skills” for what had been labeled “critical thinking” by some, “reflective thinking” by Dewey and others, and “problem solving” by still others (Bloom et al. 1956: 38). Thus, the so-called “higher-order thinking skills” at the taxonomy’s top levels of analysis, synthesis and evaluation are just critical thinking skills, although they do not come with general criteria for their assessment (Ennis 1981b). The revised version of Bloom’s taxonomy (Anderson et al. 2001) likewise treats critical thinking as cutting across those types of cognitive process that involve more than remembering (Anderson et al. 2001: 269–270). For details, see the Supplement on History .

As to creative thinking, it overlaps with critical thinking (Bailin 1987, 1988). Thinking about the explanation of some phenomenon or event, as in Ferryboat , requires creative imagination in constructing plausible explanatory hypotheses. Likewise, thinking about a policy question, as in Candidate , requires creativity in coming up with options. Conversely, creativity in any field needs to be balanced by critical appraisal of the draft painting or novel or mathematical theory.

• Abrami, Philip C., Robert M. Bernard, Eugene Borokhovski, David I. Waddington, C. Anne Wade, and Tonje Person, 2015, “Strategies for Teaching Students to Think Critically: A Meta-analysis”, Review of Educational Research , 85(2): 275–314. doi:10.3102/0034654314551063
• Aikin, Wilford M., 1942, The Story of the Eight-year Study, with Conclusions and Recommendations , Volume I of Adventure in American Education , New York and London: Harper & Brothers. [ Aikin 1942 available online ]
• Alston, Kal, 1995, “Begging the Question: Is Critical Thinking Biased?”, Educational Theory , 45(2): 225–233. doi:10.1111/j.1741-5446.1995.00225.x
• –––, 2001, “Re/Thinking Critical Thinking: The Seductions of Everyday Life”, Studies in Philosophy and Education , 20(1): 27–40. doi:10.1023/A:1005247128053
• American Educational Research Association, 2014, Standards for Educational and Psychological Testing / American Educational Research Association, American Psychological Association, National Council on Measurement in Education , Washington, DC: American Educational Research Association.
• Anderson, Lorin W., David R. Krathwohl, Peter W. Airiasian, Kathleen A. Cruikshank, Richard E. Mayer, Paul R. Pintrich, James Raths, and Merlin C. Wittrock, 2001, A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives , New York: Longman, complete edition.
• Bailin, Sharon, 1987, “Critical and Creative Thinking”, Informal Logic , 9(1): 23–30. [ Bailin 1987 available online ]
• –––, 1988, Achieving Extraordinary Ends: An Essay on Creativity , Dordrecht: Kluwer. doi:10.1007/978-94-009-2780-3
• –––, 1995, “Is Critical Thinking Biased? Clarifications and Implications”, Educational Theory , 45(2): 191–197. doi:10.1111/j.1741-5446.1995.00191.x
• Bailin, Sharon and Mark Battersby, 2009, “Inquiry: A Dialectical Approach to Teaching Critical Thinking”, in Juho Ritola (ed.), Argument Cultures: Proceedings of OSSA 09 , CD-ROM (pp. 1–10), Windsor, ON: OSSA. [ Bailin & Battersby 2009 available online ]
• –––, 2016a, “Fostering the Virtues of Inquiry”, Topoi , 35(2): 367–374. doi:10.1007/s11245-015-9307-6
• –––, 2016b, Reason in the Balance: An Inquiry Approach to Critical Thinking , Indianapolis: Hackett, 2nd edition.
• –––, 2021, “Inquiry: Teaching for Reasoned Judgment”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 31–46. doi: 10.1163/9789004444591_003
• Bailin, Sharon, Roland Case, Jerrold R. Coombs, and Leroi B. Daniels, 1999a, “Common Misconceptions of Critical Thinking”, Journal of Curriculum Studies , 31(3): 269–283. doi:10.1080/002202799183124
• –––, 1999b, “Conceptualizing Critical Thinking”, Journal of Curriculum Studies , 31(3): 285–302. doi:10.1080/002202799183133
• Blair, J. Anthony, 2021, Studies in Critical Thinking , Windsor, ON: Windsor Studies in Argumentation, 2nd edition. [Available online at https://windsor.scholarsportal.info/omp/index.php/wsia/catalog/book/106]
• Berman, Alan M., Seth J. Schwartz, William M. Kurtines, and Steven L. Berman, 2001, “The Process of Exploration in Identity Formation: The Role of Style and Competence”, Journal of Adolescence , 24(4): 513–528. doi:10.1006/jado.2001.0386
• Black, Beth (ed.), 2012, An A to Z of Critical Thinking , London: Continuum International Publishing Group.
• Bloom, Benjamin Samuel, Max D. Engelhart, Edward J. Furst, Walter H. Hill, and David R. Krathwohl, 1956, Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , New York: David McKay.
• Boardman, Frank, Nancy M. Cavender, and Howard Kahane, 2018, Logic and Contemporary Rhetoric: The Use of Reason in Everyday Life , Boston: Cengage, 13th edition.
• Browne, M. Neil and Stuart M. Keeley, 2018, Asking the Right Questions: A Guide to Critical Thinking , Hoboken, NJ: Pearson, 12th edition.
• Center for Assessment & Improvement of Learning, 2017, Critical Thinking Assessment Test , Cookeville, TN: Tennessee Technological University.
• Cleghorn, Paul. 2021. “Critical Thinking in the Elementary School: Practical Guidance for Building a Culture of Thinking”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessmen t, Leiden: Brill, pp. 150–167. doi: 10.1163/9789004444591_010
• Cohen, Jacob, 1988, Statistical Power Analysis for the Behavioral Sciences , Hillsdale, NJ: Lawrence Erlbaum Associates, 2nd edition.
• College Board, 1983, Academic Preparation for College. What Students Need to Know and Be Able to Do , New York: College Entrance Examination Board, ERIC document ED232517.
• Commission on the Relation of School and College of the Progressive Education Association, 1943, Thirty Schools Tell Their Story , Volume V of Adventure in American Education , New York and London: Harper & Brothers.
• Council for Aid to Education, 2017, CLA+ Student Guide . Available at http://cae.org/images/uploads/pdf/CLA_Student_Guide_Institution.pdf ; last accessed 2022 07 16.
• Dalgleish, Adam, Patrick Girard, and Maree Davies, 2017, “Critical Thinking, Bias and Feminist Philosophy: Building a Better Framework through Collaboration”, Informal Logic , 37(4): 351–369. [ Dalgleish et al. available online ]
• Dewey, John, 1910, How We Think , Boston: D.C. Heath. [ Dewey 1910 available online ]
• –––, 1916, Democracy and Education: An Introduction to the Philosophy of Education , New York: Macmillan.
• –––, 1933, How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process , Lexington, MA: D.C. Heath.
• –––, 1936, “The Theory of the Chicago Experiment”, Appendix II of Mayhew & Edwards 1936: 463–477.
• –––, 1938, Logic: The Theory of Inquiry , New York: Henry Holt and Company.
• Dominguez, Caroline (coord.), 2018a, A European Collection of the Critical Thinking Skills and Dispositions Needed in Different Professional Fields for the 21st Century , Vila Real, Portugal: UTAD. Available at http://bit.ly/CRITHINKEDUO1 ; last accessed 2022 07 16.
• ––– (coord.), 2018b, A European Review on Critical Thinking Educational Practices in Higher Education Institutions , Vila Real: UTAD. Available at http://bit.ly/CRITHINKEDUO2 ; last accessed 2022 07 16.
• ––– (coord.), 2018c, The CRITHINKEDU European Course on Critical Thinking Education for University Teachers: From Conception to Delivery , Vila Real: UTAD. Available at http:/bit.ly/CRITHINKEDU03; last accessed 2022 07 16.
• Dominguez Caroline and Rita Payan-Carreira (eds.), 2019, Promoting Critical Thinking in European Higher Education Institutions: Towards an Educational Protocol , Vila Real: UTAD. Available at http:/bit.ly/CRITHINKEDU04; last accessed 2022 07 16.
• Ennis, Robert H., 1958, “An Appraisal of the Watson-Glaser Critical Thinking Appraisal”, The Journal of Educational Research , 52(4): 155–158. doi:10.1080/00220671.1958.10882558
• –––, 1962, “A Concept of Critical Thinking: A Proposed Basis for Research on the Teaching and Evaluation of Critical Thinking Ability”, Harvard Educational Review , 32(1): 81–111.
• –––, 1981a, “A Conception of Deductive Logical Competence”, Teaching Philosophy , 4(3/4): 337–385. doi:10.5840/teachphil198143/429
• –––, 1981b, “Eight Fallacies in Bloom’s Taxonomy”, in C. J. B. Macmillan (ed.), Philosophy of Education 1980: Proceedings of the Thirty-seventh Annual Meeting of the Philosophy of Education Society , Bloomington, IL: Philosophy of Education Society, pp. 269–273.
• –––, 1984, “Problems in Testing Informal Logic, Critical Thinking, Reasoning Ability”, Informal Logic , 6(1): 3–9. [ Ennis 1984 available online ]
• –––, 1987, “A Taxonomy of Critical Thinking Dispositions and Abilities”, in Joan Boykoff Baron and Robert J. Sternberg (eds.), Teaching Thinking Skills: Theory and Practice , New York: W. H. Freeman, pp. 9–26.
• –––, 1989, “Critical Thinking and Subject Specificity: Clarification and Needed Research”, Educational Researcher , 18(3): 4–10. doi:10.3102/0013189X018003004
• –––, 1991, “Critical Thinking: A Streamlined Conception”, Teaching Philosophy , 14(1): 5–24. doi:10.5840/teachphil19911412
• –––, 1996, “Critical Thinking Dispositions: Their Nature and Assessability”, Informal Logic , 18(2–3): 165–182. [ Ennis 1996 available online ]
• –––, 1998, “Is Critical Thinking Culturally Biased?”, Teaching Philosophy , 21(1): 15–33. doi:10.5840/teachphil19982113
• –––, 2011, “Critical Thinking: Reflection and Perspective Part I”, Inquiry: Critical Thinking across the Disciplines , 26(1): 4–18. doi:10.5840/inquiryctnews20112613
• –––, 2013, “Critical Thinking across the Curriculum: The Wisdom CTAC Program”, Inquiry: Critical Thinking across the Disciplines , 28(2): 25–45. doi:10.5840/inquiryct20132828
• –––, 2016, “Definition: A Three-Dimensional Analysis with Bearing on Key Concepts”, in Patrick Bondy and Laura Benacquista (eds.), Argumentation, Objectivity, and Bias: Proceedings of the 11th International Conference of the Ontario Society for the Study of Argumentation (OSSA), 18–21 May 2016 , Windsor, ON: OSSA, pp. 1–19. Available at http://scholar.uwindsor.ca/ossaarchive/OSSA11/papersandcommentaries/105 ; last accessed 2022 07 16.
• –––, 2018, “Critical Thinking Across the Curriculum: A Vision”, Topoi , 37(1): 165–184. doi:10.1007/s11245-016-9401-4
• Ennis, Robert H., and Jason Millman, 1971, Manual for Cornell Critical Thinking Test, Level X, and Cornell Critical Thinking Test, Level Z , Urbana, IL: Critical Thinking Project, University of Illinois.
• Ennis, Robert H., Jason Millman, and Thomas Norbert Tomko, 1985, Cornell Critical Thinking Tests Level X & Level Z: Manual , Pacific Grove, CA: Midwest Publication, 3rd edition.
• –––, 2005, Cornell Critical Thinking Tests Level X & Level Z: Manual , Seaside, CA: Critical Thinking Company, 5th edition.
• Ennis, Robert H. and Eric Weir, 1985, The Ennis-Weir Critical Thinking Essay Test: Test, Manual, Criteria, Scoring Sheet: An Instrument for Teaching and Testing , Pacific Grove, CA: Midwest Publications.
• Facione, Peter A., 1990a, Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction , Research Findings and Recommendations Prepared for the Committee on Pre-College Philosophy of the American Philosophical Association, ERIC Document ED315423.
• –––, 1990b, California Critical Thinking Skills Test, CCTST – Form A , Millbrae, CA: The California Academic Press.
• –––, 1990c, The California Critical Thinking Skills Test--College Level. Technical Report #3. Gender, Ethnicity, Major, CT Self-Esteem, and the CCTST , ERIC Document ED326584.
• –––, 1992, California Critical Thinking Skills Test: CCTST – Form B, Millbrae, CA: The California Academic Press.
• –––, 2000, “The Disposition Toward Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill”, Informal Logic , 20(1): 61–84. [ Facione 2000 available online ]
• Facione, Peter A. and Noreen C. Facione, 1992, CCTDI: A Disposition Inventory , Millbrae, CA: The California Academic Press.
• Facione, Peter A., Noreen C. Facione, and Carol Ann F. Giancarlo, 2001, California Critical Thinking Disposition Inventory: CCTDI: Inventory Manual , Millbrae, CA: The California Academic Press.
• Facione, Peter A., Carol A. Sánchez, and Noreen C. Facione, 1994, Are College Students Disposed to Think? , Millbrae, CA: The California Academic Press. ERIC Document ED368311.
• Fisher, Alec, and Michael Scriven, 1997, Critical Thinking: Its Definition and Assessment , Norwich: Centre for Research in Critical Thinking, University of East Anglia.
• Freire, Paulo, 1968 [1970], Pedagogia do Oprimido . Translated as Pedagogy of the Oppressed , Myra Bergman Ramos (trans.), New York: Continuum, 1970.
• Gigerenzer, Gerd, 2001, “The Adaptive Toolbox”, in Gerd Gigerenzer and Reinhard Selten (eds.), Bounded Rationality: The Adaptive Toolbox , Cambridge, MA: MIT Press, pp. 37–50.
• Glaser, Edward Maynard, 1941, An Experiment in the Development of Critical Thinking , New York: Bureau of Publications, Teachers College, Columbia University.
• Groarke, Leo A. and Christopher W. Tindale, 2012, Good Reasoning Matters! A Constructive Approach to Critical Thinking , Don Mills, ON: Oxford University Press, 5th edition.
• Halpern, Diane F., 1998, “Teaching Critical Thinking for Transfer Across Domains: Disposition, Skills, Structure Training, and Metacognitive Monitoring”, American Psychologist , 53(4): 449–455. doi:10.1037/0003-066X.53.4.449
• –––, 2016, Manual: Halpern Critical Thinking Assessment , Mödling, Austria: Schuhfried. Available at https://pdfcoffee.com/hcta-test-manual-pdf-free.html; last accessed 2022 07 16.
• Hamby, Benjamin, 2014, The Virtues of Critical Thinkers , Doctoral dissertation, Philosophy, McMaster University. [ Hamby 2014 available online ]
• –––, 2015, “Willingness to Inquire: The Cardinal Critical Thinking Virtue”, in Martin Davies and Ronald Barnett (eds.), The Palgrave Handbook of Critical Thinking in Higher Education , New York: Palgrave Macmillan, pp. 77–87.
• Haran, Uriel, Ilana Ritov, and Barbara A. Mellers, 2013, “The Role of Actively Open-minded Thinking in Information Acquisition, Accuracy, and Calibration”, Judgment and Decision Making , 8(3): 188–201.
• Hatcher, Donald and Kevin Possin, 2021, “Commentary: Thinking Critically about Critical Thinking Assessment”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 298–322. doi: 10.1163/9789004444591_017
• Haynes, Ada, Elizabeth Lisic, Kevin Harris, Katie Leming, Kyle Shanks, and Barry Stein, 2015, “Using the Critical Thinking Assessment Test (CAT) as a Model for Designing Within-Course Assessments: Changing How Faculty Assess Student Learning”, Inquiry: Critical Thinking Across the Disciplines , 30(3): 38–48. doi:10.5840/inquiryct201530316
• Haynes, Ada and Barry Stein, 2021, “Observations from a Long-Term Effort to Assess and Improve Critical Thinking”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 231–254. doi: 10.1163/9789004444591_014
• Hiner, Amanda L. 2021. “Equipping Students for Success in College and Beyond: Placing Critical Thinking Instruction at the Heart of a General Education Program”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 188–208. doi: 10.1163/9789004444591_012
• Hitchcock, David, 2017, “Critical Thinking as an Educational Ideal”, in his On Reasoning and Argument: Essays in Informal Logic and on Critical Thinking , Dordrecht: Springer, pp. 477–497. doi:10.1007/978-3-319-53562-3_30
• –––, 2021, “Seven Philosophical Implications of Critical Thinking: Themes, Variations, Implications”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 9–30. doi: 10.1163/9789004444591_002
• hooks, bell, 1994, Teaching to Transgress: Education as the Practice of Freedom , New York and London: Routledge.
• –––, 2010, Teaching Critical Thinking: Practical Wisdom , New York and London: Routledge.
• Johnson, Ralph H., 1992, “The Problem of Defining Critical Thinking”, in Stephen P, Norris (ed.), The Generalizability of Critical Thinking , New York: Teachers College Press, pp. 38–53.
• Kahane, Howard, 1971, Logic and Contemporary Rhetoric: The Use of Reason in Everyday Life , Belmont, CA: Wadsworth.
• Kahneman, Daniel, 2011, Thinking, Fast and Slow , New York: Farrar, Straus and Giroux.
• Kahneman, Daniel, Olivier Sibony, & Cass R. Sunstein, 2021, Noise: A Flaw in Human Judgment , New York: Little, Brown Spark.
• Kenyon, Tim, and Guillaume Beaulac, 2014, “Critical Thinking Education and Debasing”, Informal Logic , 34(4): 341–363. [ Kenyon & Beaulac 2014 available online ]
• Krathwohl, David R., Benjamin S. Bloom, and Bertram B. Masia, 1964, Taxonomy of Educational Objectives, Handbook II: Affective Domain , New York: David McKay.
• Kuhn, Deanna, 1991, The Skills of Argument , New York: Cambridge University Press. doi:10.1017/CBO9780511571350
• –––, 2019, “Critical Thinking as Discourse”, Human Development, 62 (3): 146–164. doi:10.1159/000500171
• Lipman, Matthew, 1987, “Critical Thinking–What Can It Be?”, Analytic Teaching , 8(1): 5–12. [ Lipman 1987 available online ]
• –––, 2003, Thinking in Education , Cambridge: Cambridge University Press, 2nd edition.
• Loftus, Elizabeth F., 2017, “Eavesdropping on Memory”, Annual Review of Psychology , 68: 1–18. doi:10.1146/annurev-psych-010416-044138
• Makaiau, Amber Strong, 2021, “The Good Thinker’s Tool Kit: How to Engage Critical Thinking and Reasoning in Secondary Education”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 168–187. doi: 10.1163/9789004444591_011
• Martin, Jane Roland, 1992, “Critical Thinking for a Humane World”, in Stephen P. Norris (ed.), The Generalizability of Critical Thinking , New York: Teachers College Press, pp. 163–180.
• Mayhew, Katherine Camp, and Anna Camp Edwards, 1936, The Dewey School: The Laboratory School of the University of Chicago, 1896–1903 , New York: Appleton-Century. [ Mayhew & Edwards 1936 available online ]
• McPeck, John E., 1981, Critical Thinking and Education , New York: St. Martin’s Press.
• Moore, Brooke Noel and Richard Parker, 2020, Critical Thinking , New York: McGraw-Hill, 13th edition.
• Nickerson, Raymond S., 1998, “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises”, Review of General Psychology , 2(2): 175–220. doi:10.1037/1089-2680.2.2.175
• Nieto, Ana Maria, and Jorge Valenzuela, 2012, “A Study of the Internal Structure of Critical Thinking Dispositions”, Inquiry: Critical Thinking across the Disciplines , 27(1): 31–38. doi:10.5840/inquiryct20122713
• Norris, Stephen P., 1985, “Controlling for Background Beliefs When Developing Multiple-choice Critical Thinking Tests”, Educational Measurement: Issues and Practice , 7(3): 5–11. doi:10.1111/j.1745-3992.1988.tb00437.x
• Norris, Stephen P. and Robert H. Ennis, 1989, Evaluating Critical Thinking (The Practitioners’ Guide to Teaching Thinking Series), Pacific Grove, CA: Midwest Publications.
• Norris, Stephen P. and Ruth Elizabeth King, 1983, Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland.
• –––, 1984, The Design of a Critical Thinking Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland. ERIC Document ED260083.
• –––, 1985, Test on Appraising Observations: Manual , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland.
• –––, 1990a, Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland, 2nd edition.
• –––, 1990b, Test on Appraising Observations: Manual , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland, 2nd edition.
• OCR [Oxford, Cambridge and RSA Examinations], 2011, AS/A Level GCE: Critical Thinking – H052, H452 , Cambridge: OCR. Past papers available at https://pastpapers.co/ocr/?dir=A-Level/Critical-Thinking-H052-H452; last accessed 2022 07 16.
• Ontario Ministry of Education, 2013, The Ontario Curriculum Grades 9 to 12: Social Sciences and Humanities . Available at http://www.edu.gov.on.ca/eng/curriculum/secondary/ssciences9to122013.pdf ; last accessed 2022 07 16.
• Passmore, John Arthur, 1980, The Philosophy of Teaching , London: Duckworth.
• Paul, Richard W., 1981, “Teaching Critical Thinking in the ‘Strong’ Sense: A Focus on Self-Deception, World Views, and a Dialectical Mode of Analysis”, Informal Logic , 4(2): 2–7. [ Paul 1981 available online ]
• –––, 1984, “Critical Thinking: Fundamental to Education for a Free Society”, Educational Leadership , 42(1): 4–14.
• –––, 1985, “McPeck’s Mistakes”, Informal Logic , 7(1): 35–43. [ Paul 1985 available online ]
• Paul, Richard W. and Linda Elder, 2006, The Miniature Guide to Critical Thinking: Concepts and Tools , Dillon Beach, CA: Foundation for Critical Thinking, 4th edition.
• Payette, Patricia, and Edna Ross, 2016, “Making a Campus-Wide Commitment to Critical Thinking: Insights and Promising Practices Utilizing the Paul-Elder Approach at the University of Louisville”, Inquiry: Critical Thinking Across the Disciplines , 31(1): 98–110. doi:10.5840/inquiryct20163118
• Possin, Kevin, 2008, “A Field Guide to Critical-Thinking Assessment”, Teaching Philosophy , 31(3): 201–228. doi:10.5840/teachphil200831324
• –––, 2013a, “Some Problems with the Halpern Critical Thinking Assessment (HCTA) Test”, Inquiry: Critical Thinking across the Disciplines , 28(3): 4–12. doi:10.5840/inquiryct201328313
• –––, 2013b, “A Serious Flaw in the Collegiate Learning Assessment (CLA) Test”, Informal Logic , 33(3): 390–405. [ Possin 2013b available online ]
• –––, 2013c, “A Fatal Flaw in the Collegiate Learning Assessment Test”, Assessment Update , 25 (1): 8–12.
• –––, 2014, “Critique of the Watson-Glaser Critical Thinking Appraisal Test: The More You Know, the Lower Your Score”, Informal Logic , 34(4): 393–416. [ Possin 2014 available online ]
• –––, 2020, “CAT Scan: A Critical Review of the Critical-Thinking Assessment Test”, Informal Logic , 40 (3): 489–508. [Available online at https://informallogic.ca/index.php/informal_logic/article/view/6243]
• Rawls, John, 1971, A Theory of Justice , Cambridge, MA: Harvard University Press.
• Rear, David, 2019, “One Size Fits All? The Limitations of Standardised Assessment in Critical Thinking”, Assessment & Evaluation in Higher Education , 44(5): 664–675. doi: 10.1080/02602938.2018.1526255
• Rousseau, Jean-Jacques, 1762, Émile , Amsterdam: Jean Néaulme.
• Scheffler, Israel, 1960, The Language of Education , Springfield, IL: Charles C. Thomas.
• Scriven, Michael, and Richard W. Paul, 1987, Defining Critical Thinking , Draft statement written for the National Council for Excellence in Critical Thinking Instruction. Available at http://www.criticalthinking.org/pages/defining-critical-thinking/766 ; last accessed 2022 07 16.
• Sheffield, Clarence Burton Jr., 2018, “Promoting Critical Thinking in Higher Education: My Experiences as the Inaugural Eugene H. Fram Chair in Applied Critical Thinking at Rochester Institute of Technology”, Topoi , 37(1): 155–163. doi:10.1007/s11245-016-9392-1
• Siegel, Harvey, 1985, “McPeck, Informal Logic and the Nature of Critical Thinking”, in David Nyberg (ed.), Philosophy of Education 1985: Proceedings of the Forty-First Annual Meeting of the Philosophy of Education Society , Normal, IL: Philosophy of Education Society, pp. 61–72.
• –––, 1988, Educating Reason: Rationality, Critical Thinking, and Education , New York: Routledge.
• –––, 1999, “What (Good) Are Thinking Dispositions?”, Educational Theory , 49(2): 207–221. doi:10.1111/j.1741-5446.1999.00207.x
• Simon, Herbert A., 1956, “Rational Choice and the Structure of the Environment”, Psychological Review , 63(2): 129–138. doi: 10.1037/h0042769
• Simpson, Elizabeth, 1966–67, “The Classification of Educational Objectives: Psychomotor Domain”, Illinois Teacher of Home Economics , 10(4): 110–144, ERIC document ED0103613. [ Simpson 1966–67 available online ]
• Skolverket, 2018, Curriculum for the Compulsory School, Preschool Class and School-age Educare , Stockholm: Skolverket, revised 2018. Available at https://www.skolverket.se/download/18.31c292d516e7445866a218f/1576654682907/pdf3984.pdf; last accessed 2022 07 15.
• Smith, B. Othanel, 1953, “The Improvement of Critical Thinking”, Progressive Education , 30(5): 129–134.
• Smith, Eugene Randolph, Ralph Winfred Tyler, and the Evaluation Staff, 1942, Appraising and Recording Student Progress , Volume III of Adventure in American Education , New York and London: Harper & Brothers.
• Splitter, Laurance J., 1987, “Educational Reform through Philosophy for Children”, Thinking: The Journal of Philosophy for Children , 7(2): 32–39. doi:10.5840/thinking1987729
• Stanovich Keith E., and Paula J. Stanovich, 2010, “A Framework for Critical Thinking, Rational Thinking, and Intelligence”, in David D. Preiss and Robert J. Sternberg (eds), Innovations in Educational Psychology: Perspectives on Learning, Teaching and Human Development , New York: Springer Publishing, pp 195–237.
• Stanovich Keith E., Richard F. West, and Maggie E. Toplak, 2011, “Intelligence and Rationality”, in Robert J. Sternberg and Scott Barry Kaufman (eds.), Cambridge Handbook of Intelligence , Cambridge: Cambridge University Press, 3rd edition, pp. 784–826. doi:10.1017/CBO9780511977244.040
• Tankersley, Karen, 2005, Literacy Strategies for Grades 4–12: Reinforcing the Threads of Reading , Alexandria, VA: Association for Supervision and Curriculum Development.
• Thayer-Bacon, Barbara J., 1992, “Is Modern Critical Thinking Theory Sexist?”, Inquiry: Critical Thinking Across the Disciplines , 10(1): 3–7. doi:10.5840/inquiryctnews199210123
• –––, 1993, “Caring and Its Relationship to Critical Thinking”, Educational Theory , 43(3): 323–340. doi:10.1111/j.1741-5446.1993.00323.x
• –––, 1995a, “Constructive Thinking: Personal Voice”, Journal of Thought , 30(1): 55–70.
• –––, 1995b, “Doubting and Believing: Both are Important for Critical Thinking”, Inquiry: Critical Thinking across the Disciplines , 15(2): 59–66. doi:10.5840/inquiryctnews199515226
• –––, 2000, Transforming Critical Thinking: Thinking Constructively , New York: Teachers College Press.
• Toulmin, Stephen Edelston, 1958, The Uses of Argument , Cambridge: Cambridge University Press.
• Turri, John, Mark Alfano, and John Greco, 2017, “Virtue Epistemology”, in Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Winter 2017 Edition). URL = < https://plato.stanford.edu/archives/win2017/entries/epistemology-virtue/ >
• Vincent-Lancrin, Stéphan, Carlos González-Sancho, Mathias Bouckaert, Federico de Luca, Meritxell Fernández-Barrerra, Gwénaël Jacotin, Joaquin Urgel, and Quentin Vidal, 2019, Fostering Students’ Creativity and Critical Thinking: What It Means in School. Educational Research and Innovation , Paris: OECD Publishing.
• Warren, Karen J. 1988. “Critical Thinking and Feminism”, Informal Logic , 10(1): 31–44. [ Warren 1988 available online ]
• Watson, Goodwin, and Edward M. Glaser, 1980a, Watson-Glaser Critical Thinking Appraisal, Form A , San Antonio, TX: Psychological Corporation.
• –––, 1980b, Watson-Glaser Critical Thinking Appraisal: Forms A and B; Manual , San Antonio, TX: Psychological Corporation,
• –––, 1994, Watson-Glaser Critical Thinking Appraisal, Form B , San Antonio, TX: Psychological Corporation.
• Weinstein, Mark, 1990, “Towards a Research Agenda for Informal Logic and Critical Thinking”, Informal Logic , 12(3): 121–143. [ Weinstein 1990 available online ]
• –––, 2013, Logic, Truth and Inquiry , London: College Publications.
• Willingham, Daniel T., 2019, “How to Teach Critical Thinking”, Education: Future Frontiers , 1: 1–17. [Available online at https://prod65.education.nsw.gov.au/content/dam/main-education/teaching-and-learning/education-for-a-changing-world/media/documents/How-to-teach-critical-thinking-Willingham.pdf.]
• Zagzebski, Linda Trinkaus, 1996, Virtues of the Mind: An Inquiry into the Nature of Virtue and the Ethical Foundations of Knowledge , Cambridge: Cambridge University Press. doi:10.1017/CBO9781139174763

How to cite this entry . Preview the PDF version of this entry at the Friends of the SEP Society . Look up topics and thinkers related to this entry at the Internet Philosophy Ontology Project (InPhO). Enhanced bibliography for this entry at PhilPapers , with links to its database.

• Association for Informal Logic and Critical Thinking (AILACT)
• Critical Thinking Across the European Higher Education Curricula (CRITHINKEDU)
• Critical Thinking Definition, Instruction, and Assessment: A Rigorous Approach
• Critical Thinking Research (RAIL)
• Foundation for Critical Thinking
• Insight Assessment
• Partnership for 21st Century Learning (P21)
• The Critical Thinking Consortium
• The Nature of Critical Thinking: An Outline of Critical Thinking Dispositions and Abilities , by Robert H. Ennis

abilities | bias, implicit | children, philosophy for | civic education | decision-making capacity | Dewey, John | dispositions | education, philosophy of | epistemology: virtue | logic: informal

Copyright © 2022 by David Hitchcock < hitchckd @ mcmaster . ca >

• Accessibility

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2024 by The Metaphysics Research Lab , Department of Philosophy, Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

Tool: Checklist for Critical Thinking

RECOMMEND VIA EMAIL

This checklist identifies some of the specific behaviors of critical thinking and can assist educators in assessing themselves and their work.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Glob Health Sci Pract
• v.6(2); 2018 Jun 27

Observe Before You Leap: Why Observation Provides Critical Insights for Formative Research and Intervention Design That You'll Never Get From Focus Groups, Interviews, or KAP Surveys

Steven a. harvey.

a Social and Behavioral Interventions Program, Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.

Associated Data

Four case studies show how observation can uncover issues critical to making a health intervention succeed or, sometimes, reveal reasons why it is likely to fail. Observation can be particularly valuable for interventions that depend on mechanical or clinical skills; service delivery processes; effects of the built environment; and habitual tasks that practitioners find difficult to articulate.

Formative research is essential to designing both study instruments and interventions in global health. While formative research may employ many qualitative methods, focus group discussions and in-depth interviews are the most common. Observation is less common but can generate insights unlikely to emerge from any other method. This article presents 4 case studies in which observation revealed critical insights: corralling domestic poultry to reduce childhood diarrhea, promoting insecticide-treated bed nets (ITNs) to prevent malaria, evaluating skilled birth attendant competency to manage life-threatening obstetric and neonatal complications, and assessing community health worker (CHW) ability to use malaria rapid diagnostic tests (RDTs). Observation of Zambian CHWs to design malaria RDT training materials revealed a need for training on how to take finger-stick blood samples, a procedure second nature to many health workers but one that few CHWs had ever performed. In Lima, Peru, study participants reported keeping their birds corralled “all the time,” but observers frequently found them loose, a difference potentially explained by an alternative interpretation of the phrase “all the time” to mean “all the time (except at some specific seemingly obvious times).” In the Peruvian Amazon, observation revealed a potential limitation of bed net efficacy due to the built environment: In houses constructed on stilts, many people sleep directly on the floor, allowing mosquitoes to bite from below through gaps in the floorboards. Observation forms and checklists from each case study are included as supplemental files; these may serve as models for designing new observation guides. The case studies illustrate the value of observation to clearly understanding clinical practices and skills, details about how people carry out certain tasks, routine behaviors people would most likely not think to describe in an interview, and environmental barriers that must be overcome if an intervention is to succeed. Observation provides a way to triangulate for social desirability bias and to measure details that interview or focus group participants are unlikely to recognize, remember, or be able to describe with precision.

INTRODUCTION

Let's play a quick game of word association: If I say “formative research,” what's the first word or phrase that comes to mind? Some of you, thinking of purpose, might say that formative research is what you do before designing a behavior change campaign. Others, thinking of methods, might say “focus groups.” Both would be wrong. Well, at least partially wrong.

Formative research is important to the design of behavior change campaigns, but it serves many other purposes as well. It is essential to developing research instruments and global health interventions of many kinds. 1 – 4 It can provide the basis for assessing clinical practice, determining how to measure intervention outcomes, planning quality improvement initiatives, and understanding many other aspects of global health programming. 5 – 14 As medical anthropologist Margaret Bentley explains 15 :

The purpose [of formative research] is to provide input into the design of a research study or intervention, including the identification of target populations and appropriate recruitment, retention or consent strategies, development of assessment or evaluation measures, and refinement of intervention components. Formative research allows community participation in the design of research and program protocols, which leads to greater community acceptance.

So formative research is about much more than just behavior change interventions.

Now, what about methods? If you want to do formative research, how should you go about it? Formative research can incorporate many methods, both qualitative and quantitative. Focus groups tend to be the most common, perhaps because they are most familiar. Interviews and knowledge, attitude, and practice (KAP) surveys are also popular. However, as you've probably gathered by now, I'm going to argue that those methods are often insufficient. If you're doing formative research, you should also consider observation.

If you're doing formative research, you should consider observation.

Researchers seem more hesitant about observation than other methods, perhaps because they don't know how to do it, consider it too labor-intensive or costly, feel uncomfortable with the idea of watching other people, or worry about reactivity—the phenomenon where those being observed change their behavior due to the observer's presence. 16 – 18 But observation can generate insights you won't get using any other method. And those insights can often prove critical.

In this article, I present 4 case studies on different global health topics, from corralling domestic poultry to measuring the competency of skilled birth attendants (SBAs). 19 – 21 These examples illustrate some of the scenarios in which observation—both structured and unstructured—can be useful, and they highlight the types of insights it can provide. In each case study, observation yielded critical information that would have been difficult or impossible to obtain any other way. For each case study, I provide a brief description of the research and the context from which it was drawn, then focus more extensively on the observational methods used and the unique insights they generated. Complete descriptions of the original research can be found elsewhere. 22 – 28 I've provided the observation instruments used for each case study as supplemental files.

Ethics Review

The research cited in case studies #1 and #2 was reviewed and approved by the Institutional Review Board of the Johns Hopkins Bloomberg School of Public Health in Baltimore, MD, USA, and by the Ethics Committee of the Asociación Benéfica PRISMA in Lima, Peru. The research cited in case study #3 was reviewed for compliance with the ethics guidelines of the Quality Assurance Project funded by the United States Agency for International Development and approved by Ministry of Health ethics committees or their equivalent in each study country. The research cited in case study #4 received ethics approval from the World Health Organization Special Programme for Research and Training in Tropical Diseases (WHO/TDR) and by the Tropical Disease Research Centre Ethics Committee – Ndola, Zambia.

CASE STUDY #1: CORRALLING DOMESTIC POULTRY TO REDUCE CHILDHOOD DIARRHEA IN LIMA, PERU

Campylobacter jejuni is a common bacterial contributor to diarrheal disease worldwide. 23 , 29 – 31 The bacteria is found almost universally in the intestinal tracts of chickens and can be transmitted to humans from contact with chicken feces or consumption of undercooked chicken. 23 , 32 – 36 In the shanty town outside Lima, Peru, where this study took place, the link between C. jejuni in domestic poultry and childhood diarrhea has been established for decades and confirmed repeatedly. 23 , 32 , 37

Study Context and Observation Methods

The observations described here took place as formative research for a trial to test whether corralling free-range chickens and other domestic poultry would reduce Campylobacter- associated diarrhea by minimizing contact between children and birds. 23 The research team recruited 12 local families raising domestic poultry, built corrals for the poultry at each household, and asked each family to test the corral for 8 weeks. A study team member made weekly visits to each household to complete a 19-item structured observation form ( Supplement 1 ) with space to record variables such as number of birds present; number inside and outside each corral; visual evidence that birds might have been outside the corral recently (e.g., feathers or bird droppings in the yard or inside the house); interaction, if any, between birds and children; cleanliness and structural soundness of each corral; and presence and cleanliness of food and water. The weekly visits were carried out at preselected random times during daylight hours Monday–Saturday. Participants were not notified of visits in advance. This unannounced random schedule made it possible to observe the natural state of each household and corral on different days of the week and at different times of day. In addition, the project sociologist made 3–4 random semi-structured spot checks per household over the 8-week period (30 total across the 12 participating households) noting whether, at the moment of arrival, birds were corralled, children were interacting with birds, birds had adequate food and water, and corrals were in good condition. The sociologist took unstructured notes on anything he judged relevant to feasibility or acceptability of corralling.

Critical Findings

Extent of corralling.

In interviews, participants stated that they kept their birds corralled “all the time.” However, observers found birds loose during 13% of observation visits and 33% of spot checks. Asked about this difference, participants clarified that they let the birds out at certain times such as while cleaning the corrals or to give them time to play ( recrearse )—an activity owners considered essential to their birds' well-being.

Why did participants say they kept their birds corralled all the time when they really didn't? One possible reason is courtesy bias: The project had built them corrals, and so participants may have felt they would disappoint us or seem ungrateful by admitting they didn't always use them. Another possible reason is that they meant something different than we did by “all the time.” Participants took for granted that—like themselves—everyone would understand the need to let birds loose at certain times for practical or health reasons, a “fact” seemingly so obvious as to be unworthy of mention. “All the time” really meant “all the time except at certain (presumably obvious) specific times.” Had we relied solely on interviews (reported behavior), we might never have known that birds were sometimes loose or might never have thought to ask why. Triangulation between what people told us and what we observed revealed critical information about why the intervention might not work.

Participants took for granted that—like themselves—everyone would understand the need to let birds loose at certain times for practical or health reasons, a “fact” seemingly so obvious as to be unworthy of mention.

Sufficient Food and Water

For the local population, one advantage of raising loose poultry was that the birds could find their own food and water. With a corral, the household needed to provide a constant supply of food and water and maintain hygienic conditions. As shown in Figure 1 , both structured observations and spot checks revealed that over the 8-week surveillance only 46% of corrals had food and only 43% had water. Further, corral floors were often wet after birds overturned their water dishes, and food was often rotting. In earlier interviews, participants had expressed concern that corralling would be unhealthy for their birds. Observations made clear that a corralling intervention might validate these concerns unless participants received training on how to keep corrals clean and corralled birds healthy. The data also showed that corralling took more time and effort since someone had to clean the corrals regularly and ensure availability of food and water.

Percentage of Domestic Poultry Corrals Containing Food or Water During Weekly Random Observations, Lima, Peru (N=122 Observations)

Contact Between Poultry and Children

The primary objective of corralling was to break the Campylobacter transmission cycle by separating birds from children. Observations demonstrated that children took a keen interest in the new corrals, often swinging on the doors, sticking their fingers through the mesh, or entering to play with the birds. Attempts to childproof corrals with latches or convince parents to keep children away were largely ineffective: Observers continued to encounter children inside. Parents explained that this was natural and appropriate: They wanted their children to grow up around animals. Children as young as 3 were assigned to collect eggs every day. Instead of isolating children from C. jejuni , observations suggested that corralling actually concentrated exposure. This may help explain the finding from a later study that rates of Campylobacter -associated diarrhea among children under 6 were 2 to 7 times higher in corralling households than non-corralling households with the same number of chickens. 38 Without observation, we might have missed the child-bird contact.

Handling of Poultry Manure

One contributor to child Campylobacter exposure not revealed in interviews was household handling of chicken manure. With manure now concentrated in a smaller space, poultry-raising households began to collect it to use as fertilizer. Observers documented that manure removed from coops was often stored in tin cans or buckets outside the coop within easy reach of children. Uncovered storage also allowed the wind to scatter dried manure around the outside of the living area, thus increasing potential contact.

Contrast Between Human and Bird Habitation

Though not part of formal data collection, observers also noted the contrast between human and animal living space. Residents of this area had settled outside Lima as squatters, often after fleeing rural terrorism in the 1980s. Most worked as casual laborers, domestic servants, or textile piece-workers earning the equivalent of $4.00 to $5.00 per day. Many lived in houses cobbled together from discarded materials, often scavenged from construction sites or garbage dumps. Corrals, though built as cheaply as possible, were made from new material at an average cost of $60.00 per household. Figure 2 shows a project-constructed corral to the left with the human habitation in the center. After receiving their corrals, more than 1 participant joked that their birds now enjoyed a higher standard of living than the human members of the family. Documenting this contrast offered a perspective beyond that likely to be achieved through interviews or focus groups alone.

Contrast Between Human and Animal Living Spaces Documented Through Observation, Las Pampas de San Juan de Miraflores, Peru

Project-constructed poultry corral (left foreground) vs. human habitation (center background). Project participants sometimes joked that the birds in the project enjoyed a better standard of living than the people. © 1999 Steven Harvey.

CASE STUDY #2: BED NETS FOR MALARIA PREVENTION IN THE PERUVIAN AMAZON

Malaria was virtually eliminated from the Peruvian Amazon during the 1970s and 1980s but began to reappear sporadically in the mid-1990s, culminating in an epidemic outbreak in 1997. 39 In response that year, the Peruvian Ministry of Health began distributing ITNs to affected communities. This case study involves observations carried out to evaluate the social acceptability of ITNs and to assess their potential efficacy based on human behavior during the peak biting hours of local malaria-transmitting mosquitoes.

The study took place in 1 peri-urban community and 3 rural villages, all within 30 km of Iquitos, the Peruvian Amazon's largest city. Over 9 months, 4 observers carried out 1 dusk-to-dawn observation in each of 60 households. Upon arrival, the observer used a structured form ( Supplement 2 ) to collect information about the number, ages, and relationships of household occupants; the number and types of sleeping spaces; and the number and types of bed nets. The observer then took unstructured notes at 5-minute intervals throughout the night, recording the location and activities of each household member. Most households consisted of a wooden platform on stilts raised about 2 meters off the ground and covered with a thatched roof. These structures had few rooms or interior dividers, so observers could follow most household activities from a single vantage point. 17 , 40

Net Use During Peak Biting Hours

A key concern about ITN effectiveness in the Americas is whether people are likely to be inside a net during the hours when local malaria-transmitting mosquitoes bite. Observation allowed us to systematically document net use. As shown in Figure 3 , people began to enter their nets for the night as early as 7:00 p.m., but only about half the population was inside a net by 8:30 p.m. and slightly less than 80% by 9:30 p.m., the peak biting hour for Anopheles darlingi , the Amazon's most important malaria vector. 42 This suggests that ITNs might be somewhat effective, but not as effective as in Africa where principal vector species feed later at night. Rather than observing all night, we might have simply asked people what time each member of the household went to bed the previous night, but in a setting where few people had watches or clocks, it would have been hard for them to respond with much precision. Social desirability bias might also have affected people's reports about their own behavior: At the time, the Ministry of Health was running a campaign encouraging people to enter their nets at dusk—a practice unlikely to be feasible in an area near the equator where the sun sets around 6:30 p.m. throughout the year.

Percentage of the Population in Bed by Half-Hour (N=60 Observations) Compared With Anopheles darlingi Feeding Behavior, a Department of Loreto, Peru

a Data on mosquito feeding behavior come from Vittor (2003). 41

Multiple Entries and Exits

One unanticipated finding was the number of times people enter and exit ITNs during the night. 43 Each time the net is lifted, mosquitoes have an opportunity to enter. Parents who share nets with children may spend considerable time outside the net unprotected after their children have gone to sleep. The Table shows an example of a single sleeping space occupied by a 23-year-old mother and her 2-year-old son. The net was lifted a total of 20 times between 7:00 p.m. and 6:30 a.m. The mother spent 195 minutes outside the net between the first time she entered with her son at 7:00 p.m. and the time both of them got out of bed at 6:30 the next morning.

Observational Bed Net Entry and Exit Data From a Single Sleeping Space With 2 Occupants, a 23-Year-Old Mother and Her 2-Year-Old Son, Peruvian Amazon

An unanticipated finding from an observational study of bed net use was the number of times people enter and exit their bed nets during the night—as many as 20 times for 1 mother with a young son.

Additional Potential Risk Factors

Observations revealed other phenomena that would have been difficult to capture with interviews or focus groups. For instance, observers took detailed notes on sleeping spaces in participating households. These notes revealed that many people slept directly on cane flooring rather than on a bed. The flooring had gaps between the cane staves. Since many houses were built on stilts, this meant mosquitoes could enter the sleeping space from below. A net alone could not provide adequate protection in this setting: An effective malaria prevention intervention would need to help at-risk individuals find a way to protect themselves from below as well as from above. Observers also documented other practices that might increase exposure risk: attending evening church services during peak biting hours, bathing after dark, running small home-based stores where community members came to buy food or basic necessities in the evening hours, and other nighttime activities such as hunting, fishing, or charcoal production. While study participants reported some of these activities during interviews, direct observation allowed the study team to document them more systematically.

CASE STUDY #3: ASSESSING THE COMPETENCY OF SKILLED BIRTH ATTENDANTS IN 7 COUNTRIES

About 90% of the 300,000–350,000 annual maternal deaths worldwide are caused by 5 common obstetric complications: postpartum hemorrhage, pregnancy-induced hypertension, obstructed labor, perinatal sepsis, and postabortion complications. 44 , 45 Risk for experiencing one of these life-threatening complications cannot be reliably predicted in advance, but most can be treated successfully if the woman experiencing them has access to basic or comprehensive essential obstetric care delivered by an SBA. For this reason, the World Health Organization (WHO) recommends that all pregnant women be assisted by an SBA during labor and delivery. 46 Several international organizations have defined the competencies necessary to manage these complications. The observations described below were carried out as part of developing a method to assess these competencies among practicing SBAs in low- and middle-income countries.

Testing a clinician's competency to manage a complication according to standards requires assessing not only abstract knowledge but also physical or manual ability. Knowledge can be measured using a written exam, but the only way to assess manual skill is by watching someone perform a task to see whether she or he does it correctly. Assessing skills on actual patients, however, is problematic. Ethically, an observer qualified to evaluate clinical competency would need to stop observing and intervene before allowing an insufficiently skilled provider to endanger a patient's life or well-being. Moreover, even common obstetric complications are relatively rare. This makes it impossible to assess the skill of more than a handful of providers using actual patients.

While knowledge can be measured using a written exam, the only way to assess manual skill is by watching someone perform a task.

The observations discussed here were designed to test SBA competency at performing 4 critical procedures. The first 3 procedures—active management of the third stage of labor (AMTSL), manual removal of the placenta, and bimanual uterine compression—are performed to prevent or control postpartum hemorrhage in a mother who has just given birth. The fourth, neonatal resuscitation with an Ambu bag, is used to treat neonatal asphyxia. The project, eventually carried out in Benin, Ecuador, Jamaica, Kenya, Nicaragua, Rwanda, and Tanzania, used expert obstetrician/gynecologists and pediatricians from host countries as observers. SBAs being assessed performed each procedure on an anatomical model (Gaumard S500 Advanced Childbirth Simulator and Simulaids Sani-Baby CPR mannequin or Gaumard S320 Newborn Airway Trainer); observers assessed competency using a structured step-by-step checklist ( Supplement 3 ). 27 , 28

Correct hand position and movement are essential to successfully performing all 4 tasks. Controlled cord traction, an elective component of AMTSL, requires exerting a gentle downward pull on the umbilical cord with one hand while using the other to prevent uterine inversion by applying counter-traction just above the pubic bone. 47 In case of retained placenta, manual removal requires inserting the hand through the vaginal canal and using a gentle lateral motion to detach the placenta intact, leaving no fragments that could provoke continued bleeding or cause sepsis. Figure 4 shows an expert observer demonstrating manual removal with the Gaumard Advanced Childbirth Simulator. The open abdominal cavity allows the observer to assess the technique of the SBA being observed. Some SBAs might be able to describe these or similar procedures, but even a precise detailed description would not necessarily indicate ability to perform them.

Demonstration of the Correct Hand Position for Manual Removal of a Retained Placenta on an Anatomical Model

© 2006 Steven Harvey

Observations across the 7 study countries revealed the following:

• Though AMTSL is commonly included in national standards for managing uncomplicated delivery, most SBAs did not know how to perform controlled cord traction.
• Similarly, most SBAs could not demonstrate the correct hand positions for carrying out the manual removal of a retained placenta. Although bimanual uterine compression is a relatively simple procedure requiring no instruments or equipment, virtually no SBA was familiar with it.

Neonatal Resuscitation With an Ambu Bag: Correct vs. Incorrect Positioning

Left: Correct positioning of mask, bag, and newborn's head to achieve a good seal, with bag perpendicular to the newborn's body. © 2006 Steven Harvey.

Right: Incorrect positioning, with bag parallel with the newborn's body, making it more difficult to achieve a good seal. © 2002 Steven Harvey.

Using checklists adapted to each country's norms, observation also enabled the study team to assess whether SBAs followed prescribed infection prevention guidelines including handwashing, gloving, and post-procedure decontamination. Participating SBAs were provided with all necessary supplies and equipment. At the beginning of each assessment, the observer instructed each participant to “begin by preparing yourself, the equipment, and the patient,” then noted if the SBA proceeded in accordance with norms. At the end, the observer similarly instructed each participant to “please tell me what more you would do or ask someone else to do once you have finished the procedure.”

It's tempting to classify this research as summative since its initial objective was to assess existing health worker skills. But it was also formative , because the results helped shape interventions: In the short term, observers offered feedback and retraining to each participant, and sometimes—when many participants had a particular weakness in common—to the entire group. In the longer term, findings have influenced training programs and assessment methods in participating countries and around the globe.

CASE STUDY #4: ASSESSING CHW ABILITY TO USE MALARIA RAPID DIAGNOSTIC TESTS IN ZAMBIA

For decades leading up to the early 2000s, malaria in sub-Saharan Africa was diagnosed presumptively: Anyone with a fever was presumed to have malaria and treated with antimalarials. This practice developed because the supply of both microscopes and trained microscopists was too limited to diagnose more than a tiny fraction of febrile patients. In addition, first-line antimalarial drugs were cheap and adverse effects negligible, so presumptive treatment involved minimal cost and risk. After introduction of artemisinin combination therapy as first-line treatment for malaria starting around 2004, WHO recommended parasite-based diagnosis first for adults and older children, then for all suspected cases of malaria regardless of age. 48 Malaria rapid diagnostic tests (RDTs) make parasite-based diagnosis possible even at health facilities with no laboratory, microscope, or microscopist. In many areas, however, febrile patients seek treatment at the community level without ever visiting a health facility. The observations described in this case study were carried out to determine whether volunteer community health workers (CHWs) could use RDTs safely and accurately and, if so, what sort of training materials they needed.

Based on focus group discussions with Zambian CHWs, the study team designed a job aid and brief training curriculum. We used structured observation to pilot test these materials. Study team members observed 79 CHWs prepare 3 RDTs each and recorded the results on a 16-item checklist ( Supplement 4 ). 24 , 25

• Malaria RDTs require using a sterile lancet to draw a finger-stick blood sample, a procedure that is second nature to many professional health workers. Due to concerns about HIV and other blood-borne diseases, however, most African CHWs were prohibited from taking finger-stick blood samples. The Zambian Medical Council authorized the practice for this study, but few participating CHWs had ever taken a sample or used a lancet. During training, observers noticed that instead of drawing blood with a quick stab—the preferred approach—many CHWs set the point of the lancet on the patient's fingertip, then pushed it into the skin. Participants explained they were doing this for fear that stabbing would cause the patients too much pain, but the effect was just the opposite: Pushing was more painful. In addition, it often produced too little blood, thus necessitating a second, third, or even fourth finger prick. Observing this made clear that CHWs needed specific training on proper lancet technique. The study team subsequently developed a training module demonstrating how to extract sufficient blood with a single prick. Improved CHW technique reduced patient discomfort and increased testing quality.
• Watching CHWs transfer blood from fingertip to test cassette yielded a similar revelation. The project RDT came packaged with a loop-shaped blood transfer device designed to collect a 5 μl film of blood across the width of the loop. CHWs did the finger prick with the ball of the patient's finger facing up, then tried to collect the drop from above. This often conveyed too little blood to the test cassette even after multiple tries. Noting this, an experienced observer suggested pricking the finger, rotating the patient's hand 180°, then collecting the drop from underneath with the ball of the finger facing down. In most cases, this made it possible to collect and transfer the precise volume of blood required on the first attempt.
• A key concern related to blood safety was correct disposal of the blood-contaminated lancet. To minimize danger to patients, CHWs, and the community, the research team distributed sharps boxes to all participating CHWs and instructed them to deposit the used lancet into the sharps box immediately after pricking the patient's finger. Setting down the used lancet prior to disposal heightens risk of finger-stick injuries. Observers noticed that positioning the sharps box appropriately made immediate disposal convenient: For a right-handed CHW, this meant placing the sharps box on the right side of the work space, and vice versa for a left-handed CHW. Placing the box on the opposite side of the CHW's dominant hand forced the CHW to reach across both his or her own body and that of the patient. This made handling the used lancet more risky and immediate disposal more difficult.

Malaria Rapid Diagnostic Test Job Aid

A job aid for community health workers lists at the top all supplies and equipment that the worker needs to assemble prior to conducting a rapid diagnostic test for malaria.

• Watching CHWs provide services from home led to another observational finding: Many CHW homes lack electricity and thus have poor-quality artificial lighting. This fact can affect the accuracy of test interpretation when RDTs are prepared inside, especially after dusk or during inclement weather. The RDT's positive test line—indicating that a patient is infected with malaria—can often be quite faint. With inadequate artificial lighting inside and insufficient natural light outside, a CHW could easily misread a faint positive result as negative, thus leaving an infected patient untreated. Realizing this led to added emphasis during training that positive lines are sometimes quite faint and that CHWs should read results in the brightest light possible to avoid missing a faint positive.

Observation produced novel insights in the case studies just described, but how do you decide when observation might be valuable or even essential for your intervention or study? To answer this, it's useful to think in terms of categories of events or processes. Among others, these might include mechanical skills, health service delivery processes, effects of the built environment, and habitual practices that people would have difficulty articulating, sometimes known as “tacit knowledge.” 49 , 50

Observation can produce novel insights, but how do you decide when it might be valuable or even essential for your intervention or study?

Mechanical Skills

The SBA and RDT case studies both illustrate the value of observation to understanding mechanical skills, including critical details such as the correct hand position needed to effectively carry out a lifesaving obstetric or neonatal intervention. Manual removal of a retained placenta or resuscitation of an asphyxiated newborn are two examples. Although lancet technique, sharps box position, or collecting blood with the fingertip facing up or down might seem like minute details when preparing an RDT, they can make the difference between effective, efficient, safe practices and practices that lead to incorrect results or endanger the patient, the health worker, or the community. Observation in these cases is critical not only to diagnose lapses but also to identify interventions that can address them. Observation thus led to additional practical training for SBAs and to development of specific training modules and revised job aid illustrations for malaria RDTs. Beyond their specific substantive findings, these two studies highlight the value of observation to understanding both health worker and community behavior.

Sequential Processes

Many public health interventions involve sequential processes: Not only must each step be performed properly, it must also be performed in the proper order. Again, the RDT case study offers an illustrative example: The study team identified 16 discrete steps necessary to correctly prepare and interpret the test; performing them in the wrong order (e.g., opening the sterile lancet before cleaning the finger with an alcohol swab) or the wrong way (depositing the blood drop where the buffer solution is supposed to go) could compromise test accuracy or patient or health worker safety. The observation checklist ( Supplement 4 ) enabled the team to determine the proportion of health workers who completed all steps correctly, identify specific steps where health workers had problems, and modify training to address the problems observed. Greenland et al. used a similar approach in Zambia to determine what proportion of caregivers of young children with diarrhea could prepare oral rehydration solution correctly. 51 Hurley et al. used a combination of structured and unstructured observations to track the flow of pregnant women through antenatal care in Mali and better understand why many completed their visits without receiving intermittent preventive treatment for malaria in pregnancy (IPTp) or received it without any information about the purpose of IPTp. 52 Hermida et al. found observation to be more accurate than patient exit interviews or medical record review for assessing facility-based provider adherence to standards of care for acute lower respiratory infection, diarrheal disease, and family planning counseling. 53 For this reason, observation is often a key component of quality improvement research. 53 , 54 In sum, observation can be an invaluable tool for documenting the necessary steps in a process, identifying where breakdowns occur, and thus pinpointing where intervention is needed. This type of analysis can be useful at the household, community, and health facility levels.

Understanding the Built Environment

The built environment—and sometimes its relationship to the natural environment—can significantly affect disease risk, health service delivery, and the feasibility of health interventions. The Campylobacter study setting consists of dusty desert hills where water is scarce and rain nonexistent (natural environment). Since the poorest people live at the top of those hills with neither wells nor piped water (built environment), many families struggle to provide water for themselves. Water for corralled birds becomes, at best, a secondary priority. Observing the difficulty of obtaining water helped study team members better understand owners' concerns about the effect of corralling on birds' health. Wind (natural environment) combined with open storage of concentrated chicken manure cleaned from the corrals (built environment) turned out to be one form of continued contact between humans and Campylobacter despite corralling.

The built environment was likewise a critical aspect of the bed net study. The structure of a typical bed in the study setting—no mattress and gaps between the wooden or bamboo slats that allowed mosquitoes to bite from underneath—might never have occurred to public health practitioners, most of whom presumably sleep in beds with mattresses. Even had it occurred to them, they would not have been able to collect systematic data on bed configurations without observation. Thus, observation revealed one potential limitation of bed net efficacy in the study setting. This, in turn, revealed a necessary component of any improvement intervention: figure out how to block the gaps between flooring that allowed mosquitoes to enter.

Systematically observing the built environment can be revealing in many settings. By documenting patient flow at health centers and hospitals, maternal health researchers from the Quality Assurance Project helped explain why women arriving with an obstetric complication might encounter significant, sometimes life-threatening, delays before seeing a clinician. 55 – 58 Observing both the size of rooms in a house and their use for multiple purposes (sleeping at night, running a small retail shop during the day) helped explain why some households in Ghana were reluctant to permanently install bed nets over their sleeping spaces and why, in some cases, residents preferred conical nets to rectangular. 59 Observing the dim lighting in CHWs' houses helped explain why CHWs might miss weak positive RDT results and why training programs needed to emphasize the importance of reading test results under bright light. 25 Many U.S. researchers have used observation to study the relationships between built environment, physical activity, available food choices, and chronic diseases such as obesity and diabetes. 60 – 63 As with the discussion of sequential processes above, it is worth reiterating that observations related to the built and natural environments can be useful at the household, community, and health facility levels.

Habitual Practices and Tacit Knowledge

In any setting, people perform a variety of routine activities, the procedures for which they learned at some point in the past, committed to memory, and carry out automatically, almost as if by instinct. Because these activities are habitual, those who perform them often have difficulty articulating the step-by-step process and even come to think of that process as self-evident. Collecting a finger-stick blood sample is a case in point. A health care provider who has done it many times considers it second nature and wonders why a novice finds it so difficult. Observation reveals that the process involves numerous steps: assemble all the supplies before starting, swab the fingertip with alcohol, wait for it to dry, massage the finger to work the blood up into the fingertip, open the sterile lancet, puncture the fingertip with a quick stab, orient the fingertip with the blood drop in the optimum position for the particular blood collection device being used, etc. The experienced provider has internalized all this and performs it without needing to think. The novice may fail to massage the finger, stab too timidly and thus extract too little blood, or orient the fingertip in a less than optimal position and thus collect too little blood, or too much. Observing both expert and novice helps distinguish the differences and thus determine what training the novice requires.

People who perform habitual activities often have difficulty articulating the step-by-step process and even come to think of that process as self-evident.

The Campylobacter study provides additional examples: Interview or focus group participants might fail to mention the many points of contact between children and birds either because they knew the intervention was meant to separate the two (courtesy bias) or because the types of contact were so commonplace as to seem unworthy of mention. Observing children play with birds, feed and water them, collect eggs, and clean corrals provides tangible evidence that those designing public health interventions should take into account both human nature (children like to play with animals) and economic and cultural practices (even a very young child may be assigned household chores; parents may view learning to raise animals as a key life skill). Cumulative findings from these observations contributed to a conclusion that the intervention was unlikely to succeed, a conclusion confirmed by subsequent research demonstrating that corralling, instead of decreasing risk of Campylobacter -associated diarrhea in children, actually doubled it. 38

The bed net study also provides examples: Absent observation, as noted above, public health practitioners might not have thought to ask about bed design. Conversely, mentioning bed design—an aspect of daily existence so routine as to pass virtually unperceived—might never have occurred to a member of the at-risk population. Had interviewers thought to ask, net occupants might also have mentioned that they enter and exit their nets more than once per night, but it is unlikely that they could have reported very precisely the number of entries and exits, the amount of time the net was lifted, or the amount of time different occupants spend outside the net. Observation made it possible to quantify this phenomenon much more systematically. 43

After validating the method, Gittelsohn used structured mealtime observations to estimate differences in caloric and micronutrient intake between men, women, and children in lowland south-central Nepal. 64 – 66 It is unlikely that parents would have been able to provide such detailed information about intra-household food allocation. Bentley et al. used structured observation during formative research to document child feeding practices prior to a nutritional intervention to improve infant growth and development in Andhra Pradesh, India. 10 Brummell used observation to discover tacit knowledge related to the prognosis of patients suffering cardiac arrest and whether to attempt resuscitation in 2 UK hospital emergency departments. 67 Huot and Laliberte Rudman, who used participant observation to learn about the daily routines of refugees in Canada, explain why observation can be so important for understanding habitual phenomena 68 :

The tacit nature of daily occupation can make the details involved in participation difficult to verbalize because respondents may not have reflected upon their occupational engagement in such detail, or may assume that such “minutia” may not be relevant for research.

This statement could be extended to many areas of health at individual, household, community, and facility levels. Often observation, used together with more common methods like interviews or focus groups, is the only way to make such tacit knowledge explicit.

Triangulating Observation Data With Data From Other Methods

In both the case studies described here and many of the examples cited, researchers used observation together with other methods to achieve a more complete picture of a setting, practice, or intervention. Using observation to triangulate information gathered from interviews or focus group discussions can bring to light differences between what people say they do (reported behavior) and what people actually do (observed behavior). In some cases, this may reveal social desirability bias: People over- or under-report a particular behavior because it violates what they perceive to be social norms. Hygiene studies, for instance, have often found that people over-report handwashing at critical times; observation shows much lower levels. 69 , 70

Using observation to triangulate information from interviews or focus group discussions can bring to light differences between what people say they do and what they actually do.

There is no Peruvian data on reported ITN use that we can compare to the case study #2 observation. But there is at least a plausible basis for comparison in Ghana: Nighttime observation of net use in Northern and Upper West Regions found that only 17% of the population used a net at any time during the night. 71 In a malaria indicator survey of the same 2 districts, 51% and 54% of the population reported sleeping under a net. 72 The numbers are not directly comparable for many reasons, so these differences should be interpreted with caution. The observation study is based on a small purposive sample, the survey on a population-based representative sample; the data were collected in different years and at different times of year. But the wide gap suggests a considerable difference between reported and actual net use. Also, for the observation sample, we know when each individual entered and exited his or her net and how long individuals spent protected versus unprotected. All we know from the survey is that the individual reported sleeping under the net at some point during the night—we have no idea for how long.

Triangulation may also reveal that a word, phrase, or concept means something different to participants than to the researcher. The possibility, in the Campylobacter study, that participants who reported keeping their birds in the corral “all the time,” really meant “all the time except for certain specific seemingly obvious times” is one example. Had we employed only interviews in that study, we would likely have concluded—incorrectly—that birds were never loose. Had we employed only observation, we would likely have concluded that birds were loose 20% of the time—more accurate, but not the whole story. Only the combination revealed the differences in meaning and their conflicting unspoken assumptions.

Observation and Reactivity

A key objection to observation is that it leads to reactivity: Those under observation may change their behavior because they know they are being observed. However, this problem is not unique to observation: People also change their behavior when they are being studied in other ways. Survey and interview respondents may answer questions based on what they think society (social desirability bias) or the interviewer (courtesy bias) expect of them. Observer expectancy effect refers to how an observer can shape behavior—deliberately or subconsciously—by providing subtle nonverbal cues such as slight changes in facial expression. The Hawthorne effect was named for a study in which factory workers from both intervention and control groups became more productive because they knew that researchers were testing possible interventions (such as better lighting) to improve productivity. More detailed definitions are beyond the scope of this article but can be found in many social science references. 73 – 76

In one example of reactivity, P.V. Ram and colleagues found evidence of a 35% increase in handwashing when an observer was present compared with when there was no observer and handwashing was detected by a motion sensor hidden within a bar of soap.7 77 But while reactivity often does occur, researchers can measure and adjust for it. 17 Reactivity also diminishes with time: The longer amount of time or the greater number of times people are observed, the less likely they are to react to an observer's presence. 78 – 80 Ram's study concluded that their findings “call into question the validity of structured observation details because it appears that a majority of participating caregivers substantially altered their behavior in the presence of an observer.” But the study included only 1 observation per household. Had Ram's team observed each household multiple times and waited until household members became accustomed to the observer's presence, their results might have been different.

Ram and her colleagues have a point that in some cases a less invasive technological method might be preferable to observation. For example, studies exploring household use of cleaner cookstoves to reduce indoor air pollution often use temperature sensors (called stove use monitors or SUMs) to track which stove is being used when and for how long. 81 , 82 At least one recent study reports that combining observation and SUMs data provides a more accurate picture than SUMs data alone. 83

Moreover, reactivity is often unrelated to the focus behavior. In the bed net study, we identified 339 instances of reactivity across 60 observations using the broadest possible definition: any interaction whatsoever between the observer and any member of the observed household. Of these 339 instances, only 2 were directly related to the behavior of interest: protecting against mosquito bites. 17 In a similar way, John Schnelle and colleagues found that observations did not change provider treatment of nursing home residents in the United Kingdom. 84

Another way to control reactivity is through unannounced spot checks similar to those we used in case study #1. Nazmul Chaudhury and colleagues used this method to chronicle the degree of health worker and teacher absenteeism in health facilities and primary schools in Bangladesh, Ecuador, India, Indonesia, Peru, and Uganda. 85 In his classic article about nighttime observations among the Samukundi Abelam, Richard Scaglion describes how he used spot checks to document time allocation within this Papua New Guinea ethnic group. 86 Scaglion admits, however, that he was not always able to maintain the element of spontaneity that spot check observations are meant to provide:

… it is not easy for an anthropologist in the field to come upon an Abelam unawares. Since I did not want to record “greeting anthropologist” as a frequent activity when people were first observed, I often had to reconstruct what they were doing immediately before I arrived.

In sum, observation can be an essential tool in formative research. As a stand-alone method, it can measure phenomena not measurable by any other method. In combination with interviews or focus groups, it can suggest questions to be posed through these other methods. It can also triangulate findings from other methods, reveal potential differences between reported and observed behavior, and thus help assess social desirability bias. Given these benefits, observation—either alone or in combination with other methods—is something both investigators and program managers should consider when undertaking formative research.

Supplementary Material

Acknowledgments.

I am grateful to Marianne Henry for her help with literature review and manuscript preparation. I wish to thank the editor and editorial staff of GHSP as well as the 3 anonymous reviewers, all of whose comments considerably strengthened this manuscript. I also wish to thank the many participants in the 4 studies described here for their time, patience, and willingness to participate. Finally, I am grateful for the comments and suggestions of the many students with whom I have discussed these concepts in formative research classes over nearly a decade and to Drs. Elli Leontsini and Peter Winch for inviting me to do so.

Peer Reviewed

Competing Interests: None declared.

Funding: Funding for case study #1 was provided by the Thrasher Research Fund (award 02813-1). Funding for case study #2 was provided by the US Agency for International Development (USAID) under Grant Number 527G001000070. Case study #3 was supported by the Quality Assurance Project under contracts number HRN-C-00-96-90013 and GPH-C-00-02-00004-00 with the United States Agency for International Development (USAID). Funding for case study #4 was provided by the Australian Agency for International Development (AusAID), the WHO Special Programme for Research and Training in Tropical Diseases (TDR), and the United States Agency for International Development (USAID) under the Quality Assurance and Workforce Development Project at University Research Co., LLC (contract number GPH-C-00-02-00004-00). Conclusions and opinions are the sole responsibility of the author and do not necessarily reflect the views or policies of the funders.

First Published Online: May 23, 2018

Cite this article as: Harvey SA. Observe before you leap: why observation provides critical insights for formative research and intervention design that you'll never get from focus groups, Interviews, or KAP Surveys. Glob Health Sci Pract. 2018;6(2):299-316. https://doi.org/10.9745/GHSP-D-17-00328

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons
• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

15.6: Teachers’ observation, questioning, and record keeping

• Last updated
• Save as PDF
• Page ID 87553

During teaching, teachers not only have to communicate the information they planned, but also continuously monitor students’ learning and motivation in order to determine whether modifications have to be made (Airasian, 2005). Beginning teachers find this more difficult than experienced teachers because of the complex cognitive skills required to improvise and be responsive to students’ needs while simultaneously keeping in mind the goals and plans of the lesson (Borko & Livingston, 1989).

The informal assessment strategies teachers most often use during instruction are observation and questioning.

Observation

Effective teachers observe their students from the time they enter the classroom. Some teachers greet their students at the door not only to welcome them but also to observe their mood and motivation. Are Hannah and Naomi still not talking to each other? Does Ethan have his materials with him? Gaining information on such questions can help the teacher foster student learning more effectively (e.g. suggesting Ethan goes back to his locker to get his materials before the bell rings or avoiding assigning Hannah and Naomi to the same group).

During instruction, teachers observe students’ behavior to gain information about students’ level of interest and understanding of the material or activity. Observation includes looking at non-verbal behaviors as well as listening to what the students are saying. For example, a teacher may observe that a number of students are looking out of the window rather than watching the science demonstration, or a teacher may hear students making comments in their group indicating they do not understand what they are supposed to be doing.

Observations also help teachers decide which student to call on next, whether to speed up or slow down the pace of the lesson, when more examples are needed, whether to begin or end an activity, how well students are performing a physical activity, and if there are potential behavior problems (Airasian, 2005). Many teachers find that moving around the classroom helps them observe more effectively because they can see more students from a variety of perspectives. However, the fast pace and complexity of most classrooms makes it difficult for teachers to gain as much information as they want.

Questioning

Teachers ask questions for many instructional reasons, including keeping students’ attention on the lesson, highlighting important points and ideas, promoting critical thinking, allowing students to learn from each other’s answers, and providing information about students’ learning. Devising good appropriate questions and using students’ responses to make effective instantaneous instructional decisions is very difficult.

Some strategies to improve questioning include:

• planning and writing down the instructional questions that will be asked
• allowing sufficient wait time for students to respond
• listening carefully to what students say rather than listening for what is expected
• varying the types of questions asked
• making sure some of the questions are higher level
• asking follow-up questions

Table \(\PageIndex{2}\): : Validity and reliability of observation and questioning (problem and strategies to alleviate the problem

Record keeping

Keeping records of observations improves reliability and can be used to enhance understanding of one student, a group, or the whole class’ interactions. Sometimes this requires help from other teachers. For example, Alexis, a beginning science teacher is aware of the research documenting that longer wait time enhances students’ learning (e.g. Rowe, 2003) but is unsure of her behaviors so she asks a colleague to observe and record her wait times during one class period. Alexis learns her wait times are very short for all students so she starts practicing silently counting to five whenever she asks students a question.

Teachers can keep anecdotal records about students without help from peers. These records contain descriptions of incidents of a student’s behavior, the time and place the incident takes place, and a tentative interpretation of the incident. For example, the description of the incident might involve Joseph, a second-grade student, who fell asleep during the mathematics class on a Monday morning.

A tentative interpretation could be the student did not get enough sleep over the weekend, but alternative explanations could be the student is sick or is on medications that make him drowsy. Obviously, additional information is needed and the teacher could ask Joseph why he is so sleepy and also observe him to see if he looks tired and sleepy over the next couple of weeks.

Anecdotal records often provide important information and are better than relying on one’s memory, but they take time to maintain and it is difficult for teachers to be objective. For example, after seeing Joseph fall asleep the teacher may now look for any signs of Joseph’s sleepiness—ignoring the days he is not sleepy. Also, it is hard for teachers to sample a wide enough range of data for their observations to be highly reliable.

Teachers also conduct more formal observations, especially for students who have IEP’s. An example of the importance of informal and formal observations in a preschool follows:

The class of preschoolers in a suburban neighborhood of a large city has eight special needs students and four students—the peer models—who have been selected because of their well-developed language and social skills. Some of the special needs students have been diagnosed with delayed language, some with behavior disorders, and several with autism.

The students are sitting on the mat with the teacher who has a box with sets of three “cool” things of varying size (e.g. toy pandas) and the students are asked to put the things in order by size, big, medium and small. Students who are able are also requested to point to each item in turn and say “This is the big one”, “This is the medium one” and “This is the little one”.

For some students, only two choices (big and little) are offered because that is appropriate for their developmental level. The teacher informally observes that one of the boys is having trouble keeping his legs still so she quietly asks the aid for a weighted pad that she places on the boy’s legs to help him keep them still. The activity continues and the aide carefully observes student’s behaviors and records on IEP progress cards whether a child meets specific objectives such as: “When given two picture or object choices, Mark will point to the appropriate object in 80 per cent of the opportunities.”

The teacher and aides keep records of the relevant behavior of the special needs students during the half day they are in preschool. The daily records are summarized weekly. If there are not enough observations that have been recorded for a specific objective, the teacher and aide focus their observations more on that child, and if necessary, try to create specific situations that relate to that objective. At end of each month the teacher calculates whether the special needs children are meeting their IEP objectives.

National Geographic Education Blog

Bring the spirit of exploration to your classroom.

Ultimate Critical Thinking Cheat Sheet

18 thoughts on “ Ultimate Critical Thinking Cheat Sheet ”

Can I use this for company training? Is there a purchase/copyright.

Hi, Dale: You will need to contact the Global Digital Citizen Foundation to request permission to use this resource.

#hi, where can i get a chart?

Hi, Alma! The good folks at the Global Digital Citizen Foundation can help you here: https://globaldigitalcitizen.org/critical-thinking-skills-cheatsheet-infographic

Where is the actual critical aspect of the “thinking”? I would want to see, “What are the premises of the argument? Are they actually true? What evidence supports it and contradicts it? What other explanations might cause that result? What would be the result of the intervention? Does the proposed intervention actually address the problem identified? Is the problem identified the right problem?”

Thank you, PaulR! The fact that these basic rhetorical questions are missing from this infographic illustrates what is wrong with our current methods of “debate” on issues from climate change to healthcare.

Great post. Thanks

Excellent. Useful for the classroom.

Go to the previous page where you saw this graphic. There is a link above it that will allow you to download it.

I want to purchase

https://globaldigitalcitizen.org/critical-thinking-skills-cheatsheet-infographic

How may I obtain a copy? Thanks.

I’m interested in a copy too.

I’d like to get one too. How can one be purchased? Thank you

Is this a poster that can be purchased? How can I get a copy?

I would appreciate to know if someone have translated that Cheatsheet in French language

Leave a Reply Cancel reply

404 Not found

One ultimate student engagement observation schedule

Nicole Richard January 17, 2019

Teachers know that truly meaningful lerning happens when students are genuinely engagement. Student engagement by the they are actively thinking, asking questions, investigating problems, developing solutions, and taking ownership of the learning process. Engagement plus resources so schoolteachers zu facilitators a education, instead of unmittelbarer training. But what does on involved classroom actually take like in practice, plus how ability we measure it?

There what lots ways an engaged classroom might show itself. The following checklists can help you evaluate engagement strengths also areas for improvement. Apiece measure may not apply to all classrooms or all lessons but taken together, these markers capacity help teachers develop an overall picture of their classroom’s level of engagement. My Viewing Catalog.

4 outstanding classroom observation check forward student engagement

1. Social and emotional highlighters checklist

Soft your like social and emotional learning exist difficult to measure in a classroom, but that doesn’t middling they aren’t crucial on seine function. Creative an atmosphere that is conducting to lerning, while allowing learners to feel safe and heard is negative easy feat. An following prompts may support you establish a plan in make your students feel included in their learning.

☑ Is there find to propose ensure students feel safely attentively sharing an unpopular opinion oder a view that may challenge their teacher?

Contemplate how the teacher reacts to student voices in the grade, particularly ones that, in an reasonable manner, challenge an supply or express disagreement. Are actual apprentice voices heard? Your a range of student vote heard?

☑ Did aforementioned teacher create a “growth mindset” vibes in the classroom?

Good classrooms create to environment which encourages growth. Evidence of like a growth thought atmosphere may be observed taken and teacher’s language as learners meets challenges or through artwork with growth mindset messages in to room. For example, a course who states “I can’t do this” might can encouraged to continue trying with a positive statement such as, “Don’t bother! You’ll get better in practice.” Or may the teacher uses motivational posters over positive messages, such as, “Mistakes are proof that you’re trying.” Ok.  The title is misleadingly.  On purpose. I’ve been observing of newly appointed staff at Highbury Woody go and last few weeks which holds was a real-time pleasure.  For the beginning start, we̵…

☑ Are learner successes celebrated?

How does the teacher react to additionally recognize momentary locus students overcome challenges furthermore reach their goals? Are there systems in place for acknowledging these accomplishments? Analyzing student work able help principals become more efficient instructional coaches.

☑ Is student erroneous redirected through clear, matter-of-fact guidance?

Emotional reactions can escalate student misbehavior and conflicts, which drives to disengagement fork the item student and their surrounding gazes. The teacher must be firm and straightforward, but not emotionally engaged if handling misbehavior. The Prime as Formative Coach

☑ Are contents clear, immediate, predictable, and connected to specific behaviors?

When our understand ampere teacher’s schaft to addressing behavior, misbehaviors can may managed quickly plus about minimalist losing of learning hour.

☑ Are student voices heard as often as the teacher’s voice?

In a student-centered classroom, student choir are involve in a variety of instructional moments — from clarifying directions to exploring challenging questions. This teacher should avoid what the thinking for their students up advance genuine request press exploration. Encouraged higher order thinking through questioning or activities. Addressed the needs of diverse learners though multiple, developmentally appropriate ...

☑ Is there exhibits on student joy in the school?

This measure may be witness through graduate demeanor — smiles, smile, real humor. It might also be evidenced by student excitement to share their thinking alternatively a job product. The ultimate course engagement observation checklist - Resource ...

☑ Is present proof a teacher joy in to education work?

This measure may be evidenced through a teacher’s demeanor — again, grins, laughter, and human. It may including be seen in their enthusiasm for educational one topic of their lesson.

2. Intellect markers checklist

Grades and report cards don’t tell the whole historical! There are many different ways in which students expression his understanding of the subjects, and adenine lot of these can not be directly appears. You bottle use aforementioned follow questions to create at inclusive learning setting .

☑ Can every student have an opportunity to share their thinking verbally, visually, or through writing?

Students may feel most comfortable sharing they ideas through various modes. Does the lesson provide more longer one way for graduate to share the ideas? Does the instruction encourage any pupils to share?

☑ Does one teacher use strategies to solicit participation from every student in the room?

That may being evidenced included a number of path. The teacher might use playing cards otherwise popsicle sticks to randomly select learners for participation, such because for answering a question. The your might use pre-alerting (privately alert a student before calling on them) to prepare a unwilling speaker required sharing; alternatively, the teacher might also ask reluctant sharers in repeat or respond into the notes off other students. And finalize, the educator might used questioning to help guide a student into that correct answer. Approaches here will vary but should ultimately be effective are eliciting take. This criteria should doesn go beyond the oem performance task, but reflect higher to thinking skills which students could demonstrate within ...

☑ Are objectives plain and einrahmung around higher-order-thinking questions?

Clear objectives invite purpose to student work. Higher-order-thinking questions producing engagement because they are typically open-ended and explorable. Classroom Observation Checkout Speech Therapy - web.mei.edu

☑ Exist targets relevant to students? Are undergraduate mentioned why a lesson is relevant?

Apprentices what more engaged when they feel a sensation of purpose in what they live doing. If which relevance of a particular activity is not immediately clear, the teacher must take steps to explain instead show how the learning has connected to students’ present or future lives. LESSON OBSERVATION CHECKLIST

☑ How students have the opportunity to discuss higher-order-thinking questions with their peer?

The teacher might facilitate peer-to-peer discussions by building in procedures such as turn real shows or pair shares — where apprentices quickly decide a pose or release their work with an classmate — or procedures like balcony walks, fish discussions , and Socratic kirchenkreise.  

☑ Does classroom chat move from peer to peer, allowing them to reaction to peer thinking, instead of always moving between teacher press student?

Often, classroom discussions ends up looking more like short interactions between the teachers and specific students. Unfortunately, this affords other students the opportunity to misbehave or stop paying attention. Instead, the teacher can help facilitate peer-to-peer discussions, where students directly add towards, asked, or respond to jede other’s statements. Masters can support this by providing discussions sentence startup, such while, “I agree/disagree because ___,” “I’m hearing you say ___,” also “Adding onto ___’s idea…” A type of questions or tasks that promote higher-order thinking skills (e.g., literal, analytical, and interpretive questions). Interplay. 4. 3. 2. 1. 0.

☑ Do students have the shot on solve problematic together with ihr gazes?

For effective knowledge, students must to occasionally works together on associations and in-class recent. This sort von group work should need discussion, debate, additionally reflection with peers.

☑ Are there staging in place to help all student meet the equivalent objectives?

A task that is too easy or too difficult will cause students until abandon their learning efforts. How does the teaches support students with differences aptitude levels so that they can all reaching and same goal? IMPROVING READING COMPREHENSION THROUGH HIGHER ...

☑ A there evidence that assignments have been differentiating based on student needs?

Differentiation does not mean giving different reassignments otherwise duty — he are making one assignment or work accesible to all pupils. Differentiation may must evidenced by assignment adaptations enjoy sentence starters or the show of a writing organizer. Builds on or relates to participants' prior learning. 10. Engages participants in higher-order thinking to studying each critical term. 11. Prompts each ...

☑ Lives risk-taking encouraged in adenine safe learning surrounding?

A can be challenge to measure risk-taking by observational evidence. Anyway, this might be seen through one student’s willingness to venture an get to a challenging question and by their reaction upon failure. Consider such fine whether students in the room keep getting closer down after facing challenges. How rabbits the teacher reply to press prompt students to persist in their learning after a challenge or failure?

☑ Does the teacher have strategies by checkout every students’ understanding?

In other words, does the teacher make use of formative assessments? A student who felt lost or behind may disengage from the lesson to avoid an discomfort of failure. Formative judgments ensure that instructor quickly name and address student disorientation.

☑ Is high standards for apprentice behavior and work quality apparent in the schoolroom?

This may be evidenced throughout a teacher’s feedback to students, their requirements forward an assignment, of rigor is an assignment itself, or adenine clear operation for meas student outcomes, such like adenine exhaustive rubric that outlines total expectations. ☑ Do current have the opportunity for discuss higher-order-thinking faq are their peers? The teacher might facilitate peer-to-peer ...

3. Physical markers checklist

Sometimes, minor objects like the adjust up a your classify can have unforeseen impact on its atmosphere. View these few prompts to confirm if my classroom configuration is conducive to learning.

☑ Are desks or seats arranged to facilitate discussion or association? Is the desk with seat arrangement student-centered?

Many traditional seating arrangements are inordinately teacher centered — such as an arrangement of several rows of desktops — and therefore ineffective. Does that physiology setup of the classroom help appropriate student-to-student discussions? One example of such a seating arrangement is the horseshoe-shaped layout, where students can send please the front board both one another, and the instructor is given amply room up walk about plus address all students.

☑ Are there options for pliant seating?

Flexible seating means students can choose between a type of different seating options, such as a traditional desk, a group table, a bebean, or the rug. Flexible places is not appropriate in all classroom contexts, nevertheless e offers college with a sense of control and is worth considering locus appropriate. Teamwork Assessment Think Social. Publishing, Incl. NOTE: Save ... Create tests that focus on higher-order thinking skills. CLASSROOM ...

☑ Do my visuals support content or skill goals from the curriculum? Perform classroom visuals enhance or support student understanding of the material?

Is may involve posting an aimed or essential question where it’s visible to keep students focused on a knowledge goal throughout the duration of the class period. It may also participate the use of show and videos to accompany traditional instruction. Critical thinking is a higher-order thinking ability. Higher-order thinking skills go beyond basic observation out facts and memorization. They are what we are ...

4. Structural markers checklist

A “one-size-fits-all” class planning doesn’t exist, but present are cunning improvements that you may make to introduce a set that is inclusive to your students. Balancing determined rituals with looser scheduled time and punctuating periods on centered work the more whimsical activities can be a wonderful step forward. These questions can help you assess your appointment and its effects on our students.

☑ Can here free routines furthermore procedures in place such student having learned?

Well-defined school procedures and routines may not seem right networked on engagement — but when students have internalized these structures real can habit them without receiving constant reminders, then class time is releasing up for more interesting academe chasing. Moreover, with clear procedures in space, apprentices will feel a greater sensory the independence and maturity wenn their don’t feel like they are constantly being reminded to make something. students' higher-order thinking skills own not been developed or mastered. ... understanding the the reading strategies was an observation checklist. Beginning ...

☑ Are lessons structured to include multiple learning modalities?

Teachers know that students have different learning styles, suchlike such graphic, kinesthetic, oder auditory. As it allow not been any on tailor events to every modality, a sound instructional should providing multiple avenues for completion or participation so that different kinds of students can succeed. The Sheltered Instruction Observation Protocol (SIOP) (Echevarria ...

☑ Are teach structured to avoid downtime?

ONE well-planned lesson ensures such students are does waiting for their tutor to set up activities or by peers to finish a task before they pot move on. A downtime-free lesson might include challenge work for early finishing, a list of classroom jobs, or simply one teacher moving some students through activities without crush one instructional flow for others. Listings, Site Scales and Rubrics (Assessment)

☑ Do learn include frequently changing tasks or activities?

Students, especially in elementary grades, struggle to stays focused on tasks for long periods away laufzeit. Is the lesson divisible into a range of quick-moving activities to keep students alert and off their toes?

☑ Are “brain breaks” present include the lesson structure?

Again, we know that learners struggle to maintain her long-term focus. Mastermind breaks can be in the form of five minutes of liberate type, the opportunity to watch a funny video, ampere meditation break, or ampere movement break that allow students to get up and out of their seats. These breaks can developmentally appropriate forward children and teens and allow them till recoup before diving back into rigorous work. Just like too much home can overpowering the test counterproductive to learning, so can an overly jam-packed lesson.

Find what works for your students

Student engagement maybe face different in each kurs additionally in everyone lesson, but fundamentally, it always implicated that same principle — students wanting (and being able) to learn. Save markers intention help you more concretely assess whether a teacher’s classroom is conducive to scholar conflict.

Photo credit: Lowrey Sullivan ; Pettycon  / Flickr.com; Pixabay.com

Student Engagement