research gap in the literature

The Research Gap (Literature Gap)

Everything you need to know to find a quality research gap

By: Ethar Al-Saraf (PhD) | Expert Reviewed By: Eunice Rautenbach (DTech) | November 2022

If you’re just starting out in research, chances are you’ve heard about the elusive research gap (also called a literature gap). In this post, we’ll explore the tricky topic of research gaps. We’ll explain what a research gap is, look at the four most common types of research gaps, and unpack how you can go about finding a suitable research gap for your dissertation, thesis or research project.

Overview: Research Gap 101

• What is a research gap
• Four common types of research gaps
• Practical examples
• How to find research gaps
• Recap & key takeaways

What (exactly) is a research gap?

Well, at the simplest level, a research gap is essentially an unanswered question or unresolved problem in a field, which reflects a lack of existing research in that space. Alternatively, a research gap can also exist when there’s already a fair deal of existing research, but where the findings of the studies pull in different directions , making it difficult to draw firm conclusions.

For example, let’s say your research aims to identify the cause (or causes) of a particular disease. Upon reviewing the literature, you may find that there’s a body of research that points toward cigarette smoking as a key factor – but at the same time, a large body of research that finds no link between smoking and the disease. In that case, you may have something of a research gap that warrants further investigation.

Now that we’ve defined what a research gap is – an unanswered question or unresolved problem – let’s look at a few different types of research gaps.

Types of research gaps

While there are many different types of research gaps, the four most common ones we encounter when helping students at Grad Coach are as follows:

• The classic literature gap
• The disagreement gap
• The contextual gap, and
• The methodological gap

Need a helping hand?

1. The Classic Literature Gap

First up is the classic literature gap. This type of research gap emerges when there’s a new concept or phenomenon that hasn’t been studied much, or at all. For example, when a social media platform is launched, there’s an opportunity to explore its impacts on users, how it could be leveraged for marketing, its impact on society, and so on. The same applies for new technologies, new modes of communication, transportation, etc.

Classic literature gaps can present exciting research opportunities , but a drawback you need to be aware of is that with this type of research gap, you’ll be exploring completely new territory . This means you’ll have to draw on adjacent literature (that is, research in adjacent fields) to build your literature review, as there naturally won’t be very many existing studies that directly relate to the topic. While this is manageable, it can be challenging for first-time researchers, so be careful not to bite off more than you can chew.

2. The Disagreement Gap

As the name suggests, the disagreement gap emerges when there are contrasting or contradictory findings in the existing research regarding a specific research question (or set of questions). The hypothetical example we looked at earlier regarding the causes of a disease reflects a disagreement gap.

Importantly, for this type of research gap, there needs to be a relatively balanced set of opposing findings . In other words, a situation where 95% of studies find one result and 5% find the opposite result wouldn’t quite constitute a disagreement in the literature. Of course, it’s hard to quantify exactly how much weight to give to each study, but you’ll need to at least show that the opposing findings aren’t simply a corner-case anomaly .

3. The Contextual Gap

The third type of research gap is the contextual gap. Simply put, a contextual gap exists when there’s already a decent body of existing research on a particular topic, but an absence of research in specific contexts .

For example, there could be a lack of research on:

• A specific population – perhaps a certain age group, gender or ethnicity
• A geographic area – for example, a city, country or region
• A certain time period – perhaps the bulk of the studies took place many years or even decades ago and the landscape has changed.

The contextual gap is a popular option for dissertations and theses, especially for first-time researchers, as it allows you to develop your research on a solid foundation of existing literature and potentially even use existing survey measures.

Importantly, if you’re gonna go this route, you need to ensure that there’s a plausible reason why you’d expect potential differences in the specific context you choose. If there’s no reason to expect different results between existing and new contexts, the research gap wouldn’t be well justified. So, make sure that you can clearly articulate why your chosen context is “different” from existing studies and why that might reasonably result in different findings.

4. The Methodological Gap

Last but not least, we have the methodological gap. As the name suggests, this type of research gap emerges as a result of the research methodology or design of existing studies. With this approach, you’d argue that the methodology of existing studies is lacking in some way , or that they’re missing a certain perspective.

For example, you might argue that the bulk of the existing research has taken a quantitative approach, and therefore there is a lack of rich insight and texture that a qualitative study could provide. Similarly, you might argue that existing studies have primarily taken a cross-sectional approach , and as a result, have only provided a snapshot view of the situation – whereas a longitudinal approach could help uncover how constructs or variables have evolved over time.

Practical Examples

Let’s take a look at some practical examples so that you can see how research gaps are typically expressed in written form. Keep in mind that these are just examples – not actual current gaps (we’ll show you how to find these a little later!).

Context: Healthcare

Despite extensive research on diabetes management, there’s a research gap in terms of understanding the effectiveness of digital health interventions in rural populations (compared to urban ones) within Eastern Europe.

Context: Environmental Science

While a wealth of research exists regarding plastic pollution in oceans, there is significantly less understanding of microplastic accumulation in freshwater ecosystems like rivers and lakes, particularly within Southern Africa.

Context: Education

While empirical research surrounding online learning has grown over the past five years, there remains a lack of comprehensive studies regarding the effectiveness of online learning for students with special educational needs.

As you can see in each of these examples, the author begins by clearly acknowledging the existing research and then proceeds to explain where the current area of lack (i.e., the research gap) exists.

How To Find A Research Gap

Now that you’ve got a clearer picture of the different types of research gaps, the next question is of course, “how do you find these research gaps?” .

Well, we cover the process of how to find original, high-value research gaps in a separate post . But, for now, I’ll share a basic two-step strategy here to help you find potential research gaps.

As a starting point, you should find as many literature reviews, systematic reviews and meta-analyses as you can, covering your area of interest. Additionally, you should dig into the most recent journal articles to wrap your head around the current state of knowledge. It’s also a good idea to look at recent dissertations and theses (especially doctoral-level ones). Dissertation databases such as ProQuest, EBSCO and Open Access are a goldmine for this sort of thing. Importantly, make sure that you’re looking at recent resources (ideally those published in the last year or two), or the gaps you find might have already been plugged by other researchers.

Once you’ve gathered a meaty collection of resources, the section that you really want to focus on is the one titled “ further research opportunities ” or “further research is needed”. In this section, the researchers will explicitly state where more studies are required – in other words, where potential research gaps may exist. You can also look at the “ limitations ” section of the studies, as this will often spur ideas for methodology-based research gaps.

By following this process, you’ll orient yourself with the current state of research , which will lay the foundation for you to identify potential research gaps. You can then start drawing up a shortlist of ideas and evaluating them as candidate topics . But remember, make sure you’re looking at recent articles – there’s no use going down a rabbit hole only to find that someone’s already filled the gap 🙂

Let’s Recap

We’ve covered a lot of ground in this post. Here are the key takeaways:

• A research gap is an unanswered question or unresolved problem in a field, which reflects a lack of existing research in that space.
• The four most common types of research gaps are the classic literature gap, the disagreement gap, the contextual gap and the methodological gap. 
• To find potential research gaps, start by reviewing recent journal articles in your area of interest, paying particular attention to the FRIN section .

If you’re keen to learn more about research gaps and research topic ideation in general, be sure to check out the rest of the Grad Coach Blog . Alternatively, if you’re looking for 1-on-1 support with your dissertation, thesis or research project, be sure to check out our private coaching service .

Psst… there’s more (for free)

This post is part of our dissertation mini-course, which covers everything you need to get started with your dissertation, thesis or research project. 

You Might Also Like:

29 Comments

This post is REALLY more than useful, Thank you very very much

Very helpful specialy, for those who are new for writing a research! So thank you very much!!

I found it very helpful article. Thank you.

Just at the time when I needed it, really helpful.

Very helpful and well-explained. Thank you

VERY HELPFUL

We’re very grateful for your guidance, indeed we have been learning a lot from you , so thank you abundantly once again.

hello brother could you explain to me this question explain the gaps that researchers are coming up with ?

Am just starting to write my research paper. your publication is very helpful. Thanks so much

How to cite the author of this?

your explanation very help me for research paper. thank you

Very important presentation. Thanks.

Best Ideas. Thank you.

I found it’s an excellent blog to get more insights about the Research Gap. I appreciate it!

Kindly explain to me how to generate good research objectives.

This is very helpful, thank you

Very helpful, thank you.

Thanks a lot for this great insight!

This is really helpful indeed!

This article is really helpfull in discussing how will we be able to define better a research problem of our interest. Thanks so much.

Reading this just in good time as i prepare the proposal for my PhD topic defense.

Very helpful Thanks a lot.

Thank you very much

This was very timely. Kudos

Great one! Thank you all.

Thank you very much.

This is so enlightening. Disagreement gap. Thanks for the insight.

How do I Cite this document please?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• Privacy Policy

Buy Me a Coffee

Home » Research Gap – Types, Examples and How to Identify

Research Gap – Types, Examples and How to Identify

Table of Contents

Research Gap

Definition:

Research gap refers to an area or topic within a field of study that has not yet been extensively researched or is yet to be explored. It is a question, problem or issue that has not been addressed or resolved by previous research.

How to Identify Research Gap

Identifying a research gap is an essential step in conducting research that adds value and contributes to the existing body of knowledge. Research gap requires critical thinking, creativity, and a thorough understanding of the existing literature . It is an iterative process that may require revisiting and refining your research questions and ideas multiple times.

Here are some steps that can help you identify a research gap:

• Review existing literature: Conduct a thorough review of the existing literature in your research area. This will help you identify what has already been studied and what gaps still exist.
• Identify a research problem: Identify a specific research problem or question that you want to address.
• Analyze existing research: Analyze the existing research related to your research problem. This will help you identify areas that have not been studied, inconsistencies in the findings, or limitations of the previous research.
• Brainstorm potential research ideas : Based on your analysis, brainstorm potential research ideas that address the identified gaps.
• Consult with experts: Consult with experts in your research area to get their opinions on potential research ideas and to identify any additional gaps that you may have missed.
• Refine research questions: Refine your research questions and hypotheses based on the identified gaps and potential research ideas.
• Develop a research proposal: Develop a research proposal that outlines your research questions, objectives, and methods to address the identified research gap.

Types of Research Gap

There are different types of research gaps that can be identified, and each type is associated with a specific situation or problem. Here are the main types of research gaps and their explanations:

Theoretical Gap

This type of research gap refers to a lack of theoretical understanding or knowledge in a particular area. It can occur when there is a discrepancy between existing theories and empirical evidence or when there is no theory that can explain a particular phenomenon. Identifying theoretical gaps can lead to the development of new theories or the refinement of existing ones.

Empirical Gap

An empirical gap occurs when there is a lack of empirical evidence or data in a particular area. It can happen when there is a lack of research on a specific topic or when existing research is inadequate or inconclusive. Identifying empirical gaps can lead to the development of new research studies to collect data or the refinement of existing research methods to improve the quality of data collected.

Methodological Gap

This type of research gap refers to a lack of appropriate research methods or techniques to answer a research question. It can occur when existing methods are inadequate, outdated, or inappropriate for the research question. Identifying methodological gaps can lead to the development of new research methods or the modification of existing ones to better address the research question.

Practical Gap

A practical gap occurs when there is a lack of practical applications or implementation of research findings. It can occur when research findings are not implemented due to financial, political, or social constraints. Identifying practical gaps can lead to the development of strategies for the effective implementation of research findings in practice.

Knowledge Gap

This type of research gap occurs when there is a lack of knowledge or information on a particular topic. It can happen when a new area of research is emerging, or when research is conducted in a different context or population. Identifying knowledge gaps can lead to the development of new research studies or the extension of existing research to fill the gap.

Examples of Research Gap

Here are some examples of research gaps that researchers might identify:

• Theoretical Gap Example : In the field of psychology, there might be a theoretical gap related to the lack of understanding of the relationship between social media use and mental health. Although there is existing research on the topic, there might be a lack of consensus on the mechanisms that link social media use to mental health outcomes.
• Empirical Gap Example : In the field of environmental science, there might be an empirical gap related to the lack of data on the long-term effects of climate change on biodiversity in specific regions. Although there might be some studies on the topic, there might be a lack of data on the long-term effects of climate change on specific species or ecosystems.
• Methodological Gap Example : In the field of education, there might be a methodological gap related to the lack of appropriate research methods to assess the impact of online learning on student outcomes. Although there might be some studies on the topic, existing research methods might not be appropriate to assess the complex relationships between online learning and student outcomes.
• Practical Gap Example: In the field of healthcare, there might be a practical gap related to the lack of effective strategies to implement evidence-based practices in clinical settings. Although there might be existing research on the effectiveness of certain practices, they might not be implemented in practice due to various barriers, such as financial constraints or lack of resources.
• Knowledge Gap Example: In the field of anthropology, there might be a knowledge gap related to the lack of understanding of the cultural practices of indigenous communities in certain regions. Although there might be some research on the topic, there might be a lack of knowledge about specific cultural practices or beliefs that are unique to those communities.

Examples of Research Gap In Literature Review, Thesis, and Research Paper might be:

• Literature review : A literature review on the topic of machine learning and healthcare might identify a research gap in the lack of studies that investigate the use of machine learning for early detection of rare diseases.
• Thesis : A thesis on the topic of cybersecurity might identify a research gap in the lack of studies that investigate the effectiveness of artificial intelligence in detecting and preventing cyber attacks.
• Research paper : A research paper on the topic of natural language processing might identify a research gap in the lack of studies that investigate the use of natural language processing techniques for sentiment analysis in non-English languages.

How to Write Research Gap

By following these steps, you can effectively write about research gaps in your paper and clearly articulate the contribution that your study will make to the existing body of knowledge.

Here are some steps to follow when writing about research gaps in your paper:

• Identify the research question : Before writing about research gaps, you need to identify your research question or problem. This will help you to understand the scope of your research and identify areas where additional research is needed.
• Review the literature: Conduct a thorough review of the literature related to your research question. This will help you to identify the current state of knowledge in the field and the gaps that exist.
• Identify the research gap: Based on your review of the literature, identify the specific research gap that your study will address. This could be a theoretical, empirical, methodological, practical, or knowledge gap.
• Provide evidence: Provide evidence to support your claim that the research gap exists. This could include a summary of the existing literature, a discussion of the limitations of previous studies, or an analysis of the current state of knowledge in the field.
• Explain the importance: Explain why it is important to fill the research gap. This could include a discussion of the potential implications of filling the gap, the significance of the research for the field, or the potential benefits to society.
• State your research objectives: State your research objectives, which should be aligned with the research gap you have identified. This will help you to clearly articulate the purpose of your study and how it will address the research gap.

Importance of Research Gap

The importance of research gaps can be summarized as follows:

• Advancing knowledge: Identifying research gaps is crucial for advancing knowledge in a particular field. By identifying areas where additional research is needed, researchers can fill gaps in the existing body of knowledge and contribute to the development of new theories and practices.
• Guiding research: Research gaps can guide researchers in designing studies that fill those gaps. By identifying research gaps, researchers can develop research questions and objectives that are aligned with the needs of the field and contribute to the development of new knowledge.
• Enhancing research quality: By identifying research gaps, researchers can avoid duplicating previous research and instead focus on developing innovative research that fills gaps in the existing body of knowledge. This can lead to more impactful research and higher-quality research outputs.
• Informing policy and practice: Research gaps can inform policy and practice by highlighting areas where additional research is needed to inform decision-making. By filling research gaps, researchers can provide evidence-based recommendations that have the potential to improve policy and practice in a particular field.

Applications of Research Gap

Here are some potential applications of research gap:

• Informing research priorities: Research gaps can help guide research funding agencies and researchers to prioritize research areas that require more attention and resources.
• Identifying practical implications: Identifying gaps in knowledge can help identify practical applications of research that are still unexplored or underdeveloped.
• Stimulating innovation: Research gaps can encourage innovation and the development of new approaches or methodologies to address unexplored areas.
• Improving policy-making: Research gaps can inform policy-making decisions by highlighting areas where more research is needed to make informed policy decisions.
• Enhancing academic discourse: Research gaps can lead to new and constructive debates and discussions within academic communities, leading to more robust and comprehensive research.

Advantages of Research Gap

Here are some of the advantages of research gap:

• Identifies new research opportunities: Identifying research gaps can help researchers identify areas that require further exploration, which can lead to new research opportunities.
• Improves the quality of research: By identifying gaps in current research, researchers can focus their efforts on addressing unanswered questions, which can improve the overall quality of research.
• Enhances the relevance of research: Research that addresses existing gaps can have significant implications for the development of theories, policies, and practices, and can therefore increase the relevance and impact of research.
• Helps avoid duplication of effort: Identifying existing research can help researchers avoid duplicating efforts, saving time and resources.
• Helps to refine research questions: Research gaps can help researchers refine their research questions, making them more focused and relevant to the needs of the field.
• Promotes collaboration: By identifying areas of research that require further investigation, researchers can collaborate with others to conduct research that addresses these gaps, which can lead to more comprehensive and impactful research outcomes.

Disadvantages of Research Gap

While research gaps can be advantageous, there are also some potential disadvantages that should be considered:

• Difficulty in identifying gaps: Identifying gaps in existing research can be challenging, particularly in fields where there is a large volume of research or where research findings are scattered across different disciplines.
• Lack of funding: Addressing research gaps may require significant resources, and researchers may struggle to secure funding for their work if it is perceived as too risky or uncertain.
• Time-consuming: Conducting research to address gaps can be time-consuming, particularly if the research involves collecting new data or developing new methods.
• Risk of oversimplification: Addressing research gaps may require researchers to simplify complex problems, which can lead to oversimplification and a failure to capture the complexity of the issues.
• Bias : Identifying research gaps can be influenced by researchers’ personal biases or perspectives, which can lead to a skewed understanding of the field.
• Potential for disagreement: Identifying research gaps can be subjective, and different researchers may have different views on what constitutes a gap in the field, leading to disagreements and debate.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

Institutional Review Board – Application Sample...

Evaluating Research – Process, Examples and...

info This is a space for the teal alert bar.

notifications This is a space for the yellow alert bar.

Research Process

• Brainstorming
• Explore Google This link opens in a new window
• Explore Web Resources
• Explore Background Information
• Explore Books
• Explore Scholarly Articles
• Narrowing a Topic
• Primary and Secondary Resources
• Academic, Popular & Trade Publications
• Scholarly and Peer-Reviewed Journals
• Grey Literature
• Clinical Trials
• Evidence Based Treatment
• Scholarly Research
• Database Research Log
• Search Limits
• Keyword Searching
• Boolean Operators
• Phrase Searching
• Truncation & Wildcard Symbols
• Proximity Searching
• Field Codes
• Subject Terms and Database Thesauri
• Reading a Scientific Article
• Website Evaluation
• Article Keywords and Subject Terms
• Cited References
• Citing Articles
• Related Results
• Search Within Publication
• Database Alerts & RSS Feeds
• Personal Database Accounts
• Persistent URLs
• Literature Gap and Future Research
• Web of Knowledge
• Annual Reviews
• Systematic Reviews & Meta-Analyses
• Finding Seminal Works
• Exhausting the Literature
• Finding Dissertations
• Researching Theoretical Frameworks
• Research Methodology & Design
• Tests and Measurements
• Organizing Research & Citations This link opens in a new window
• Scholarly Publication
• Learn the Library This link opens in a new window

Research Articles

These examples below illustrate how researchers from different disciplines identified gaps in existing literature. For additional examples, try a NavigatorSearch using this search string: ("Literature review") AND (gap*)

• Addressing the Recent Developments and Potential Gaps in the Literature of Corporate Sustainability
• Applications of Psychological Science to Teaching and Learning: Gaps in the Literature
• Attitudes, Risk Factors, and Behaviours of Gambling Among Adolescents and Young People: A Literature Review and Gap Analysis
• Do Psychological Diversity Climate, HRM Practices, and Personality Traits (Big Five) Influence Multicultural Workforce Job Satisfaction and Performance? Current Scenario, Literature Gap, and Future Research Directions
• Entrepreneurship Education: A Systematic Literature Review and Identification of an Existing Gap in the Field
• Evidence and Gaps in the Literature on HIV/STI Prevention Interventions Targeting Migrants in Receiving Countries: A Scoping Review
• Homeless Indigenous Veterans and the Current Gaps in Knowledge: The State of the Literature
• A Literature Review and Gap Analysis of Emerging Technologies and New Trends in Gambling
• A Review of Higher Education Image and Reputation Literature: Knowledge Gaps and a Research Agenda
• Trends and Gaps in Empirical Research on Open Educational Resources (OER): A Systematic Mapping of the Literature from 2015 to 2019
• Where Should We Go From Here? Identified Gaps in the Literature in Psychosocial Interventions for Youth With Autism Spectrum Disorder and Comorbid Anxiety

What is a ‘gap in the literature’?

The gap, also considered the missing piece or pieces in the research literature, is the area that has not yet been explored or is under-explored. This could be a population or sample (size, type, location, etc.), research method, data collection and/or analysis, or other research variables or conditions.

It is important to keep in mind, however, that just because you identify a gap in the research, it doesn't necessarily mean that your research question is worthy of exploration. You will want to make sure that your research will have valuable practical and/or theoretical implications. In other words, answering the research question could either improve existing practice and/or inform professional decision-making (Applied Degree), or it could revise, build upon, or create theoretical frameworks informing research design and practice (Ph.D Degree). See the Dissertation Center  for additional information about dissertation criteria at NU.

For a additional information on gap statements, see the following:

• How to Find a Gap in the Literature
• Write Like a Scientist: Gap Statements

How do you identify the gaps?

Conducting an exhaustive literature review is your first step. As you search for journal articles, you will need to read critically across the breadth of the literature to identify these gaps. You goal should be to find a ‘space’ or opening for contributing new research. The first step is gathering a broad range of research articles on your topic. You may want to look for research that approaches the topic from a variety of methods – qualitative, quantitative, or mixed methods. 

See the videos below for further instruction on identifying a gap in the literature.

Identifying a Gap in the Literature - Dr. Laurie Bedford

How Do You Identify Gaps in Literature? - SAGE Research Methods

Literature Gap & Future Research - Library Workshop

This workshop presents effective search techniques for identifying a gap in the literature and recommendations for future research.

Where can you locate research gaps?

As you begin to gather the literature, you will want to critically read for what has, and has not, been learned from the research. Use the Discussion and Future Research sections of the articles to understand what the researchers have found and where they point out future or additional research areas. This is similar to identifying a gap in the literature, however, future research statements come from a single study rather than an exhaustive search. You will want to check the literature to see if those research questions have already been answered.

Roadrunner Search

Identifying the gap in the research relies on an exhaustive review of the literature. Remember, researchers may not explicitly state that a gap in the literature exists; you may need to thoroughly review and assess the research to make that determination yourself.

However, there are techniques that you can use when searching in NavigatorSearch to help identify gaps in the literature. You may use search terms such as "literature gap " or "future research" "along with your subject keywords to pinpoint articles that include these types of statements.

Was this resource helpful?

• << Previous: Resources for a Literature Review
• Next: Web of Knowledge >>
• Last Updated: Mar 31, 2024 4:57 PM
• URL: https://resources.nu.edu/researchprocess

© Copyright 2024 National University. All Rights Reserved.

Privacy Policy | Consumer Information

How to find and fill gaps in the literature [Research Gaps Made Easy]

As we dive deeper into the realm of research, one term repeatedly echoes in the corridors of academia: “gap in literature.”

But what does it mean to find a gap in the literature, and why is it so crucial for your research project?

A gap in the literature refers to an area that hasn’t been studied or lacks substantial inquiry in your field of study. Identifying such gaps allows you to contribute fresh insights and innovation, thereby extending the existing body of knowledge.

It’s the cornerstone for every dissertation or research paper, setting the stage for an introduction that explicitly outlines the scope and aim of your investigation.

This gap review isn’t limited to what has been published in peer-reviewed journals; it may also include conference papers, dissertations, or technical reports, i.e., types of papers that provide an overview of ongoing research. 

This step is where your detective work comes in—by spotting trends, common methodologies, and unanswered questions, you can unearth an opportunity to explore an unexplored domain, thereby finding a research gap. 

Why Looking for Research Gaps is Essential

Looking for research gaps is essential as it enables the discovery of novel and unique contributions to a particular field.

By identifying these gaps, found through methods such as analyzing concluding remarks of recent papers, literature reviews, examining research groups’ non-peer-reviewed outputs, and utilizing specific search terms on Google Scholar, one can discern the trajectory of ongoing research and unearth opportunities for original inquiry.

These gaps highlight areas of potential innovation, unexplored paths, and disputed concepts, serving as the catalyst for valuable contributions and progression in the field. Hence, finding research gaps forms the basis of substantial and impactful scientific exploration.

Then your research can contribute by finding and filling the gap in knowledge. 

Method 1: Utilizing Concluding Remarks of Recent Research

When embarking on a quest to find research gaps, the concluding remarks of recent research papers can serve as an unexpected treasure map.

This section of a paper often contains insightful comments on the limitations of the work and speculates on future research directions.

These comments, although not directly pointing to a research gap, can hint at where the research is heading and what areas require further exploration.

Consider these remarks as signposts, pointing you towards uncharted territories in your field of interest.

For example, you may come across a conclusion in a recent paper on artificial intelligence that indicates a need for more research on ethical considerations. This gives you a direction to explore – the ethical implications of AI. 

However, it’s important to bear in mind that while these statements provide valuable leads, they aren’t definitive indicators of research gaps. They provide a starting point, a clue to the vast research puzzle.

Your task is to take these hints, explore further, and discern the most promising areas for your investigation. It’s a bit like being a detective, except your clues come from scholarly papers instead of crime scenes!

Method 2: Examining Research Groups and Non-peer Reviewed Outputs

If concluding remarks are signposts to potential research gaps, non-peer reviewed outputs such as preprints, conference presentations, and dissertations are detailed maps guiding you towards the frontier of research.

These resources reflect the real-time development in the field, giving you a sense of the “buzz” that surrounds hot topics.

These materials, presented but not formally published, offer a sneak peek into ongoing studies, providing you with a rich source of information to identify emerging trends and potential research gaps.

For instance, a presentation on the impact of climate change on mental health might reveal a new line of research that’s in its early stages.

One word of caution: while these resources can be enlightening, they have not undergone the rigorous peer review process that published articles have.

This means the quality of research may vary and the findings should be interpreted with a critical eye. Remember, the key is to pinpoint where the research is heading and then carve out your niche within that sphere.

Exploring non-peer reviewed outputs allows you to stay ahead of the curve, harnessing the opportunity to investigate and contribute to a burgeoning area of study before it becomes mainstream.

Method 3: Searching for ‘Promising’ and ‘Preliminary’ Results on Google Scholar

With a plethora of research at your fingertips, Google Scholar can serve as a remarkable tool in your quest to discover research gaps. The magic lies in a simple trick: search for the phrases “promising results” or “preliminary results” within your research area. Why these specific phrases? Scientists often use them when they have encouraging but not yet fully verified findings.

To illustrate, consider an example. Type “promising results and solar cell” into Google Scholar, and filter by recent publications.

The search results will show you recent studies where researchers have achieved promising outcomes but may not have fully developed their ideas or resolved all challenges.

These “promising” or “preliminary” results often represent areas ripe for further exploration.

They hint at a research question that has been opened but not fully answered. However, tread carefully.

While these findings can indeed point to potential research gaps, they can also lead to dead ends. It’s crucial to examine these leads with a critical eye and further corroborate them with a comprehensive review of related research.

Nevertheless, this approach provides a simple, effective starting point for identifying research gaps, serving as a launchpad for your explorations.

Method 4: Reading Around the Subject

Comprehensive reading forms the bedrock of effective research. When hunting for research gaps, you need to move beyond just the preliminary findings and delve deeper into the context surrounding these results.

This involves broadening your view and reading extensively around your topic of interest.

In the course of your reading, you will start identifying common themes, reoccurring questions, and shared challenges in the research.

Over time, patterns will emerge, helping you recognize areas where research is thin or missing.

For instance, in studying autonomous vehicles, you might find recurring questions about regulatory frameworks, pointing to a potential gap in the legal aspects of this technology.

However, this method is not about scanning through a huge volume of literature aimlessly. It involves strategic and critical reading, looking for patterns, inconsistencies, and areas where the existing literature falls short.

It’s akin to painting a picture where some parts are vividly detailed while others remain sketchy. Your goal is to identify these sketchy areas and fill in the details.

So grab your academic reading list, and start diving into the ocean of knowledge. Remember, it’s not just about the depth, but also the breadth of your reading, that will lead you to a meaningful research gap.

Method 5: Consulting with Current Researchers

Few methods are as effective in uncovering research gaps as engaging in conversations with active researchers in your field of interest.

Current researchers, whether they are PhD students, postdoctoral researchers, or supervisors, are often deeply engaged in ongoing studies and understand the current challenges in their respective fields.

Start by expressing genuine interest in their work. Rather than directly asking for research gaps, inquire about the challenges they are currently facing in their projects.

You can ask, “What are the current challenges in your research?”

Their responses can highlight potential areas of exploration, setting you on the path to identifying meaningful research gaps.

Moreover, supervisors, particularly those overseeing PhD and Master’s students, often have ideas for potential research topics. By asking the right questions, you can tap into their wealth of knowledge and identify fruitful areas of study.

While the act of discovering research gaps can feel like a solitary journey, it doesn’t have to be.

Engaging with others who are grappling with similar challenges can provide valuable insights and guide your path. After all, the world of research thrives on collaboration and shared intellectual curiosity.

Method 6: Utilizing Online Tools

The digital age has made uncovering research gaps easier, thanks to a plethora of online tools that help visualize the interconnectedness of research literature.

Platforms such as:

• Connected Papers,
• ResearchRabbit, and

allow you to see how different papers in your field relate to one another, thereby creating a web of knowledge.

Upon creating this visual web, you may notice that many papers point towards a certain area, but then abruptly stop. This could indicate a potential research gap, suggesting that the topic hasn’t been adequately addressed or has been sidelined for some reason.

By further reading around this apparent gap, you can understand if it’s a genuine knowledge deficit or merely a research path that was abandoned due to inherent challenges or a dead end.

These online tools provide a bird’s eye view of the literature, helping you understand the broader landscape of research in your area of interest.

By examining patterns and relationships among studies, you can effectively zero in on unexplored areas, making these tools a valuable asset in your quest for research gaps.

Method 7: Seeking Conflicting Ideas in the Literature

In scientific research, areas of conflict can often be fertile ground for finding research gaps. These are areas where there’s a considerable amount of disagreement or ongoing debate among researchers.

If you can bring a fresh perspective, a new technique, or a novel hypothesis to such a contentious issue, you may well be on your way to uncovering a significant research gap.

Take, for instance, an area in psychology where there is a heated debate about the influence of nature versus nurture.

If you can introduce a new dimension to the debate or a method to test a novel hypothesis, you could potentially fill a significant gap in the literature.

Investigating areas of conflict not only opens avenues for exploring research gaps, but it also provides opportunities for you to make substantial contributions to your field. The key is to be able to see the potential for a new angle and to muster the courage to dive into contentious waters.

However, engaging with conflicts in research requires careful navigation.

Striking the right balance between acknowledging existing research and championing new ideas is crucial.

In the end, resolving these conflicts or adding significant depth to the debate can be incredibly rewarding and contribute greatly to your field.

The Right Perspective Towards Research Gaps

The traditional understanding of research gaps often involves seeking out a ‘bubble’ of missing knowledge in the sea of existing research, a niche yet to be explored. However, in today’s fast-paced research environment, these bubbles are becoming increasingly rare.

The paradigm of finding research gaps is shifting. It’s no longer just about seeking out holes in existing knowledge, but about understanding the leading edge of research and the directions it could take. It involves not just filling in the gaps but extending the boundaries of knowledge.

To identify such opportunities, develop a comprehensive understanding of the research landscape, identify emerging trends, and keep a close eye on recent advancements.

Look for the tendrils of knowledge extending out into the unknown and think about how you can push them further. It might be a challenging task, but it offers the potential for making substantial, impactful contributions to your field. 

Remember, every great innovation begins at the edge of what is known. That’s where your research gap might be hiding.

Wrapping up – Literature and research gaps

Finding and filling a gap in the literature is a task crucial to every research project. It begins with a systematic review of existing literature – a quest to identify what has been studied and more importantly, what hasn’t.

You must delve into the rich terrain of literature in their field, from the seminal, citation-heavy research articles to the fresh perspective of conference papers. Identifying the gap in the literature necessitates a thorough evaluation of existing studies to refine your area of interest and map the scope and aim of your future research.

The purpose is to explicitly identify the gap that exists, so you can contribute to the body of knowledge by providing fresh insights. The process involves a series of steps, from consulting with faculty and experts in the field to identify potential trends and outdated methodologies, to being methodological in your approach to identify gaps that have emerged.

Upon finding a gap in the literature, we’ll ideally have a clearer picture of the research need and an opportunity to explore this unexplored domain.

It is important to remember that the task does not end with identifying the gap. The real challenge lies in drafting a research proposal that’s objective, answerable, and can quantify the impact of filling this gap. 

It’s important to consult with your advisor, and also look at commonly used parameters and preliminary evidence. Only then can we complete the task of turning an identified gap in the literature into a valuable contribution to your field, a contribution that’s peer-reviewed and adds to the body of knowledge.

To find a research gap is to stand on the shoulders of giants, looking beyond the existing research to further expand our understanding of the world.

Dr Andrew Stapleton has a Masters and PhD in Chemistry from the UK and Australia. He has many years of research experience and has worked as a Postdoctoral Fellow and Associate at a number of Universities. Although having secured funding for his own research, he left academia to help others with his YouTube channel all about the inner workings of academia and how to make it work for you.

Thank you for visiting Academia Insider.

We are here to help you navigate Academia as painlessly as possible. We are supported by our readers and by visiting you are helping us earn a small amount through ads and affiliate revenue - Thank you!

2024 © Academia Insider

Identifying Research Gaps to Pursue Innovative Research

This article is an excerpt from a lecture given by my Ph.D. guide, a researcher in public health. She advised us on how to identify research gaps to pursue innovative research in our fields.

What is a Research Gap?

Today we are talking about the research gap: what is it, how to identify it, and how to make use of it so that you can pursue innovative research. Now, how many of you have ever felt you had discovered a new and exciting research question , only to find that it had already been written about? I have experienced this more times than I can count. Graduate studies come with pressure to add new knowledge to the field. We can contribute to the progress and knowledge of humanity. To do this, we need to first learn to identify research gaps in the existing literature.

A research gap is, simply, a topic or area for which missing or insufficient information limits the ability to reach a conclusion for a question. It should not be confused with a research question, however. For example, if we ask the research question of what the healthiest diet for humans is, we would find many studies and possible answers to this question. On the other hand, if we were to ask the research question of what are the effects of antidepressants on pregnant women, we would not find much-existing data. This is a research gap. When we identify a research gap, we identify a direction for potentially new and exciting research.

How to Identify Research Gap?

Considering the volume of existing research, identifying research gaps can seem overwhelming or even impossible. I don’t have time to read every paper published on public health. Similarly, you guys don’t have time to read every paper. So how can you identify a research gap?

There are different techniques in various disciplines, but we can reduce most of them down to a few steps, which are:

• Identify your key motivating issue/question
• Identify key terms associated with this issue
• Review the literature, searching for these key terms and identifying relevant publications
• Review the literature cited by the key publications which you located in the above step
• Identify issues not addressed by  the literature relating to your critical  motivating issue

It is the last step which we all find the most challenging. It can be difficult to figure out what an article is  not  saying. I like to keep a list of notes of biased or inconsistent information. You could also track what authors write as “directions for future research,” which often can point us towards the existing gaps.

Different Types of Research Gaps

Identifying research gaps is an essential step in conducting research, as it helps researchers to refine their research questions and to focus their research efforts on areas where there is a need for more knowledge or understanding.

1. Knowledge gaps

These are gaps in knowledge or understanding of a subject, where more research is needed to fill the gaps. For example, there may be a lack of understanding of the mechanisms behind a particular disease or how a specific technology works.

2. Conceptual gaps

These are gaps in the conceptual framework or theoretical understanding of a subject. For example, there may be a need for more research to understand the relationship between two concepts or to refine a theoretical framework.

3. Methodological gaps

These are gaps in the methods used to study a particular subject. For example, there may be a need for more research to develop new research methods or to refine existing methods to address specific research questions.

4. Data gaps

These are gaps in the data available on a particular subject. For example, there may be a need for more research to collect data on a specific population or to develop new measures to collect data on a particular construct.

5. Practical gaps

These are gaps in the application of research findings to practical situations. For example, there may be a need for more research to understand how to implement evidence-based practices in real-world settings or to identify barriers to implementing such practices.

Examples of Research Gap

Limited understanding of the underlying mechanisms of a disease:.

Despite significant research on a particular disease, there may be a lack of understanding of the underlying mechanisms of the disease. For example, although much research has been done on Alzheimer’s disease, the exact mechanisms that lead to the disease are not yet fully understood.

Inconsistencies in the findings of previous research:

When previous research on a particular topic has inconsistent findings, there may be a need for further research to clarify or resolve these inconsistencies. For example, previous research on the effectiveness of a particular treatment for a medical condition may have produced inconsistent findings, indicating a need for further research to determine the true effectiveness of the treatment.

Limited research on emerging technologies:

As new technologies emerge, there may be limited research on their applications, benefits, and potential drawbacks. For example, with the increasing use of artificial intelligence in various industries, there is a need for further research on the ethical, legal, and social implications of AI.

How to Deal with Literature Gap?

Once you have identified the literature gaps, it is critical to prioritize. You may find many questions which remain to be answered in the literature. Often one question must be answered before the next can be addressed. In prioritizing the gaps, you have identified, you should consider your funding agency or stakeholders, the needs of the field, and the relevance of your questions to what is currently being studied. Also, consider your own resources and ability to conduct the research you’re considering. Once you have done this, you can narrow your search down to an appropriate question.

Tools to Help Your Search

There are thousands of new articles published every day, and staying up to date on the literature can be overwhelming. You should take advantage of the technology that is available. Some services include  PubCrawler ,  Feedly ,  Google Scholar , and PubMed updates. Stay up to date on social media forums where scholars share new discoveries, such as Twitter. Reference managers such as  Mendeley  can help you keep your references well-organized. I personally have had success using Google Scholar and PubMed to stay current on new developments and track which gaps remain in my personal areas of interest.

The most important thing I want to impress upon you today is that you will struggle to  choose a research topic  that is innovative and exciting if you don’t know the existing literature well. This is why identifying research gaps starts with an extensive and thorough  literature review . But give yourself some boundaries.  You don’t need to read every paper that has ever been written on a topic. You may find yourself thinking you’re on the right track and then suddenly coming across a paper that you had intended to write! It happens to everyone- it happens to me quite often. Don’t give up- keep reading and you’ll find what you’re looking for.

Class dismissed!

How do you identify research gaps? Share your thoughts in the comments section below.

Frequently Asked Questions

A research gap can be identified by looking for a topic or area with missing or insufficient information that limits the ability to reach a conclusion for a question.

Identifying a research gap is important as it provides a direction for potentially new research or helps bridge the gap in existing literature.

Gap in research is a topic or area with missing or insufficient information. A research gap limits the ability to reach a conclusion for a question.

Thank u for your suggestion.

Very useful tips specially for a beginner

Thank you. This is helpful. I find that I’m overwhelmed with literatures. As I read on a particular topic, and in a particular direction I find that other conflicting issues, topic a and ideas keep popping up, making me more confused.

I am very grateful for your advice. It’s just on point.

The clearest, exhaustive, and brief explanation I have ever read.

Thanks for sharing

Thank you very much.The work is brief and understandable

Thank you it is very informative

Thanks for sharing this educative article

Thank you for such informative explanation.

Great job smart guy! Really outdid yourself!

Nice one! I thank you for this as it is just what I was looking for!😃🤟

Thank you so much for this. Much appreciated

Thank you so much.

Thankyou for ur briefing…its so helpful

Thank you so much .I’ved learn a lot from this.❤️

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Reporting Research

Choosing the Right Analytical Approach: Thematic analysis vs. content analysis for data interpretation

In research, choosing the right approach to understand data is crucial for deriving meaningful insights.…

Comparing Cross Sectional and Longitudinal Studies: 5 steps for choosing the right approach

The process of choosing the right research design can put ourselves at the crossroads of…

• Career Corner

Unlocking the Power of Networking in Academic Conferences

Embarking on your first academic conference experience? Fear not, we got you covered! Academic conferences…

Research Recommendations – Guiding policy-makers for evidence-based decision making

Research recommendations play a crucial role in guiding scholars and researchers toward fruitful avenues of…

• AI in Academia

Disclosing the Use of Generative AI: Best practices for authors in manuscript preparation

The rapid proliferation of generative and other AI-based tools in research writing has ignited an…

Intersectionality in Academia: Dealing with diverse perspectives

Meritocracy and Diversity in Science: Increasing inclusivity in STEM education

Avoiding the AI Trap: Pitfalls of relying on ChatGPT for PhD applications

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What should universities' stance be on AI tools in research and academic writing?

UNE Library Services

Research Help

Gaps in the literature.

Gaps in the literature are missing pieces or insufficient information in the published research on a topic. These are areas that have opportunities for further research because they are unexplored, under-explored, or outdated. 

Finding Gaps

Gaps can be missing or incomplete:

• Population or sample: size, type, location etc…
• Research methods: qualitative, quantitative, or mixed
• Data collection or analysis
• Research variables or conditions

Conduct a thorough literature search to find a broad range of research articles on your topic. Search research databases ;  you can find recommended databases for your subject area in  research by subject  for your course or program.

Identifying Gaps

If you do not find articles in your literature search, this may indicate a gap.

If you do find articles, the goal is to find a gap for contributing new research. Authors signal that there is a gap using words such as:

• Has not been clarified, studied, reported, or elucidated
• Further research is required or needed
• Is not well reported
• Suggestions for further research
• Key question is or remains
• It is important to address
• Poorly understood or known
• Lack of studies
• These findings provide valuable insights into the potential benefits of mindfulness-based interventions for stress management,however, further study is needed to address several limitations and extend our understanding in this area .
• While this study provides preliminary evidence of the potential efficacy of VRET in reducing PTSD symptoms, several aspects related to its implementation and specific treatment outcomes  remain inadequately clarified, highlighting the need for further research .
• Although the studies reviewed provide valuable insights into the potential effects of climate change on species composition and ecosystem functioning.  The question of how climate change will interact with other anthropogenic stressors to influence the resilience and adaptive capacity of tropical rainforest ecosystems remains unanswered, highlighting the need for further research .

Questions & Help

If you have questions on this, or another, topic, contact a librarian for help!

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Link to facebook
• Link to linkedin
• Link to twitter
• Link to youtube
• Writing Tips

How to Identify a Research Gap

• 5-minute read
• 10th January 2024

If you’ve been tasked with producing a thesis or dissertation, one of your first steps will be identifying a research gap. Although finding a research gap may sound daunting, don’t fret! In this post, we will define a research gap, discuss its importance, and offer a step-by-step guide that will provide you with the essential know-how to complete this critical step and move on to the rest of your research project.

What Is a Research Gap?

Simply put, a research gap is an area that hasn’t been explored in the existing literature. This could be an unexplored population, an untested method, or a condition that hasn’t been investigated yet. 

Why Is Identifying a Research Gap Important?

Identifying a research gap is a foundational step in the research process. It ensures that your research is significant and has the ability to advance knowledge within a specific area. It also helps you align your work with the current needs and challenges of your field. Identifying a research gap has many potential benefits.

1. Avoid Redundancy in Your Research

Understanding the existing literature helps researchers avoid duplication. This means you can steer clear of topics that have already been extensively studied. This ensures your work is novel and contributes something new to the field.

2. Guide the Research Design

Identifying a research gap helps shape your research design and questions. You can tailor your studies to specifically address the identified gap. This ensures that your work directly contributes to filling the void in knowledge.

3. Practical Applications

Research that addresses a gap is more likely to have practical applications and contributions. Whether in academia, industry, or policymaking, research that fills a gap in knowledge is often more applicable and can inform decision-making and practices in real-world contexts.

4. Field Advancements

Addressing a research gap can lead to advancements in the field . It may result in the development of new theories, methodologies, or technologies that push the boundaries of current understanding.

5. Strategic Research Planning

Identifying a research gap is crucial for strategic planning . It helps researchers and institutions prioritize areas that need attention so they can allocate resources effectively. This ensures that efforts are directed toward the most critical gaps in knowledge.

6. Academic and Professional Recognition

Researchers who successfully address significant research gaps often receive peer recognition within their academic and professional communities. This recognition can lead to opportunities for collaboration, funding, and career advancement.

How Do I Identify a Research Gap?

1. clearly define your research topic .

Begin by clearly defining your research topic. A well-scoped topic serves as the foundation for your studies. Make sure it’s not too broad or too narrow; striking the right balance will make it easier to identify gaps in existing literature.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

2. Conduct a Thorough Literature Review

A comprehensive literature review is a vital step in any research. Dive deep into the existing research related to your topic. Look for patterns, recurring themes, and consensus among scholars. Pay attention to areas where conflicting opinions or gaps in understanding emerge.

3. Evaluate Existing Studies

Critically evaluate the studies you encounter during your literature review. Assess the paradigms , methodologies, findings, and limitations of each. Note any discrepancies, unanswered questions, or areas where further investigation is warranted. These are potential indicators of research gaps.

4. Identify Unexplored Perspectives

Consider the perspectives presented in the existing literature. Are there alternative viewpoints or marginalized voices that haven’t been adequately explored? Identifying and incorporating diverse perspectives can often lead to uncharted territory and help you pinpoint a unique research gap.

Additional Tips

Stay up to date with emerging trends.

The field of research is dynamic, with new developments and emerging trends constantly shaping the landscape. Stay up to date with the latest publications, conferences, and discussions in your field and make sure to regularly check relevant academic search engines . Often, identifying a research gap involves being at the forefront of current debates and discussions.

Seek Guidance From Experts

Don’t hesitate to reach out to experts in your field for guidance. Attend conferences, workshops, or seminars where you can interact with seasoned researchers. Their insights and experience can provide valuable perspectives on potential research gaps that you may have overlooked. You can also seek advice from your academic advisor .

Use Research Tools and Analytics

Leverage tech tools to analyze patterns and trends in the existing literature. Tools like citation analysis, keyword mapping, and data visualization can help you identify gaps and areas with limited exploration.

Identifying a research gap is a skill that evolves with experience and dedication. By defining your research topic, meticulously navigating the existing literature, critically evaluating studies, and recognizing unexplored perspectives, you’ll be on your way to identifying a research gap that will serve as the foundation for your paper, thesis, or dissertation topic .

If you need any help with proofreading your research paper , we can help with our research paper editing services . You can even try a sample of our services for free . Good luck with all your research!

Share this article:

Post A New Comment

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

3-minute read

What Is a Content Editor?

Are you interested in learning more about the role of a content editor and the...

4-minute read

The Benefits of Using an Online Proofreading Service

Proofreading is important to ensure your writing is clear and concise for your readers. Whether...

2-minute read

6 Online AI Presentation Maker Tools

Creating presentations can be time-consuming and frustrating. Trying to construct a visually appealing and informative...

What Is Market Research?

No matter your industry, conducting market research helps you keep up to date with shifting...

8 Press Release Distribution Services for Your Business

In a world where you need to stand out, press releases are key to being...

How to Get a Patent

In the United States, the US Patent and Trademarks Office issues patents. In the United...

Make sure your writing is the best it can be with our expert English proofreading and editing.

• Research Process

What is a Research Gap

• 3 minute read
• 282.6K views

Table of Contents

If you are a young researcher, or even still finishing your studies, you’ll probably notice that your academic environment revolves around certain research topics, probably linked to your department or to the interest of your mentor and direct colleagues. For example, if your department is currently doing research in nanotechnology applied to medicine, it is only natural that you feel compelled to follow this line of research. Hopefully, it’s something you feel familiar with and interested in – although you might take your own twists and turns along your career.

Many scientists end up continuing their academic legacy during their professional careers, writing about their own practical experiences in the field and adapting classic methodologies to a present context. However, each and every researcher dreams about being a pioneer in a subject one day, by discovering a topic that hasn’t been approached before by any other scientist. This is a research gap.

Research gaps are particularly useful for the advance of science, in general. Finding a research gap and having the means to develop a complete and sustained study on it can be very rewarding for the scientist (or team of scientists), not to mention how its new findings can positively impact our whole society.

How to Find a Gap in Research

How many times have you felt that you have finally formulated THAT new and exciting question, only to find out later that it had been addressed before? Probably more times than you can count.

There are some steps you can take to help identify research gaps, since it is impossible to go through all the information and research available nowadays:

• Select a topic or question that motivates you: Research can take a long time and surely a large amount of physical, intellectual and emotional effort, therefore choose a topic that can keep you motivated throughout the process.
• Find keywords and related terms to your selected topic: Besides synthesizing the topic to its essential core, this will help you in the next step.
• Use the identified keywords to search literature: From your findings in the above step, identify relevant publications and cited literature in those publications.
• Look for topics or issues that are missing or not addressed within (or related to) your main topic.
• Read systematic reviews: These documents plunge deeply into scholarly literature and identify trends and paradigm shifts in fields of study. Sometimes they reveal areas or topics that need more attention from researchers and scientists.

Keeping track of all the new literature being published every day is an impossible mission. Remember that there is technology to make your daily tasks easier, and reviewing literature can be one of them. Some online databases offer up-to-date publication lists with quite effective search features:

• Elsevier’s Scope
• Google Scholar

Of course, these tools may be more or less effective depending on knowledge fields. There might be even better ones for your specific topic of research; you can learn about them from more experienced colleagues or mentors.

Find out how FINER research framework can help you formulate your research question.

Literature Gap

The expression “literature gap” is used with the same intention as “research gap.” When there is a gap in the research itself, there will also naturally be a gap in the literature. Nevertheless, it is important to stress out the importance of language or text formulations that can help identify a research/literature gap or, on the other hand, making clear that a research gap is being addressed.

When looking for research gaps across publications you may have noticed sentences like:

…has/have not been… (studied/reported/elucidated) …is required/needed… …the key question is/remains… …it is important to address…

These expressions often indicate gaps; issues or topics related to the main question that still hasn’t been subject to a scientific study. Therefore, it is important to take notice of them: who knows if one of these sentences is hiding your way to fame.

Language Editing Services by Elsevier Author Services:

• Manuscript Review

Systematic Review VS Meta-Analysis

Literature Review in Research Writing

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

• Library databases
• Library website

Library Guide to Capstone Literature Reviews: Find a Research Gap

Find a research gap: tips to get started.

Finding a research gap is not an easy process and there is no one linear path. These tips and suggestions are just examples of possible ways to begin. 

In Ph.D. dissertations, students identify a gap in research. In other programs, students identify a gap in practice. The literature review for a gap in practice will show the context of the problem and the current state of the research. 

Research gap definition

A research gap exists when:

• a question or problem has not been answered by existing studies/research in the field 
• a concept or new idea has not been studied at all
• all the existing literature on a topic is outdated 
• a specific population/location/age group etc has not been studied 

A research gap should be:

• grounded in the literature
• amenable to scientific study
• Litmus Test for a Doctoral-Level Research Problem (Word) This tool helps students determine if they have identified a doctoral level research problem.

Identify a research gap

To find a gap you must become very familiar with a particular field of study. This will involve a lot of research and reading, because a gap is defined by what does (and does not) surround it.

• Search the research literature and dissertations (search all university dissertations, not just Walden!).
• Understand your topic! Review background information in books and encyclopedias . 
• Look for literature reviews, systematic reviews, and meta-analyses.
• Take notes on concepts, themes, and subject terms . 
• Look closely at each article's limitations, conclusions, and recommendations for future research. 
• Organize, analyze, and repeat! 

• Quick Answer: How do I find dissertations on a topic?

Start with broad searches

Use the Library Search (formerly Thoreau)  to do a broad search with just one concept at a time . Broad searches give you an idea of the academic conversation surrounding your topic.

• Try the terms you know (keywords) first.
• Look at the Subject Terms (controlled language) to brainstorm terms. 
• Subject terms help you understand what terms are most used, and what other terms to try.
• No matter what your topic is, not every researcher will be using the same terms. Keep an eye open for additional ways to describe your topic.
• Guide: Subject Terms & Index Searches: Index Overview

Keep a list of terms

• Create a list of terms
• Example list of terms

This list will be a record of what terms are: 

• related to or represent your topic
• synonyms or antonyms
• more or less commonly used
• keywords (natural language) or subject terms (controlled language)
• Synonyms & antonyms (database search skills)
• Turn keywords into subject terms

Term I started with:

culturally aware 

Subject terms I discovered:

cultural awareness (SU) 

cultural sensitivity (SU) 

cultural competence (SU) 

Search with different combinations of terms

• Combine search terms list
• Combine search terms table
• Video: Search by Themes

Since a research gap is defined by the absence of research on a topic, you will search for articles on everything that relates to your topic. 

• List out all the themes related to your gap.
• Search different combinations of the themes as you discover them (include search by theme video at bottom) 

For example, suppose your research gap is on the work-life balance of tenured and tenure-track women in engineering professions. In that case, you might try searching different combinations of concepts, such as: 

• women and STEM 
• STEM or science or technology or engineering or mathematics
• female engineering professors 
• tenure-track women in STEM
• work-life balance and women in STEM
• work-life balance and women professors
• work-life balance and tenure 

Topic adapted from one of the award winning Walden dissertations. 

• Walden University Award Winning Dissertations
• Gossage, Lily Giang-Tien, "Work-Life Balance of Tenured and Tenure-Track Women Engineering Professors" (2019). Walden Dissertations and Doctoral Studies. 6435.

Break your topic into themes and try combining the terms from different themes in different ways. For example: 

Theme 1 and Theme 4

Theme 2 and Theme 1

Theme 3 and Theme 4

Video: Search by Themes (YouTube)

(2 min 40 sec) Recorded April 2014 Transcript

Track where more research is needed

Most research articles will identify where more research is needed. To identify research trends, use the literature review matrix to track where further research is needed. 

• Download or create your own Literature Review Matrix (examples in links below).
• Do some general database searches on broad topics.
• Find an article that looks interesting.
• When you read the article, pay attention to the conclusions and limitations sections.
• Use the Literature Review Matrix to track where  'more research is needed' or 'further research needed'. NOTE:  you might need to add a column to the template.
• As you fill in the matrix you should see trends where more research is needed.

There is no consistent section in research articles where the authors identify where more research is needed. Pay attention to these sections: 

• limitations
• conclusions
• recommendations for future research 
• Literature Review Matrix Templates: learn how to keep a record of what you have read
• Literature Review Matrix (Excel) with color coding Sample template for organizing and synthesizing your research
• Previous Page: Scope
• Next Page: Get & Stay Organized
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

•   [email protected]
• Shapiro Library
• SNHU Library Frequently Asked Questions

FAQ: What is a research gap and how do I find one?

• 7 Academic Integrity & Plagiarism
• 64 Academic Support, Writing Help, & Presentation Help
• 27 Access/Remote Access
• 7 Accessibility
• 9 Building/Facilities
• 7 Career/Job Information
• 26 Catalog/Print Books
• 26 Circulation
• 129 Citing Sources
• 14 Copyright
• 311 Databases
• 24 Directions/Location
• 18 Faculty Resources/Needs
• 7 Hours/Contacts
• 2 Innovation Lab & Makerspace/3D Printing
• 25 Interlibrary Loan
• 43 IT/Computer/Printing Support
• 3 Library Instruction
• 37 Library Technology Help
• 6 Multimedia
• 17 Online Programs
• 19 Periodicals
• 25 Policies
• 8 RefWorks/Citation Managers
• 4 Research Guides (LibGuides)
• 216 Research Help
• 23 University Services

Last Updated: Jun 27, 2023 Views: 463962

What is a research gap.

A research gap is a question or a problem that has not been answered by any of the existing studies or research within your field. Sometimes, a research gap exists when there is a concept or new idea that hasn't been studied at all. Sometimes you'll find a research gap if all the existing research is outdated and in need of new/updated research (studies on Internet use in 2001, for example). Or, perhaps a specific population has not been well studied (perhaps there are plenty of studies on teenagers and video games, but not enough studies on toddlers and video games, for example). These are just a few examples, but any research gap you find is an area where more studies and more research need to be conducted. Please view this video clip from our Sage Research Methods database for more helpful information: How Do You Identify Gaps in Literature?

How do I find one?

It will take a lot of research and reading.  You'll need to be very familiar with all the studies that have already been done, and what those studies contributed to the overall body of knowledge about that topic. Make a list of any questions you have about your topic and then do some research to see if those questions have already been answered satisfactorily. If they haven't, perhaps you've discovered a gap!  Here are some strategies you can use to make the most of your time:

• One useful trick is to look at the “suggestions for future research” or conclusion section of existing studies on your topic. Many times, the authors will identify areas where they think a research gap exists, and what studies they think need to be done in the future.
• As you are researching, you will most likely come across citations for seminal works in your research field. These are the research studies that you see mentioned again and again in the literature.  In addition to finding those and reading them, you can use a database like Web of Science to follow the research trail and discover all the other articles that have cited these. See the FAQ: I found the perfect article for my paper. How do I find other articles and books that have cited it? on how to do this. One way to quickly track down these seminal works is to use a database like SAGE Navigator, a social sciences literature review tool. It is one of the products available via our SAGE Knowledge database.
• In the PsycINFO and PsycARTICLES databases, you can select literature review, systematic review, and meta analysis under the Methodology section in the advanced search to quickly locate these. See the FAQ: Where can I find a qualitative or quantitative study? for more information on how to find the Methodology section in these two databases.
• In CINAHL , you can select Systematic review under the Publication Type field in the advanced search. 
• In Web of Science , check the box beside Review under the Document Type heading in the “Refine Results” sidebar to the right of the list of search hits.
• If the database you are searching does not offer a way to filter your results by document type, publication type, or methodology in the advanced search, you can include these phrases (“literature reviews,” meta-analyses, or “systematic reviews”) in your search string.  For example, “video games” AND “literature reviews” could be a possible search that you could try.

Please give these suggestions a try and contact a librarian for additional assistance.

Content authored by: GS

• Share on Facebook

Was this helpful? Yes 374 No 152

Frequently Asked Questions (FAQs) are a self-serve option for users to search and find answers to their questions. 

Use the search box above to type your question to search for an answer or browse existing FAQs by group, topic, etc.

Tell Me More

Link to Question Form

More assistance.

Submit a Question

Related FAQs

Advertisement

Toward a framework for selecting indicators of measuring sustainability and circular economy in the agri-food sector: a systematic literature review

• LIFE CYCLE SUSTAINABILITY ASSESSMENT
• Published: 02 March 2022

Cite this article

• Cecilia Silvestri   ORCID: orcid.org/0000-0003-2528-601X 1 ,
• Luca Silvestri   ORCID: orcid.org/0000-0002-6754-899X 2 ,
• Michela Piccarozzi   ORCID: orcid.org/0000-0001-9717-9462 1 &
• Alessandro Ruggieri 1  

2853 Accesses

11 Citations

9 Altmetric

Explore all metrics

A Correction to this article was published on 24 March 2022

This article has been updated

The implementation of sustainability and circular economy (CE) models in agri-food production can promote resource efficiency, reduce environmental burdens, and ensure improved and socially responsible systems. In this context, indicators for the measurement of sustainability play a crucial role. Indicators can measure CE strategies aimed to preserve functions, products, components, materials, or embodied energy. Although there is broad literature describing sustainability and CE indicators, no study offers such a comprehensive framework of indicators for measuring sustainability and CE in the agri-food sector.

Starting from this central research gap, a systematic literature review has been developed to measure the sustainability in the agri-food sector and, based on these findings, to understand how indicators are used and for which specific purposes.

The analysis of the results allowed us to classify the sample of articles in three main clusters (“Assessment-LCA,” “Best practice,” and “Decision-making”) and has shown increasing attention to the three pillars of sustainability (triple bottom line). In this context, an integrated approach of indicators (environmental, social, and economic) offers the best solution to ensure an easier transition to sustainability.

Conclusions

The sample analysis facilitated the identification of new categories of impact that deserve attention, such as the cooperation among stakeholders in the supply chain and eco-innovation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Source: Authors’ elaboration. Notes: The graph shows the temporal distribution of the articles under analysis

Source: Authors’ elaborations. Notes: The graph shows the time distribution of articles from the three major journals

Source: Authors’ elaboration. Notes: The graph shows the composition of the sample according to the three clusters identified by the analysis

Source: Authors’ elaboration. Notes: The graph shows the distribution of articles over time by cluster

Source: Authors’ elaboration. Notes: The graph shows the network visualization

Source: Authors’ elaboration. Notes: The graph shows the overlay visualization

Source: Authors’ elaboration. Notes: The graph shows the classification of articles by scientific field

Source: Authors’ elaboration. Notes: Article classification based on their cluster to which they belong and scientific field

Source: Authors’ elaboration

Source: Authors’ elaboration. Notes: The graph shows the distribution of items over time based on TBL

Source: Authors’ elaboration. Notes: The graph shows the Pareto diagram highlighting the most used indicators in literature for measuring sustainability in the agri-food sector

Source: Authors’ elaboration. Notes: The graph shows the distribution over time of articles divided into conceptual and empirical

Source: Authors’ elaboration. Notes: The graph shows the classification of articles, divided into conceptual and empirical, in-depth analysis

Source: Authors’ elaboration. Notes: The graph shows the geographical distribution of the authors

Source: Authors’ elaboration. Notes: The graph shows the distribution of authors according to the continent from which they originate

Source: Authors’ elaboration. Notes: The graph shows the time distribution of publication of authors according to the continent from which they originate

Source: Authors’ elaboration. Notes: Sustainability measurement indicators and impact categories of LCA, S-LCA, and LCC tools should be integrated in order to provide stakeholders with best practices as guidelines and tools to support both decision-making and measurement, according to the circular economy approach

Similar content being viewed by others

The socio-economic performance of agroecology. A review

Ioanna Mouratiadou, Alexander Wezel, … Paolo Bàrberi

The Circular Economy: An Interdisciplinary Exploration of the Concept and Application in a Global Context

Alan Murray, Keith Skene & Kathryn Haynes

The sustainability of “local” food: a review for policy-makers

Alexander J. Stein & Fabien Santini

Change history

24 march 2022.

A Correction to this paper has been published: https://doi.org/10.1007/s11367-022-02038-9

Acero AP, Rodriguez C, Ciroth A (2017) LCIA methods: impact assessment methods in life cycle assessment and their impact categories. Version 1.5.6. Green Delta 1–23

Accorsi R, Versari L, Manzini R (2015) Glass vs. plastic: Life cycle assessment of extra-virgin olive oil bottles across global supply chains. Sustain 7:2818–2840. https://doi.org/10.3390/su7032818

Adjei-Bamfo P, Maloreh-Nyamekye T, Ahenkan A (2019) The role of e-government in sustainable public procurement in developing countries: a systematic literature review. Resour Conserv Recycl 142:189–203. https://doi.org/10.1016/j.resconrec.2018.12.001

Article   Google Scholar  

Aivazidou E, Tsolakis N, Vlachos D, Iakovou E (2015) Water footprint management policies for agrifood supply chains: a critical taxonomy and a system dynamics modelling approach. Chem Eng Trans 43:115–120. https://doi.org/10.3303/CET1543020

Alhaddi H (2015) Triple bottom line and sustainability: a literature review. Bus Manag Stud 1:6–10

Allaoui H, Guo Y, Sarkis J (2019) Decision support for collaboration planning in sustainable supply chains. J Clean Prod 229:761–774. https://doi.org/10.1016/j.jclepro.2019.04.367

Alshqaqeeq F, Amin Esmaeili M, Overcash M, Twomey J (2020) Quantifying hospital services by carbon footprint: a systematic literature review of patient care alternatives. Resour Conserv Recycl 154:104560. https://doi.org/10.1016/j.resconrec.2019.104560

Anwar F, Chaudhry FN, Nazeer S et al (2016) Causes of ozone layer depletion and its effects on human: review. Atmos Clim Sci 06:129–134. https://doi.org/10.4236/acs.2016.61011

Aquilani B, Silvestri C, Ruggieri A (2016). A Systematic Literature Review on Total Quality Management Critical Success Factors and the Identification of New Avenues of Research. https://doi.org/10.1108/TQM-01-2016-0003

Aramyan L, Hoste R, Van Den Broek W et al (2011) Towards sustainable food production: a scenario study of the European pork sector. J Chain Netw Sci 11:177–189. https://doi.org/10.3920/JCNS2011.Qpork8

Arfini F, Antonioli F, Cozzi E et al (2019) Sustainability, innovation and rural development: the case of Parmigiano-Reggiano PDO. Sustain 11:1–17. https://doi.org/10.3390/su11184978

Assembly UG (2005) Resolution adopted by the general assembly. New York, NY

Avilés-Palacios C, Rodríguez-Olalla A (2021) The sustainability of waste management models in circular economies. Sustain 13:1–19. https://doi.org/10.3390/su13137105

Azevedo SG, Silva ME, Matias JCO, Dias GP (2018) The influence of collaboration initiatives on the sustainability of the cashew supply chain. Sustain 10:1–29. https://doi.org/10.3390/su10062075

Bajaj S, Garg R, Sethi M (2016) Total quality management: a critical literature review using Pareto analysis. Int J Product Perform Manag 67:128–154

Banasik A, Kanellopoulos A, Bloemhof-Ruwaard JM, Claassen GDH (2019) Accounting for uncertainty in eco-efficient agri-food supply chains: a case study for mushroom production planning. J Clean Prod 216:249–256. https://doi.org/10.1016/j.jclepro.2019.01.153

Barth H, Ulvenblad PO, Ulvenblad P (2017) Towards a conceptual framework of sustainable business model innovation in the agri-food sector: a systematic literature review. Sustain 9. https://doi.org/10.3390/su9091620

Bastas A, Liyanage K (2018) Sustainable supply chain quality management: a systematic review

Beckerman W (1992) Economic growth and the environment: whose growth? Whose environment? World Dev 20:481–496. https://doi.org/10.1016/0305-750X(92)90038-W

Belaud JP, Prioux N, Vialle C, Sablayrolles C (2019) Big data for agri-food 4.0: application to sustainability management for by-products supply chain. Comput Ind 111:41–50. https://doi.org/10.1016/j.compind.2019.06.006

Bele B, Norderhaug A, Sickel H (2018) Localized agri-food systems and biodiversity. Agric 8. https://doi.org/10.3390/agriculture8020022

Bilali H El, Calabrese G, Iannetta M et al (2020) Environmental sustainability of typical agro-food products: a scientifically sound and user friendly approach. New Medit 19:69–83. https://doi.org/10.30682/nm2002e

Blanc S, Massaglia S, Brun F et al (2019) Use of bio-based plastics in the fruit supply chain: an integrated approach to assess environmental, economic, and social sustainability. Sustain 11. https://doi.org/10.3390/su11092475

Bloemhof JM, van der Vorst JGAJ, Bastl M, Allaoui H (2015) Sustainability assessment of food chain logistics. Int J Logist Res Appl 18:101–117. https://doi.org/10.1080/13675567.2015.1015508

Bonisoli L, Galdeano-Gómez E, Piedra-Muñoz L (2018) Deconstructing criteria and assessment tools to build agri-sustainability indicators and support farmers’ decision-making process. J Clean Prod 182:1080–1094. https://doi.org/10.1016/j.jclepro.2018.02.055

Bonisoli L, Galdeano-Gómez E, Piedra-Muñoz L, Pérez-Mesa JC (2019) Benchmarking agri-food sustainability certifications: evidences from applying SAFA in the Ecuadorian banana agri-system. J Clean Prod 236. https://doi.org/10.1016/j.jclepro.2019.07.054

Bornmann L, Haunschild R, Hug SE (2018) Visualizing the context of citations referencing papers published by Eugene Garfield: a new type of keyword co-occurrence analysis. Scientometrics 114:427–437. https://doi.org/10.1007/s11192-017-2591-8

Boulding KE (1966) The economics of the coming spaceship earth. New York, 1-17

Bracquené E, Dewulf W, Duflou JR (2020) Measuring the performance of more circular complex product supply chains. Resour Conserv Recycl 154:104608. https://doi.org/10.1016/j.resconrec.2019.104608

Burck J, Hagen U, Bals C et al (2021) Climate Change Performance Index

Calisto Friant M, Vermeulen WJV, Salomone R (2020) A typology of circular economy discourses: navigating the diverse visions of a contested paradigm. Resour Conserv Recycl 161:104917. https://doi.org/10.1016/j.resconrec.2020.104917

Campbell BM, Beare DJ, Bennett EM et al (2017) Agriculture production as a major driver of the earth system exceeding planetary boundaries. Ecol Soc 22. https://doi.org/10.5751/ES-09595-220408

Capitanio F, Coppola A, Pascucci S (2010) Product and process innovation in the Italian food industry. Agribusiness 26:503–518. https://doi.org/10.1002/agr.20239

Caputo P, Zagarella F, Cusenza MA et al (2020) Energy-environmental assessment of the UIA-OpenAgri case study as urban regeneration project through agriculture. Sci Total Environ 729:138819. https://doi.org/10.1016/j.scitotenv.2020.138819

Article   CAS   Google Scholar  

Chabowski BR, Mena JA, Gonzalez-Padron TL (2011) The structure of sustainability research in marketing, 1958–2008: a basis for future research opportunities. J Acad Mark Sci 39:55–70. https://doi.org/10.1007/s11747-010-0212-7

Chadegani AA, Salehi H, Yunus M et al (2017) A comparison between two main academic literature collections : Web of Science and Scopus databases. Asian Soc Sci 9:18–26. https://doi.org/10.5539/ass.v9n5p18

Chams N, Guesmi B, Gil JM (2020) Beyond scientific contribution: assessment of the societal impact of research and innovation to build a sustainable agri-food sector. J Environ Manage 264. https://doi.org/10.1016/j.jenvman.2020.110455

Chandrakumar C, McLaren SJ, Jayamaha NP, Ramilan T (2019) Absolute sustainability-based life cycle assessment (ASLCA): a benchmarking approach to operate agri-food systems within the 2°C global carbon budget. J Ind Ecol 23:906–917. https://doi.org/10.1111/jiec.12830

Chaparro-Africano AM (2019) Toward generating sustainability indicators for agroecological markets. Agroecol Sustain Food Syst 43:40–66. https://doi.org/10.1080/21683565.2019.1566192

Colicchia C, Strozzi F (2012) Supply chain risk management: a new methodology for a systematic literature review

Conca L, Manta F, Morrone D, Toma P (2021) The impact of direct environmental, social, and governance reporting: empirical evidence in European-listed companies in the agri-food sector. Bus Strateg Environ 30:1080–1093. https://doi.org/10.1002/bse.2672

Coppola A, Ianuario S, Romano S, Viccaro M (2020) Corporate social responsibility in agri-food firms: the relationship between CSR actions and firm’s performance. AIMS Environ Sci 7:542–558. https://doi.org/10.3934/environsci.2020034

Corona B, Shen L, Reike D et al (2019) Towards sustainable development through the circular economy—a review and critical assessment on current circularity metrics. Resour Conserv Recycl 151:104498. https://doi.org/10.1016/j.resconrec.2019.104498

Correia MS (2019) Sustainability: An overview of the triple bottom line and sustainability implementation. Int J Strateg Eng 2:29–38.  https://doi.org/10.4018/IJoSE.2019010103

Coteur I, Marchand F, Debruyne L, Lauwers L (2019) Structuring the myriad of sustainability assessments in agri-food systems: a case in Flanders. J Clean Prod 209:472–480. https://doi.org/10.1016/j.jclepro.2018.10.066

CREA (2020) L’agricoltura italiana conta 2019

Crenna E, Sala S, Polce C, Collina E (2017) Pollinators in life cycle assessment: towards a framework for impact assessment. J Clean Prod 140:525–536. https://doi.org/10.1016/j.jclepro.2016.02.058

D’Eusanio M, Serreli M, Zamagni A, Petti L (2018) Assessment of social dimension of a jar of honey: a methodological outline. J Clean Prod 199:503–517. https://doi.org/10.1016/j.jclepro.2018.07.157

Dania WAP, Xing K, Amer Y (2018) Collaboration behavioural factors for sustainable agri-food supply chains: a systematic review. J Clean Prod 186:851–864

De Pascale A, Arbolino R, Szopik-Depczyńska K et al (2021) A systematic review for measuring circular economy: the 61 indicators. J Clean Prod 281. https://doi.org/10.1016/j.jclepro.2020.124942

De Schoenmakere M, Gillabel J (2017) Circular by design: products in the circular economy

Del Borghi A, Gallo M, Strazza C, Del Borghi M (2014) An evaluation of environmental sustainability in the food industry through life cycle assessment: the case study of tomato products supply chain. J Clean Prod 78:121–130. https://doi.org/10.1016/j.jclepro.2014.04.083

Del Borghi A, Strazza C, Magrassi F et al (2018) Life cycle assessment for eco-design of product–package systems in the food industry—the case of legumes. Sustain Prod Consum 13:24–36. https://doi.org/10.1016/j.spc.2017.11.001

Denyer D, Tranfield D (2009) Producing a systematic review. In: Buchanan B (ed) The sage handbook of organization research methods. Sage Publications Ltd, Cornwall, pp 671–689

Google Scholar  

Dietz T, Grabs J, Chong AE (2019) Mainstreamed voluntary sustainability standards and their effectiveness: evidence from the Honduran coffee sector. Regul Gov. https://doi.org/10.1111/rego.12239

Dixon-Woods M (2011) Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med 9:9–10. https://doi.org/10.1186/1741-7015-9-39

do Canto NR, Bossle MB, Marques L, Dutra M, (2020) Supply chain collaboration for sustainability: a qualitative investigation of food supply chains in Brazil. Manag Environ Qual an Int J. https://doi.org/10.1108/MEQ-12-2019-0275

dos Santos RR, Guarnieri P (2020) Social gains for artisanal agroindustrial producers induced by cooperation and collaboration in agri-food supply chain. Soc Responsib J. https://doi.org/10.1108/SRJ-09-2019-0323

Doukidis GI, Matopoulos A, Vlachopoulou M, Manthou V, Manos B (2007) A conceptual framework for supply chain collaboration: empirical evidence from the agri‐food industry. Supply Chain Manag an Int Journal 12:177–186. https://doi.org/10.1108/13598540710742491

Durach CF, Kembro J, Wieland A (2017) A new paradigm for systematic literature reviews in supply chain management. J Supply Chain Manag 53:67–85. https://doi.org/10.1111/jscm.12145

Durán-Sánchez A, Álvarez-García J, Río-Rama D, De la Cruz M (2018) Sustainable water resources management: a bibliometric overview. Water 10:1–19. https://doi.org/10.3390/w10091191

Duru M, Therond O (2015) Livestock system sustainability and resilience in intensive production zones: which form of ecological modernization? Reg Environ Chang 15:1651–1665. https://doi.org/10.1007/s10113-014-0722-9

Edison Fondazione (2019) Le eccellenze agricole italiane. I primati europei e mondiali dell’Italia nei prodotti vegetali. Milan (IT)

Ehrenfeld JR (2005) The roots of sustainability. MIT Sloan Manag Rev 46(2)46:23–25

Elia V, Gnoni MG, Tornese F (2017) Measuring circular economy strategies through index methods: a critical analysis. J Clean Prod 142:2741–2751. https://doi.org/10.1016/j.jclepro.2016.10.196

Elkington J (1997) Cannibals with forks : the triple bottom line of 21st century business. Capstone, Oxford

Esposito B, Sessa MR, Sica D, Malandrino O (2020) Towards circular economy in the agri-food sector. A systematic literature review. Sustain 12. https://doi.org/10.3390/SU12187401

European Commission (2018) Agri-food trade in 2018

European Commission (2019) Monitoring EU agri-food trade: development until September 2019

Eurostat (2018) Small and large farms in the EU - statistics from the farm structure survey

FAO (2011) Biodiversity for food and agriculture. Italy, Rome

FAO (2012) Energy-smart food at FAO: an overview. Italy, Rome

FAO (2014) Food wastage footprint: fool cost-accounting

FAO (2016) The state of food and agriculture climate change, agriculture and food security. Italy, Rome

FAO (2017) The future of food and agriculture: trends and challenges. Italy, Rome

FAO (2020) The state of food security and nutrition in the world. Transforming Food Systems for Affordable Healthy Diets. Rome, Italy

Fassio F, Tecco N (2019) Circular economy for food: a systemic interpretation of 40 case histories in the food system in their relationships with SDGs. Systems 7:43. https://doi.org/10.3390/systems7030043

Fathollahi A, Coupe SJ (2021) Life cycle assessment (LCA) and life cycle costing (LCC) of road drainage systems for sustainability evaluation: quantifying the contribution of different life cycle phases. Sci Total Environ 776:145937. https://doi.org/10.1016/j.scitotenv.2021.145937

Ferreira VJ, Arnal ÁJ, Royo P et al (2019) Energy and resource efficiency of electroporation-assisted extraction as an emerging technology towards a sustainable bio-economy in the agri-food sector. J Clean Prod 233:1123–1132. https://doi.org/10.1016/j.jclepro.2019.06.030

Fiksel J (2006) A framework for sustainable remediation. JOM 8:15–22. https://doi.org/10.1021/es202595w

Flick U (2014) An introduction to qualitative research

Franciosi C, Voisin A, Miranda S et al (2020) Measuring maintenance impacts on sustainability of manufacturing industries : from a systematic literature review to a framework proposal. J Clean Prod 260:1–19. https://doi.org/10.1016/j.jclepro.2020.121065

Gaitán-Cremaschi D, Meuwissen MPM, Oude AGJML (2017) Total factor productivity: a framework for measuring agri-food supply chain performance towards sustainability. Appl Econ Perspect Policy 39:259–285. https://doi.org/10.1093/aepp/ppw008

Galdeano-Gómez E, Zepeda-Zepeda JA, Piedra-Muñoz L, Vega-López LL (2017) Family farm’s features influencing socio-economic sustainability: an analysis of the agri-food sector in southeast Spain. New Medit 16:50–61

Gallopín G, Herrero LMJ, Rocuts A (2014) Conceptual frameworks and visual interpretations of sustainability. Int J Sustain Dev 17:298–326. https://doi.org/10.1504/IJSD.2014.064183

Gallopín GC (2003) Sostenibilidad y desarrollo sostenible: un enfoque sistémico. Cepal, LATIN AMERICA

Garnett T (2013) Food sustainability: problems, perspectives and solutions. Proc Nutr Soc 72:29–39. https://doi.org/10.1017/S0029665112002947

Garofalo P, D’Andrea L, Tomaiuolo M et al (2017) Environmental sustainability of agri-food supply chains in Italy: the case of the whole-peeled tomato production under life cycle assessment methodology. J Food Eng 200:1–12. https://doi.org/10.1016/j.jfoodeng.2016.12.007

Gava O, Bartolini F, Venturi F et al (2018) A reflection of the use of the life cycle assessment tool for agri-food sustainability. Sustain 11. https://doi.org/10.3390/su11010071

Gazzola P, Querci E (2017) The connection between the quality of life and sustainable ecological development. Eur Sci J 7881:1857–7431

Geissdoerfer M, Savaget P, Bocken N, Hultink EJ (2017) The circular economy – a new sustainability paradigm ? The circular economy – a new sustainability paradigm ? J Clean Prod 143:757–768. https://doi.org/10.1016/j.jclepro.2016.12.048

Georgescu-Roegen N (1971) The entropy low and the economic process. Harward University Press, Cambridge Mass

Book   Google Scholar  

Gerbens-Leenes PW, Moll HC, Schoot Uiterkamp AJM (2003) Design and development of a measuring method for environmental sustainability in food production systems. Ecol Econ 46:231–248. https://doi.org/10.1016/S0921-8009(03)00140-X

Gésan-Guiziou G, Alaphilippe A, Aubin J et al (2020) Diversity and potentiality of multi-criteria decision analysis methods for agri-food research. Agron Sustain Dev 40. https://doi.org/10.1007/s13593-020-00650-3

Ghisellini P, Cialani C, Ulgiati S (2016) A review on circular economy: the expected transition to a balanced interplay of environmental and economic systems. J Clean Prod 114:11–32. https://doi.org/10.1016/j.jclepro.2015.09.007

Godoy-Durán Á, Galdeano- Gómez E, Pérez-Mesa JC, Piedra-Muñoz L (2017) Assessing eco-efficiency and the determinants of horticultural family-farming in southeast Spain. J Environ Manage 204:594–604. https://doi.org/10.1016/j.jenvman.2017.09.037

Gold S, Kunz N, Reiner G (2017) Sustainable global agrifood supply chains: exploring the barriers. J Ind Ecol 21:249–260. https://doi.org/10.1111/jiec.12440

Goucher L, Bruce R, Cameron DD et al (2017) The environmental impact of fertilizer embodied in a wheat-to-bread supply chain. Nat Plants 3:1–5. https://doi.org/10.1038/nplants.2017.12

Green A, Nemecek T, Chaudhary A, Mathys A (2020) Assessing nutritional, health, and environmental sustainability dimensions of agri-food production. Glob Food Sec 26:100406. https://doi.org/10.1016/j.gfs.2020.100406

Guinée JB, Heijungs R, Huppes G et al (2011) Life cycle assessment: past, present, and future †. Environ Sci Technol 45:90–96. https://doi.org/10.1021/es101316v

Guiomar N, Godinho S, Pinto-Correia T et al (2018) Typology and distribution of small farms in Europe: towards a better picture. Land Use Policy 75:784–798. https://doi.org/10.1016/j.landusepol.2018.04.012

Gunasekaran A, Patel C, McGaughey RE (2004) A framework for supply chain performance measurement. Int J Prod Econ 87:333–347. https://doi.org/10.1016/j.ijpe.2003.08.003

Gunasekaran A, Patel C, Tirtiroglu E (2001) Performance measures and metrics in a supply chain environment. Int J Oper Prod Manag 21:71–87. https://doi.org/10.1108/01443570110358468

Hamam M, Chinnici G, Di Vita G et al (2021) Circular economy models in agro-food systems: a review. Sustain 13

Harun SN, Hanafiah MM, Aziz NIHA (2021) An LCA-based environmental performance of rice production for developing a sustainable agri-food system in Malaysia. Environ Manage 67:146–161. https://doi.org/10.1007/s00267-020-01365-7

Harvey M, Pilgrim S (2011) The new competition for land: food, energy, and climate change. Food Policy 36:S40–S51. https://doi.org/10.1016/j.foodpol.2010.11.009

Hawkes C, Ruel MT (2006) Understanding the links between agriculture and health. DC: International Food Policy Research Institute. Washington, USA

Hellweg S, Milà i Canals L (2014) Emerging approaches, challenges and opportunities in life cycle assessment. Science (80)344:1109LP–1113. https://doi.org/10.1126/science.1248361

Higgins V, Dibden J, Cocklin C (2015) Private agri-food governance and greenhouse gas abatement: constructing a corporate carbon economy. Geoforum 66:75–84. https://doi.org/10.1016/j.geoforum.2015.09.012

Hill T (1995) Manufacturing strategy: text and cases., Macmillan

Hjeresen DD, Gonzales R (2020) Green chemistry promote sustainable agriculture?The rewards are higher yields and less environmental contamination. Environemental Sci Techonology 103–107

Horne R, Grant T, Verghese K (2009) Life cycle assessment: principles, practice, and prospects. Csiro Publishing, Collingwood, Australia

Horton P, Koh L, Guang VS (2016) An integrated theoretical framework to enhance resource efficiency, sustainability and human health in agri-food systems. J Clean Prod 120:164–169. https://doi.org/10.1016/j.jclepro.2015.08.092

Hospido A, Davis J, Berlin J, Sonesson U (2010) A review of methodological issues affecting LCA of novel food products. Int J Life Cycle Assess 15:44–52. https://doi.org/10.1007/s11367-009-0130-4

Huffman T, Liu J, Green M et al (2015) Improving and evaluating the soil cover indicator for agricultural land in Canada. Ecol Indic 48:272–281. https://doi.org/10.1016/j.ecolind.2014.07.008

Ilbery B, Maye D (2005) Food supply chains and sustainability: evidence from specialist food producers in the Scottish/English borders. Land Use Policy 22:331–344. https://doi.org/10.1016/j.landusepol.2004.06.002

Ingrao C, Faccilongo N, Valenti F et al (2019) Tomato puree in the Mediterranean region: an environmental life cycle assessment, based upon data surveyed at the supply chain level. J Clean Prod 233:292–313. https://doi.org/10.1016/j.jclepro.2019.06.056

Iocola I, Angevin F, Bockstaller C et al (2020) An actor-oriented multi-criteria assessment framework to support a transition towards sustainable agricultural systems based on crop diversification. Sustain 12. https://doi.org/10.3390/su12135434

Irabien A, Darton RC (2016) Energy–water–food nexus in the Spanish greenhouse tomato production. Clean Technol Environ Policy 18:1307–1316. https://doi.org/10.1007/s10098-015-1076-9

ISO 14040:2006 (2006) Environmental management — life cycle assessment — principles and framework

ISO 14044:2006 (2006) Environmental management — life cycle assessment — requirements and guidelines

ISO 15392:2008 (2008) Sustainability in building construction–general principles

Istat (2019) Andamento dell’economia agricola

Jaakkola E (2020) Designing conceptual articles : four approaches. AMS Rev 1–9. https://doi.org/10.1007/s13162-020-00161-0

Jin R, Yuan H, Chen Q (2019) Science mapping approach to assisting the review of construction and demolition waste management research published between 2009 and 2018. Resour Conserv Recycl 140:175–188. https://doi.org/10.1016/j.resconrec.2018.09.029

Johnston P, Everard M, Santillo D, Robèrt KH (2007) Reclaiming the definition of sustainability. Environ Sci Pollut Res Int 14:60–66. https://doi.org/10.1065/espr2007.01.375

Jorgensen SE, Burkhard B, Müller F (2013) Twenty volumes of ecological indicators-an accounting short review. Ecol Indic 28:4–9. https://doi.org/10.1016/j.ecolind.2012.12.018

Joshi S, Sharma M, Kler R (2020) Modeling circular economy dimensions in agri-tourism clusters: sustainable performance and future research directions. Int J Math Eng Manag Sci 5:1046–1061. https://doi.org/10.33889/IJMEMS.2020.5.6.080

Kamilaris A, Gao F, Prenafeta-Boldu FX, Ali MI (2017) Agri-IoT: a semantic framework for Internet of Things-enabled smart farming applications. In: 2016 IEEE 3rd World Forum on Internet of Things, WF-IoT 2016. pp 442–447

Karuppusami G, Gandhinathan R (2006) Pareto analysis of critical success factors of total quality management: a literature review and analysis. TQM Mag 18:372–385. https://doi.org/10.1108/09544780610671048

Kates RW, Parris TM, Leiserowitz AA (2005) What is sustainable development? Goals, indicators, values, and practice. Environ Sci Policy Sustain Dev 47:8–21. https://doi.org/10.1080/00139157.2005.10524444

Khounani Z, Hosseinzadeh-Bandbafha H, Moustakas K et al (2021) Environmental life cycle assessment of different biorefinery platforms valorizing olive wastes to biofuel, phosphate salts, natural antioxidant, and an oxygenated fuel additive (triacetin). J Clean Prod 278:123916. https://doi.org/10.1016/j.jclepro.2020.123916

Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering version 2.3. Engineering 45. https://doi.org/10.1145/1134285.1134500

Korhonen J, Nuur C, Feldmann A, Birkie SE (2018) Circular economy as an essentially contested concept. J Clean Prod 175:544–552. https://doi.org/10.1016/j.jclepro.2017.12.111

Kuisma M, Kahiluoto H (2017) Biotic resource loss beyond food waste: agriculture leaks worst. Resour Conserv Recycl 124:129–140. https://doi.org/10.1016/j.resconrec.2017.04.008

Laso J, Hoehn D, Margallo M et al (2018) Assessing energy and environmental efficiency of the Spanish agri-food system using the LCA/DEA methodology. Energies 11. https://doi.org/10.3390/en11123395

Lee KM (2007) So What is the “triple bottom line”? Int J Divers Organ Communities Nations Annu Rev 6:67–72. https://doi.org/10.18848/1447-9532/cgp/v06i06/39283

Lehmann RJ, Hermansen JE, Fritz M et al (2011) Information services for European pork chains - closing gaps in information infrastructures. Comput Electron Agric 79:125–136. https://doi.org/10.1016/j.compag.2011.09.002

León-Bravo V, Caniato F, Caridi M, Johnsen T (2017) Collaboration for sustainability in the food supply chain: a multi-stage study in Italy. Sustainability 9:1253

Lepage A (2009) The quality of life as attribute of sustainability. TQM J 21:105–115. https://doi.org/10.1108/17542730910938119

Li CZ, Zhao Y, Xiao B et al (2020) Research trend of the application of information technologies in construction and demolition waste management. J Clean Prod 263. https://doi.org/10.1016/j.jclepro.2020.121458

Lo Giudice A, Mbohwa C, Clasadonte MT, Ingrao C (2014) Life cycle assessment interpretation and improvement of the Sicilian artichokes production. Int J Environ Res 8:305–316. https://doi.org/10.22059/ijer.2014.721

Lueddeckens S, Saling P, Guenther E (2020) Temporal issues in life cycle assessment—a systematic review. Int J Life Cycle Assess 25:1385–1401. https://doi.org/10.1007/s11367-020-01757-1

Luo J, Ji C, Qiu C, Jia F (2018) Agri-food supply chain management: bibliometric and content analyses. Sustain 10. https://doi.org/10.3390/su10051573

Lynch J, Donnellan T, Finn JA et al (2019) Potential development of Irish agricultural sustainability indicators for current and future policy evaluation needs. J Environ Manage 230:434–445. https://doi.org/10.1016/j.jenvman.2018.09.070

MacArthur E (2013) Towards the circular economy. J Ind Ecol 2:23–44

MacArthur E (2017) Delivering the circular economy a toolkit for policymakers, The Ellen MacArthur Foundation

MacInnis DJ (2011) A framework for conceptual. J Mark 75:136–154. https://doi.org/10.1509/jmkg.75.4.136

Mangla SK, Luthra S, Rich N et al (2018) Enablers to implement sustainable initiatives in agri-food supply chains. Int J Prod Econ 203:379–393. https://doi.org/10.1016/j.ijpe.2018.07.012

Marotta G, Nazzaro C, Stanco M (2017) How the social responsibility creates value: models of innovation in Italian pasta industry. Int J Glob Small Bus 9:144–167. https://doi.org/10.1504/IJGSB.2017.088923

Martucci O, Arcese G, Montauti C, Acampora A (2019) Social aspects in the wine sector: comparison between social life cycle assessment and VIVA sustainable wine project indicators. Resources 8. https://doi.org/10.3390/resources8020069

Mayring P (2004) Forum : Qualitative social research Sozialforschung 2. History of content analysis. A Companion to Qual Res 1:159–176

McKelvey B (2002) Managing coevolutionary dynamics. In: 18th EGOS Conference. Barcelona, Spain, pp 1–21

McMichael AJ, Butler CD, Folke C (2003) New visions for addressing sustainability. Science (80- ) 302:1191–1920

Mehmood A, Ahmed S, Viza E et al (2021) Drivers and barriers towards circular economy in agri-food supply chain: a review. Bus Strateg Dev 1–17. https://doi.org/10.1002/bsd2.171

Mella P, Gazzola P (2011) Sustainability and quality of life: the development model. In: Kapounek S (ed) Enterprise and competitive environment. Mendel University: Brno, Czechia. 542–551

Merli R, Preziosi M, Acampora A (2018) How do scholars approach the circular economy ? A systematic literature review. J Clean Prod 178:703–722. https://doi.org/10.1016/j.jclepro.2017.12.112

Merli R, Preziosi M, Acampora A et al (2020) Recycled fibers in reinforced concrete: a systematic literature review. J Clean Prod 248:119207. https://doi.org/10.1016/j.jclepro.2019.119207

Miglietta PP, Morrone D (2018) Managing water sustainability: virtual water flows and economic water productivity assessment of the wine trade between Italy and the Balkans. Sustain 10. https://doi.org/10.3390/su10020543

Mitchell MGE, Chan KMA, Newlands NK, Ramankutty N (2020) Spatial correlations don’t predict changes in agricultural ecosystem services: a Canada-wide case study. Front Sustain Food Syst 4:1–17. https://doi.org/10.3389/fsufs.2020.539892

Moraga G, Huysveld S, Mathieux F et al (2019) Circular economy indicators: what do they measure?. Resour Conserv Recycl 146:452–461. https://doi.org/10.1016/j.resconrec.2019.03.045

Morrissey JE, Dunphy NP (2015) Towards sustainable agri-food systems: the role of integrated sustainability and value assessment across the supply-chain. Int J Soc Ecol Sustain Dev 6:41–58. https://doi.org/10.4018/IJSESD.2015070104

Moser G (2009) Quality of life and sustainability: toward person-environment congruity. J Environ Psychol 29:351–357. https://doi.org/10.1016/j.jenvp.2009.02.002

Muijs D (2010) Doing quantitative research in education with SPSS. London

Muller MF, Esmanioto F, Huber N, Loures ER (2019) A systematic literature review of interoperability in the green Building Information Modeling lifecycle. J Clean Prod 223:397–412. https://doi.org/10.1016/j.jclepro.2019.03.114

Muradin M, Joachimiak-Lechman K, Foltynowicz Z (2018) Evaluation of eco-efficiency of two alternative agricultural biogas plants. Appl Sci 8. https://doi.org/10.3390/app8112083

Naseer MA, ur R, Ashfaq M, Hassan S, et al (2019) Critical issues at the upstream level in sustainable supply chain management of agri-food industries: evidence from Pakistan’s citrus industry. Sustain 11:1–19. https://doi.org/10.3390/su11051326

Nattassha R, Handayati Y, Simatupang TM, Siallagan M (2020) Understanding circular economy implementation in the agri-food supply chain: the case of an Indonesian organic fertiliser producer. Agric Food Secur 9:1–16. https://doi.org/10.1186/s40066-020-00264-8

Nazari-Sharabian M, Ahmad S, Karakouzian M (2018) Climate change and eutrophication: a short review. Eng Technol Appl Sci Res 8:3668–3672. https://doi.org/10.5281/zenodo.2532694

Nazir N (2017) Understanding life cycle thinking and its practical application to agri-food system. Int J Adv Sci Eng Inf Technol 7:1861–1870. https://doi.org/10.18517/ijaseit.7.5.3578

Negra C, Remans R, Attwood S et al (2020) Sustainable agri-food investments require multi-sector co-development of decision tools. Ecol Indic 110:105851. https://doi.org/10.1016/j.ecolind.2019.105851

Newsham KK, Robinson SA (2009) Responses of plants in polar regions to UVB exposure: a meta-analysis. Glob Chang Biol 15:2574–2589. https://doi.org/10.1111/j.1365-2486.2009.01944.x

Niemeijer D, de Groot RS (2008) A conceptual framework for selecting environmental indicator sets. Ecol Indic 8:14–25. https://doi.org/10.1016/j.ecolind.2006.11.012

Niero M, Kalbar PP (2019) Coupling material circularity indicators and life cycle based indicators: a proposal to advance the assessment of circular economy strategies at the product level. Resour Conserv Recycl 140:305–312. https://doi.org/10.1016/j.resconrec.2018.10.002

Nikolaou IE, Tsagarakis KP (2021) An introduction to circular economy and sustainability: some existing lessons and future directions. Sustain Prod Consum 28:600–609. https://doi.org/10.1016/j.spc.2021.06.017

Notarnicola B, Hayashi K, Curran MA, Huisingh D (2012) Progress in working towards a more sustainable agri-food industry. J Clean Prod 28:1–8. https://doi.org/10.1016/j.jclepro.2012.02.007

Notarnicola B, Tassielli G, Renzulli PA, Monforti F (2017) Energy flows and greenhouses gases of EU (European Union) national breads using an LCA (life cycle assessment) approach. J Clean Prod 140:455–469. https://doi.org/10.1016/j.jclepro.2016.05.150

Opferkuch K, Caeiro S, Salomone R, Ramos TB (2021) Circular economy in corporate sustainability reporting: a review of organisational approaches. Bus Strateg Environ 1–22. https://doi.org/10.1002/bse.2854

Padilla-Rivera A, do Carmo BBT, Arcese G, Merveille N, (2021) Social circular economy indicators: selection through fuzzy delphi method. Sustain Prod Consum 26:101–110. https://doi.org/10.1016/j.spc.2020.09.015

Pagotto M, Halog A (2016) Towards a circular economy in Australian agri-food industry: an application of input-output oriented approaches for analyzing resource efficiency and competitiveness potential. J Ind Ecol 20:1176–1186. https://doi.org/10.1111/jiec.12373

Parent G, Lavallée S (2011) LCA potentials and limits within a sustainable agri-food statutory framework. Global food insecurity. Springer, Netherlands, Dordrecht, pp 161–171

Chapter   Google Scholar  

Pattey E, Qiu G (2012) Trends in primary particulate matter emissions from Canadian agriculture. J Air Waste Manag Assoc 62:737–747. https://doi.org/10.1080/10962247.2012.672058

Pauliuk S (2018) Critical appraisal of the circular economy standard BS 8001:2017 and a dashboard of quantitative system indicators for its implementation in organizations. Resour Conserv Recycl 129:81–92. https://doi.org/10.1016/j.resconrec.2017.10.019

Peano C, Migliorini P, Sottile F (2014) A methodology for the sustainability assessment of agri-food systems: an application to the slow food presidia project. Ecol Soc 19. https://doi.org/10.5751/ES-06972-190424

Peano C, Tecco N, Dansero E et al (2015) Evaluating the sustainability in complex agri-food systems: the SAEMETH framework. Sustain 7:6721–6741. https://doi.org/10.3390/su7066721

Pearce DW, Turner RK (1990) Economics of natural resources and the environment. Harvester Wheatsheaf, Hemel Hempstead, Herts

Pelletier N (2018) Social sustainability assessment of Canadian egg production facilities: methods, analysis, and recommendations. Sustain 10:1–17. https://doi.org/10.3390/su10051601

Peña C, Civit B, Gallego-Schmid A et al (2021) Using life cycle assessment to achieve a circular economy. Int J Life Cycle Assess 26:215–220. https://doi.org/10.1007/s11367-020-01856-z

Perez Neira D (2016) Energy sustainability of Ecuadorian cacao export and its contribution to climate change. A case study through product life cycle assessment. J Clean Prod 112:2560–2568. https://doi.org/10.1016/j.jclepro.2015.11.003

Pérez-Neira D, Grollmus-Venegas A (2018) Life-cycle energy assessment and carbon footprint of peri-urban horticulture. A comparative case study of local food systems in Spain. Landsc Urban Plan 172:60–68. https://doi.org/10.1016/j.landurbplan.2018.01.001

Pérez-Pons ME, Plaza-Hernández M, Alonso RS et al (2021) Increasing profitability and monitoring environmental performance: a case study in the agri-food industry through an edge-iot platform. Sustain 13:1–16. https://doi.org/10.3390/su13010283

Petti L, Serreli M, Di Cesare S (2018) Systematic literature review in social life cycle assessment. Int J Life Cycle Assess 23:422–431. https://doi.org/10.1007/s11367-016-1135-4

Pieroni MPP, McAloone TC, Pigosso DCA (2019) Business model innovation for circular economy and sustainability: a review of approaches. J Clean Prod 215:198–216. https://doi.org/10.1016/j.jclepro.2019.01.036

Polit DF, Beck CT (2004) Nursing research: principles and methods. Lippincott Williams & Wilkins, Philadelphia, PA

Porkka M, Gerten D, Schaphoff S et al (2016) Causes and trends of water scarcity in food production. Environ Res Lett 11:015001. https://doi.org/10.1088/1748-9326/11/1/015001

Prajapati H, Kant R, Shankar R (2019) Bequeath life to death: state-of-art review on reverse logistics. J Clean Prod 211:503–520. https://doi.org/10.1016/j.jclepro.2018.11.187

Priyadarshini P, Abhilash PC (2020) Policy recommendations for enabling transition towards sustainable agriculture in India. Land Use Policy 96:104718. https://doi.org/10.1016/j.landusepol.2020.104718

Pronti A, Coccia M (2020) Multicriteria analysis of the sustainability performance between agroecological and conventional coffee farms in the East Region of Minas Gerais (Brazil). Renew Agric Food Syst. https://doi.org/10.1017/S1742170520000332

Rabadán A, González-Moreno A, Sáez-Martínez FJ (2019) Improving firms’ performance and sustainability: the case of eco-innovation in the agri-food industry. Sustain 11. https://doi.org/10.3390/su11205590

Raut RD, Luthra S, Narkhede BE et al (2019) Examining the performance oriented indicators for implementing green management practices in the Indian agro sector. J Clean Prod 215:926–943. https://doi.org/10.1016/j.jclepro.2019.01.139

Recanati F, Marveggio D, Dotelli G (2018) From beans to bar: a life cycle assessment towards sustainable chocolate supply chain. Sci Total Environ 613–614:1013–1023. https://doi.org/10.1016/j.scitotenv.2017.09.187

Redclift M (2005) Sustainable development (1987–2005): an oxymoron comes of age. Sustain Dev 13:212–227. https://doi.org/10.1002/sd.281

Rezaei M, Soheilifard F, Keshvari A (2021) Impact of agrochemical emission models on the environmental assessment of paddy rice production using life cycle assessment approach. Energy Sources. Part A Recover Util Environ Eff 1–16

Rigamonti L, Mancini E (2021) Life cycle assessment and circularity indicators. Int J Life Cycle Assess. https://doi.org/10.1007/s11367-021-01966-2

Risku-Norja H, Mäenpää I (2007) MFA model to assess economic and environmental consequences of food production and consumption. Ecol Econ 60:700–711. https://doi.org/10.1016/j.ecolecon.2006.05.001

Ritzén S, Sandström GÖ (2017) Barriers to the circular economy – integration of perspectives and domains. Procedia CIRP 64:7–12. https://doi.org/10.1016/j.procir.2017.03.005

Rockström J, Steffen W, Noone K et al (2009) A safe operating space for humanity. Nature 461:472–475. https://doi.org/10.1038/461472a

Roos Lindgreen E, Mondello G, Salomone R et al (2021) Exploring the effectiveness of grey literature indicators and life cycle assessment in assessing circular economy at the micro level: a comparative analysis. Int J Life Cycle Assess. https://doi.org/10.1007/s11367-021-01972-4

Roselli L, Casieri A, De Gennaro BC et al (2020) Environmental and economic sustainability of table grape production in Italy. Sustain 12.  https://doi.org/10.3390/su12093670

Ross RB, Pandey V, Ross KL (2015) Sustainability and strategy in U.S. agri-food firms: an assessment of current practices. Int Food Agribus Manag Rev 18:17–48

Royo P, Ferreira VJ, López-Sabirón AM, Ferreira G. (2016) Hybrid diagnosis to characterise the energy and environmental enhancement of photovoltaic modules using smart materials. Energy 101:174–189. https://doi.org/10.1016/j.energy.2016.01.101

Ruggerio CA (2021) Sustainability and sustainable development: a review of principles and definitions. Sci Total Environ 786:147481. https://doi.org/10.1016/j.scitotenv.2021.147481

Ruiz-Almeida A, Rivera-Ferre MG (2019) Internationally-based indicators to measure agri-food systems sustainability using food sovereignty as a conceptual framework. Food Secur 11:1321–1337. https://doi.org/10.1007/s12571-019-00964-5

Ryan M, Hennessy T, Buckley C et al (2016) Developing farm-level sustainability indicators for Ireland using the Teagasc National Farm Survey. Irish J Agric Food Res 55:112–125. https://doi.org/10.1515/ijafr-2016-0011

Saade MRM, Yahia A, Amor B (2020) How has LCA been applied to 3D printing ? A systematic literature review and recommendations for future studies. J Clean Prod 244:118803. https://doi.org/10.1016/j.jclepro.2019.118803

Saitone TL, Sexton RJ (2017) Agri-food supply chain: evolution and performance with conflicting consumer and societal demands. Eur Rev Agric Econ 44:634–657. https://doi.org/10.1093/erae/jbx003

Salim N, Ab Rahman MN, Abd Wahab D (2019) A systematic literature review of internal capabilities for enhancing eco-innovation performance of manufacturing firms. J Clean Prod 209:1445–1460. https://doi.org/10.1016/j.jclepro.2018.11.105

Salimi N (2021) Circular economy in agri-food systems BT - strategic decision making for sustainable management of industrial networks. In: International S (ed) Rezaei J. Publishing, Cham, pp 57–70

Salomone R, Ioppolo G (2012) Environmental impacts of olive oil production: a life cycle assessment case study in the province of Messina (Sicily). J Clean Prod 28:88–100. https://doi.org/10.1016/j.jclepro.2011.10.004

Sánchez AD, Río DMDLC, García JÁ (2017) Bibliometric analysis of publications on wine tourism in the databases Scopus and WoS. Eur Res Manag Bus Econ 23:8–15. https://doi.org/10.1016/j.iedeen.2016.02.001

Saputri VHL, Sutopo W, Hisjam M, Ma’aram A (2019) Sustainable agri-food supply chain performance measurement model for GMO and non-GMO using data envelopment analysis method. Appl Sci 9. https://doi.org/10.3390/app9061199

Sassanelli C, Rosa P, Rocca R, Terzi S (2019) Circular economy performance assessment methods : a systematic literature review. J Clean Prod 229:440–453. https://doi.org/10.1016/j.jclepro.2019.05.019

Schiefer S, Gonzalez C, Flanigan S (2015) More than just a factor in transition processes? The role of collaboration in agriculture. In: Sutherland LA, Darnhofer I, Wilson GA, Zagata L (eds) Transition pathways towards sustainability in agriculture: case studies from Europe, CPI Group. Croydon, UK, pp. 83

Seuring S, Muller M (2008) From a literature review to a conceptual framework for sustainable supply chain management. J Clean Prod 16:1699–1710. https://doi.org/10.1016/j.jclepro.2008.04.020

Silvestri C, Silvestri L, Forcina A, et al (2021) Green chemistry contribution towards more equitable global sustainability and greater circular economy: A systematic literature review. J Clean Prod 294. https://doi.org/10.1016/j.jclepro.2021.126137

Smetana S, Schmitt E, Mathys A (2019) Sustainable use of Hermetia illucens insect biomass for feed and food: attributional and consequential life cycle assessment. Resour Conserv Recycl 144:285–296. https://doi.org/10.1016/j.resconrec.2019.01.042

Sonesson U, Berlin J, Ziegler F (2010) Environmental assessment and management in the food industry: life cycle assessment and related approaches. Woodhead Publishing, Cambridge

Soussana JF (2014) Research priorities for sustainable agri-food systems and life cycle assessment. J Clean Prod 73:19–23. https://doi.org/10.1016/j.jclepro.2014.02.061

Soylu A, Oruç C, Turkay M et al (2006) Synergy analysis of collaborative supply chain management in energy systems using multi-period MILP. Eur J Oper Res 174:387–403. https://doi.org/10.1016/j.ejor.2005.02.042

Spaiser V, Ranganathan S, Swain RB, Sumpter DJ (2017) The sustainable development oxymoron: quantifying and modelling the incompatibility of sustainable development goals. Int J Sustain Dev World Ecol 24:457–470. https://doi.org/10.1080/13504509.2016.1235624

Stewart R, Niero M (2018) Circular economy in corporate sustainability strategies: a review of corporate sustainability reports in the fast-moving consumer goods sector. Bus Strateg Environ 27:1005–1022. https://doi.org/10.1002/bse.2048

Stillitano T, Spada E, Iofrida N et al (2021) Sustainable agri-food processes and circular economy pathways in a life cycle perspective: state of the art of applicative research. Sustain 13:1–29. https://doi.org/10.3390/su13052472

Stone J, Rahimifard S (2018) Resilience in agri-food supply chains: a critical analysis of the literature and synthesis of a novel framework. Supply Chain Manag 23:207–238. https://doi.org/10.1108/SCM-06-2017-0201

Strazza C, Del Borghi A, Gallo M, Del Borghi M (2011) Resource productivity enhancement as means for promoting cleaner production: analysis of co-incineration in cement plants through a life cycle approach. J Clean Prod 19:1615–1621. https://doi.org/10.1016/j.jclepro.2011.05.014

Su B, Heshmati A, Geng Y, Yu X (2013) A review of the circular economy in China: moving from rhetoric to implementation. J Clean Prod 42:215–227. https://doi.org/10.1016/j.jclepro.2012.11.020

Suárez-Eiroa B, Fernández E, Méndez-Martínez G, Soto-Oñate D (2019) Operational principles of circular economy for sustainable development: linking theory and practice. J Clean Prod 214:952–961. https://doi.org/10.1016/j.jclepro.2018.12.271

Svensson G, Wagner B (2015) Implementing and managing economic, social and environmental efforts of business sustainability. Manag Environ Qual an Int Journal 26:195–213. https://doi.org/10.1108/MEQ-09-2013-0099

Tasca AL, Nessi S, Rigamonti L (2017) Environmental sustainability of agri-food supply chains: an LCA comparison between two alternative forms of production and distribution of endive in northern Italy. J Clean Prod 140:725–741. https://doi.org/10.1016/j.jclepro.2016.06.170

Tassielli G, Notarnicola B, Renzulli PA, Arcese G (2018) Environmental life cycle assessment of fresh and processed sweet cherries in southern Italy. J Clean Prod 171:184–197. https://doi.org/10.1016/j.jclepro.2017.09.227

Teixeira R, Pax S (2011) A survey of life cycle assessment practitioners with a focus on the agri-food sector. J Ind Ecol 15:817–820. https://doi.org/10.1111/j.1530-9290.2011.00421.x

Tobergte DR, Curtis S (2013) ILCD Handbook. J Chem Info Model. https://doi.org/10.278/33030

Tortorella MM, Di Leo S, Cosmi C et al (2020) A methodological integrated approach to analyse climate change effects in agri-food sector: the TIMES water-energy-food module. Int J Environ Res Public Health 17:1–21. https://doi.org/10.3390/ijerph17217703

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidenceinformed management knowledge by means of systematic review. Br J Manag 14:207–222

Trivellas P, Malindretos G, Reklitis P (2020) Implications of green logistics management on sustainable business and supply chain performance: evidence from a survey in the greek agri-food sector. Sustain 12:1–29. https://doi.org/10.3390/su122410515

Tsangas M, Gavriel I, Doula M et al (2020) Life cycle analysis in the framework of agricultural strategic development planning in the Balkan region. Sustain 12:1–15. https://doi.org/10.3390/su12051813

Ülgen VS, Björklund M, Simm N (2019) Inter-organizational supply chain interaction for sustainability : a systematic literature review.

UNEP S (2020) Guidelines for social life cycle assessment of products and organizations 2020.

UNEP/SETAC (2009) United Nations Environment Programme-society of Environmental Toxicology and Chemistry. Guidelines for social life cycle assessment of products. France

United Nations (2011) Guiding principles on business and human rights. Implementing the United Nations “protect, respect and remedy” framework

United Nations (2015) Transforming our world: the 2030 agenda for sustainable development. sustainabledevelopment.un.org

Van Asselt ED, Van Bussel LGJ, Van Der Voet H et al (2014) A protocol for evaluating the sustainability of agri-food production systems - a case study on potato production in peri-urban agriculture in the Netherlands. Ecol Indic 43:315–321. https://doi.org/10.1016/j.ecolind.2014.02.027

Van der Ploeg JD (2014) Peasant-driven agricultural growth and food sovereignty. J Peasant Stud 41:999–1030. https://doi.org/10.1080/03066150.2013.876997

van Eck NJ, Waltman L (2010) Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84:523–538. https://doi.org/10.1007/s11192-009-0146-3

Van Eck NJ, Waltman L (2019) Manual for VOSviwer version 1.6.10. CWTS Meaningful metrics 1–53

Vasa L, Angeloska A, Trendov NM (2017) Comparative analysis of circular agriculture development in selected Western Balkan countries based on sustainable performance indicators. Econ Ann 168:44–47. https://doi.org/10.21003/ea.V168-09

Verdecho MJ, Alarcón-Valero F, Pérez-Perales D et al (2020) A methodology to select suppliers to increase sustainability within supply chains. Cent Eur J Oper Res. https://doi.org/10.1007/s10100-019-00668-3

Vergine P, Salerno C, Libutti A et al (2017) Closing the water cycle in the agro-industrial sector by reusing treated wastewater for irrigation. J Clean Prod 164:587–596. https://doi.org/10.1016/j.jclepro.2017.06.239

WCED (1987) Our common future - call for action

Webster K (2013) What might we say about a circular economy? Some temptations to avoid if possible. World Futures 69:542–554

Wheaton E, Kulshreshtha S (2013) Agriculture and climate change: implications for environmental sustainability indicators. WIT Trans Ecol Environ 175:99–110. https://doi.org/10.2495/ECO130091

Wijewickrama MKCS, Chileshe N, Rameezdeen R, Ochoa JJ (2021) Information sharing in reverse logistics supply chain of demolition waste: a systematic literature review. J Clean Prod 280:124359. https://doi.org/10.1016/j.jclepro.2020.124359

Woodhouse A, Davis J, Pénicaud C, Östergren K (2018) Sustainability checklist in support of the design of food processing. Sustain Prod Consum 16:110–120. https://doi.org/10.1016/j.spc.2018.06.008

Wu R, Yang D, Chen J (2014) Social Life Cycle Assessment Revisited Sustain 6:4200–4226. https://doi.org/10.3390/su6074200

Yadav S, Luthra S, Garg D (2021) Modelling Internet of things (IoT)-driven global sustainability in multi-tier agri-food supply chain under natural epidemic outbreaks. Environ Sci Pollut Res 16633–16654. https://doi.org/10.1007/s11356-020-11676-1

Yee FM, Shaharudin MR, Ma G et al (2021) Green purchasing capabilities and practices towards Firm’s triple bottom line in Malaysia. J Clean Prod 307:127268. https://doi.org/10.1016/j.jclepro.2021.127268

Yigitcanlar T (2010) Rethinking sustainable development: urban management, engineering, and design. IGI Global

Zamagni A, Amerighi O, Buttol P (2011) Strengths or bias in social LCA? Int J Life Cycle Assess 16:596–598. https://doi.org/10.1007/s11367-011-0309-3

Download references

Author information

Authors and affiliations.

Department of Economy, Engineering, Society and Business Organization, University of “Tuscia, ” Via del Paradiso 47, 01100, Viterbo, VT, Italy

Cecilia Silvestri, Michela Piccarozzi & Alessandro Ruggieri

Department of Engineering, University of Rome “Niccolò Cusano, ” Via Don Carlo Gnocchi, 3, 00166, Rome, Italy

Luca Silvestri

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Cecilia Silvestri .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Communicated by Monia Niero

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: a number of ill-placed paragraph headings were removed and the source indication "Authors' elaborations" was added to Tables 1-3.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 31 KB)

Rights and permissions.

Reprints and permissions

About this article

Silvestri, C., Silvestri, L., Piccarozzi, M. et al. Toward a framework for selecting indicators of measuring sustainability and circular economy in the agri-food sector: a systematic literature review. Int J Life Cycle Assess (2022). https://doi.org/10.1007/s11367-022-02032-1

Download citation

Received : 15 June 2021

Accepted : 16 February 2022

Published : 02 March 2022

DOI : https://doi.org/10.1007/s11367-022-02032-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Agri-food sector
• Sustainability
• Circular economy
• Triple bottom line
• Life cycle assessment
• Find a journal
• Publish with us
• Track your research

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 24, Issue 2
• Five tips for developing useful literature summary tables for writing review articles
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-0157-5319 Ahtisham Younas 1 , 2 ,
• http://orcid.org/0000-0002-7839-8130 Parveen Ali 3 , 4
• 1 Memorial University of Newfoundland , St John's , Newfoundland , Canada
• 2 Swat College of Nursing , Pakistan
• 3 School of Nursing and Midwifery , University of Sheffield , Sheffield , South Yorkshire , UK
• 4 Sheffield University Interpersonal Violence Research Group , Sheffield University , Sheffield , UK
• Correspondence to Ahtisham Younas, Memorial University of Newfoundland, St John's, NL A1C 5C4, Canada; ay6133{at}mun.ca

https://doi.org/10.1136/ebnurs-2021-103417

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Literature reviews offer a critical synthesis of empirical and theoretical literature to assess the strength of evidence, develop guidelines for practice and policymaking, and identify areas for future research. 1 It is often essential and usually the first task in any research endeavour, particularly in masters or doctoral level education. For effective data extraction and rigorous synthesis in reviews, the use of literature summary tables is of utmost importance. A literature summary table provides a synopsis of an included article. It succinctly presents its purpose, methods, findings and other relevant information pertinent to the review. The aim of developing these literature summary tables is to provide the reader with the information at one glance. Since there are multiple types of reviews (eg, systematic, integrative, scoping, critical and mixed methods) with distinct purposes and techniques, 2 there could be various approaches for developing literature summary tables making it a complex task specialty for the novice researchers or reviewers. Here, we offer five tips for authors of the review articles, relevant to all types of reviews, for creating useful and relevant literature summary tables. We also provide examples from our published reviews to illustrate how useful literature summary tables can be developed and what sort of information should be provided.

Tip 1: provide detailed information about frameworks and methods

• Download figure
• Open in new tab
• Download powerpoint

Tabular literature summaries from a scoping review. Source: Rasheed et al . 3

The provision of information about conceptual and theoretical frameworks and methods is useful for several reasons. First, in quantitative (reviews synthesising the results of quantitative studies) and mixed reviews (reviews synthesising the results of both qualitative and quantitative studies to address a mixed review question), it allows the readers to assess the congruence of the core findings and methods with the adapted framework and tested assumptions. In qualitative reviews (reviews synthesising results of qualitative studies), this information is beneficial for readers to recognise the underlying philosophical and paradigmatic stance of the authors of the included articles. For example, imagine the authors of an article, included in a review, used phenomenological inquiry for their research. In that case, the review authors and the readers of the review need to know what kind of (transcendental or hermeneutic) philosophical stance guided the inquiry. Review authors should, therefore, include the philosophical stance in their literature summary for the particular article. Second, information about frameworks and methods enables review authors and readers to judge the quality of the research, which allows for discerning the strengths and limitations of the article. For example, if authors of an included article intended to develop a new scale and test its psychometric properties. To achieve this aim, they used a convenience sample of 150 participants and performed exploratory (EFA) and confirmatory factor analysis (CFA) on the same sample. Such an approach would indicate a flawed methodology because EFA and CFA should not be conducted on the same sample. The review authors must include this information in their summary table. Omitting this information from a summary could lead to the inclusion of a flawed article in the review, thereby jeopardising the review’s rigour.

Tip 2: include strengths and limitations for each article

Critical appraisal of individual articles included in a review is crucial for increasing the rigour of the review. Despite using various templates for critical appraisal, authors often do not provide detailed information about each reviewed article’s strengths and limitations. Merely noting the quality score based on standardised critical appraisal templates is not adequate because the readers should be able to identify the reasons for assigning a weak or moderate rating. Many recent critical appraisal checklists (eg, Mixed Methods Appraisal Tool) discourage review authors from assigning a quality score and recommend noting the main strengths and limitations of included studies. It is also vital that methodological and conceptual limitations and strengths of the articles included in the review are provided because not all review articles include empirical research papers. Rather some review synthesises the theoretical aspects of articles. Providing information about conceptual limitations is also important for readers to judge the quality of foundations of the research. For example, if you included a mixed-methods study in the review, reporting the methodological and conceptual limitations about ‘integration’ is critical for evaluating the study’s strength. Suppose the authors only collected qualitative and quantitative data and did not state the intent and timing of integration. In that case, the strength of the study is weak. Integration only occurred at the levels of data collection. However, integration may not have occurred at the analysis, interpretation and reporting levels.

Tip 3: write conceptual contribution of each reviewed article

While reading and evaluating review papers, we have observed that many review authors only provide core results of the article included in a review and do not explain the conceptual contribution offered by the included article. We refer to conceptual contribution as a description of how the article’s key results contribute towards the development of potential codes, themes or subthemes, or emerging patterns that are reported as the review findings. For example, the authors of a review article noted that one of the research articles included in their review demonstrated the usefulness of case studies and reflective logs as strategies for fostering compassion in nursing students. The conceptual contribution of this research article could be that experiential learning is one way to teach compassion to nursing students, as supported by case studies and reflective logs. This conceptual contribution of the article should be mentioned in the literature summary table. Delineating each reviewed article’s conceptual contribution is particularly beneficial in qualitative reviews, mixed-methods reviews, and critical reviews that often focus on developing models and describing or explaining various phenomena. Figure 2 offers an example of a literature summary table. 4

Tabular literature summaries from a critical review. Source: Younas and Maddigan. 4

Tip 4: compose potential themes from each article during summary writing

While developing literature summary tables, many authors use themes or subthemes reported in the given articles as the key results of their own review. Such an approach prevents the review authors from understanding the article’s conceptual contribution, developing rigorous synthesis and drawing reasonable interpretations of results from an individual article. Ultimately, it affects the generation of novel review findings. For example, one of the articles about women’s healthcare-seeking behaviours in developing countries reported a theme ‘social-cultural determinants of health as precursors of delays’. Instead of using this theme as one of the review findings, the reviewers should read and interpret beyond the given description in an article, compare and contrast themes, findings from one article with findings and themes from another article to find similarities and differences and to understand and explain bigger picture for their readers. Therefore, while developing literature summary tables, think twice before using the predeveloped themes. Including your themes in the summary tables (see figure 1 ) demonstrates to the readers that a robust method of data extraction and synthesis has been followed.

Tip 5: create your personalised template for literature summaries

Often templates are available for data extraction and development of literature summary tables. The available templates may be in the form of a table, chart or a structured framework that extracts some essential information about every article. The commonly used information may include authors, purpose, methods, key results and quality scores. While extracting all relevant information is important, such templates should be tailored to meet the needs of the individuals’ review. For example, for a review about the effectiveness of healthcare interventions, a literature summary table must include information about the intervention, its type, content timing, duration, setting, effectiveness, negative consequences, and receivers and implementers’ experiences of its usage. Similarly, literature summary tables for articles included in a meta-synthesis must include information about the participants’ characteristics, research context and conceptual contribution of each reviewed article so as to help the reader make an informed decision about the usefulness or lack of usefulness of the individual article in the review and the whole review.

In conclusion, narrative or systematic reviews are almost always conducted as a part of any educational project (thesis or dissertation) or academic or clinical research. Literature reviews are the foundation of research on a given topic. Robust and high-quality reviews play an instrumental role in guiding research, practice and policymaking. However, the quality of reviews is also contingent on rigorous data extraction and synthesis, which require developing literature summaries. We have outlined five tips that could enhance the quality of the data extraction and synthesis process by developing useful literature summaries.

• Aromataris E ,
• Rasheed SP ,

Twitter @Ahtisham04, @parveenazamali

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient consent for publication Not required.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

• Short report
• Open access
• Published: 12 April 2024

A modified action framework to develop and evaluate academic-policy engagement interventions

• Petra Mäkelä   ORCID: orcid.org/0000-0002-0938-1175 1 ,
• Annette Boaz   ORCID: orcid.org/0000-0003-0557-1294 2 &
• Kathryn Oliver   ORCID: orcid.org/0000-0002-4326-5258 1  

Implementation Science volume  19 , Article number:  31 ( 2024 ) Cite this article

Metrics details

There has been a proliferation of frameworks with a common goal of bridging the gap between evidence, policy, and practice, but few aim to specifically guide evaluations of academic-policy engagement. We present the modification of an action framework for the purpose of selecting, developing and evaluating interventions for academic-policy engagement.

We build on the conceptual work of an existing framework known as SPIRIT (Supporting Policy In Health with Research: an Intervention Trial), developed for the evaluation of strategies intended to increase the use of research in health policy. Our aim was to modify SPIRIT, (i) to be applicable beyond health policy contexts, for example encompassing social, environmental, and economic policy impacts and (ii) to address broader dynamics of academic-policy engagement. We used an iterative approach through literature reviews and consultation with multiple stakeholders from Higher Education Institutions (HEIs) and policy professionals working at different levels of government and across geographical contexts in England, alongside our evaluation activities in the Capabilities in Academic Policy Engagement (CAPE) programme.

Our modifications expand upon Redman et al.’s original framework, for example adding a domain of ‘Impacts and Sustainability’ to capture continued activities required in the achievement of desirable outcomes. The modified framework fulfils the criteria for a useful action framework, having a clear purpose, being informed by existing understandings, being capable of guiding targeted interventions, and providing a structure to build further knowledge.

The modified SPIRIT framework is designed to be meaningful and accessible for people working across varied contexts in the evidence-policy ecosystem. It has potential applications in how academic-policy engagement interventions might be developed, evaluated, facilitated and improved, to ultimately support the use of evidence in decision-making.

Peer Review reports

Contributions to the literature

There has been a proliferation of theories, models and frameworks relating to translation of research into practice. Few specifically relate to engagement between academia and policy.

Challenges of evidence-informed policy-making are receiving increasing attention globally. There is a growing number of academic-policy engagement interventions but a lack of published evaluations.

This article contributes a modified action framework that can be used to guide how academic-policy engagement interventions might be developed, evaluated, facilitated, and improved, to support the use of evidence in policy decision-making.

Our contribution demonstrates the potential for modification of existing, useful frameworks instead of creating brand-new frameworks. It provides an exemplar for others who are considering when and how to modify existing frameworks to address new or expanded purposes while respecting the conceptual underpinnings of the original work.

Academic-policy engagement refers to ways that Higher Education Institutions (HEIs) and their staff engage with institutions responsible for policy at national, regional, county or local levels. Academic-policy engagement is intended to support the use of evidence in decision-making and in turn, improve its effectiveness, and inform the identification of barriers and facilitators in policy implementation [ 1 , 2 , 3 ]. Challenges of evidence-informed policy-making are receiving increasing attention globally, including the implications of differences in cultural norms and mechanisms across national contexts [ 4 , 5 ]. Although challenges faced by researchers and policy-makers have been well documented [ 6 , 7 ], there has been less focus on actions at the engagement interface. Pragmatic guidance for the development, evaluation or comparison of structured responses to the challenges of academic-policy engagement is currently lacking [ 8 , 9 ].

Academic-policy engagement exists along a continuum of approaches from linear (pushing evidence out from academia or pulling evidence into policy), relational (promoting mutual understandings and partnerships), and systems approaches (addressing identified barriers and facilitators) [ 4 ]. Each approach is underpinned by sets of beliefs, assumptions and expectations, and each raises questions for implementation and evaluation. Little is known about which academic-policy engagement interventions work in which settings, with scarce empirical evidence to inform decisions about which interventions to use, when, with whom, or why, and how organisational contexts can affect motivation and capabilities for such engagement [ 10 ]. A deeper understanding through the evaluation of engagement interventions will help to identify inhibitory and facilitatory factors, which may or may not transfer across contexts [ 11 ].

The intellectual technologies [ 12 ] of implementation science have proliferated in recent decades, including models, frameworks and theories that address research translation and acknowledge difficulties in closing the gap between research, policy and practice [ 13 ]. Frameworks may serve overlapping purposes of describing or guiding processes of translating knowledge into practice (e.g. the Quality Implementation Framework [ 14 ]); or helping to explain influences on implementation outcomes (e.g. the Theoretical Domains Framework [ 15 ]); or guiding evaluation (e.g. the RE-AIM framework [ 16 , 17 ]. Frameworks can offer an efficient way to look across diverse settings and to identify implementation differences [ 18 , 19 ]. However, the abundance of options raises its own challenges when seeking a framework for a particular purpose, and the use of a framework may mean that more weight is placed on certain aspects, leading to a partial understanding [ 13 , 17 ].

‘Action frameworks’ are predictive models that intend to organise existing knowledge and enable a logical approach for the selection, implementation and evaluation of intervention strategies, thereby facilitating the expansion of that knowledge [ 20 ]. They can guide change by informing and clarifying practical steps to follow. As flexible entities, they can be adapted to accommodate new purposes. Framework modification may include the addition of constructs or changes in language to expand applicability to a broader range of settings [ 21 ].

We sought to identify one organising framework for evaluation activities in the Capabilities in Academic-Policy Engagement (CAPE) programme (2021–2023), funded by Research England. The CAPE programme aimed to understand how best to support effective and sustained engagement between academics and policy professionals across the higher education sector in England [ 22 ]. We first searched the literature and identified an action framework that was originally developed between 2011 and 2013, to underpin a trial known as SPIRIT (Supporting Policy In health with Research: an Intervention Trial) [ 20 , 23 ]. This trial evaluated strategies intended to increase the use of research in health policy and to identify modifiable points for intervention.

We selected the SPIRIT framework due to its potential suitability as an initial ‘road map’ for our evaluation of academic-policy interventions in the CAPE programme. The key elements of the original framework are catalysts, organisational capacity, engagement actions, and research use. We wished to build on the framework’s embedded conceptual work, derived from literature reviews and semi-structured interviews, to identify policymakers’ views on factors that assist policy agencies’ use of research [ 20 ]. The SPIRIT framework developers defined its “locus for change” as the policy organisation ( [ 20 ], p. 151). They proposed that it could offer the beginning of a process to identify and test pathways in policy agencies’ use of evidence.

Our goal was to modify SPIRIT to accommodate a different locus for change: the engagement interface between academia and policy. Instead of imagining a linear process in which knowledge comes from researchers and is transmitted to policy professionals, we intended to extend the framework to multidirectional relational and system interfaces. We wished to include processes and influences at individual, organisational and system levels, to be relevant for HEIs and their staff, policy bodies and professionals, funders of engagement activities, and facilitatory bodies. Ultimately, we seek to address a gap in understanding how engagement strategies work, for whom, how they are facilitated, and to improve the evaluation of academic-policy engagement.

We aimed to produce a conceptually guided action framework to enable systematic evaluation of interventions intending to support academic-policy engagement.

We used a pragmatic combination of processes for framework modification during our evaluation activities in the CAPE programme [ 22 ]. The CAPE programme included a range of interventions: seed funding for academic and policy professional collaboration in policy-focused projects, fellowships for academic placements in policy settings, or for policy professionals with HEI staff, training for policy professionals, and a range of knowledge exchange events for HEI staff and policy professionals. We modified the SPIRIT framework through iterative processes shown in Table  1 , including reviews of literature; consultations with HEI staff and policy professionals across a range of policy contexts and geographic settings in England, through the CAPE programme; and piloting, refining and seeking feedback from stakeholders in academic-policy engagement.

A number of characteristics of the original SPIRIT framework could be applied to academic-policy engagement. While keeping the core domains, we modified the framework to capture dynamics of engagement at multiple academic and policy levels (individuals, organisations and system), extending beyond the original unidirectional focus on policy agencies’ use of research. Components of the original framework, the need for modifications, and their corresponding action-oriented implications are shown in Table  2 . We added a new domain, ‘Impacts and Sustainability’, to consider transforming and enduring aspects at the engagement interface. The modified action framework is shown in Fig.  1 .

SPIRIT Action Framework Modified for Academic-Policy Engagement Interventions (SPIRIT-ME), adapted with permission from the Sax Institute. Legend: The framework acknowledges that elements in each domain may influence other elements through mechanisms of action and that these do not necessarily flow through the framework in a ‘pipeline’ sequence. Mechanisms of action are processes through which engagement strategies operate to achieve desired outcomes. They might rely on influencing factors, catalysts, an aspect of an intervention action, or a combination of elements

Identifying relevant theories or models for missing elements

Catalysts and capacity.

Within our evaluation of academic-policy interventions, we identified a need to develop the original domain of catalysts beyond ‘policy/programme need for research’ and ‘new research with potential policy relevance’. Redman et al. characterised a catalyst as “a need for information to answer a particular problem in policy or program design, or to assist in supporting a case for funding” in the original framework (p. 149). We expanded this “need for information” to a perceived need for engagement, by either HEI staff or policy professionals, linking to the potential value they perceived in engaging. Specifically, there was a need to consider catalysts at the level of individual engagement, for example HEI staff wanting research to have real-world impact, or policy professionals’ desires to improve decision-making in policy, where productive interactions between academic and policy stakeholders are “necessary interim steps in the process that lead to societal impact” ( [ 24 ], p. 214). The catalyst domain expands the original emphasis on a need for research, to take account of challenges to be overcome by both the academic and policy communities in knowing how, and with whom, to engage and collaborate with [ 25 ].

We used a model proposing that there are three components for any behaviour: capability, opportunity and motivation, which is known as the COM-B model [ 26 ]. Informed by CAPE evaluation activities and our discussions with stakeholders, we mapped the opportunity and motivation constructs into the ‘catalysts’ domain of the original framework. Opportunity is an attribute of the system that can facilitate engagement. It may be a tangible factor such as the availability of seed funding, or a perceived social opportunity such as institutional support for engagement activities. Opportunity can act at the macro level of systems and organisational structures. Motivation acts at the micro level, deriving from an individual’s mental processes that stimulate and direct their behaviours; in this case, taking part in academic-policy engagement actions. The COM-B model distinguishes between reflective motivation through conscious planning and automatic motivation that may be instinctive or affective [ 26 ].

We presented an early application of the COM-B model to catalysts for engagement at an academic conference, enabling an informal exploration of attendees’ subjective views on the clarity and appropriateness, when developing the framework. This application introduces possibilities for intervention development and support by highlighting ‘opportunities’ and ‘motivations’ as key catalysts in the modified framework.

Within the ‘capacity’ domain, we retained the original levels of individuals, organisations and systems. We introduced individual capability as a construct from the COM-B model, describing knowledge, skills and abilities to generate behaviour change as a precursor of academic-policy engagement. This reframing extends the applicability to HEI staff as well as policy professionals. It brings attention to different starting conditions for individuals, such as capabilities developed through previous experience, which can link with social opportunity (for example, through training or support) as a catalyst.

Engagement actions

We identified a need to modify the original domain ‘engagement actions’ to extend the focus beyond the use of research. We added three categories of engagement actions described by Best and Holmes [ 27 ]: linear, relational, and systems. These categories were further specified through a systematic mapping of international organisations’ academic-policy engagement activities [ 5 ]. This framework modification expands the domain to encompass: (i) linear ‘push’ of evidence from academia or ‘pull’ of evidence into policy agencies; (ii) relational approaches focused on academic-policy-maker collaboration; and (iii) systems’ strategies to facilitate engagement for example through strategic leadership, rewards or incentives [ 5 ].

We retained the elements in the original framework’s ‘outcomes’ domain (instrumental, tactical, conceptual and imposed), which we found could apply to outcomes of engagement as well as research use. For example, discussions between a policy professional and a range of academics could lead to a conceptual outcome by considering an issue through different disciplinary lenses. We expanded these elements by drawing on literature on engagement outcomes [ 28 ] and through sense-checking with stakeholders in CAPE. We added capacity-building (changes to skills and expertise), connectivity (changes to the number and quality of relationships), and changes in organisational culture or attitude change towards engagement.

Impacts and sustainability

The original framework contained endpoints described as: ‘Better health system and health outcomes’ and ‘Research-informed health policy and policy documents’. For modification beyond health contexts and to encompass broader intentions of academic-policy engagement, we replaced these elements with a new domain of ‘Impacts and sustainability’. This domain captures the continued activities required in achievement of desirable outcomes [ 29 ]. The modification allows consideration of sustainability in relation to previous stages of engagement interventions, through the identification of beneficial effects that are sustained (or not), in which ways, and for whom. Following Borst [ 30 ], we propose a shift from the expectation that ‘sustainability’ will be a fixed endpoint. Instead, we emphasise the maintenance work needed over time, to sustain productive engagement.

Influences and facilitators

We modified the overarching ‘Policy influences’ (such as public opinion and media) in the original framework, to align with factors influencing academic-policy engagement beyond policy agencies’ use of research. We included influences at the level of the individual (for example, individual moral discretion [ 31 ]), the organisation (for example, managerial practices [ 31 ]) and the system (for example, career incentives [ 32 ]). Each of these processes takes place in the broader context of social, policy and financial environments (that is, potential sources of funding for engagement actions) [ 29 ].

We modified the domain ‘Reservoir of relevant and reliable research’ underpinning the original framework, replacing it with ‘Reservoir of people skills’, to emphasise intangible facilitatory work at the engagement interface, in place of concrete research outputs. We used the ‘Promoting Action on Research Implementation in Health Services’ (PARiHS) framework [ 33 , 34 ], which gives explicit consideration to facilitation mechanisms for researchers and policy-makers [ 13 ] . Here, facilitation expertise includes mechanisms that focus on particular goals (task-oriented facilitation) or enable changes in ways of working (holistic-oriented facilitation). Task-orientated facilitation skills might include, for example, the provision of contacts, practical help or project management skills, while holistic-oriented facilitation involves building and sustaining partnerships or support skills’ development across a range of capabilities. These conceptualisations aligned with our consultations with facilitators of engagement in CAPE. We further extended these to include aspects identified in our evaluation activities: strategic planning, contextual awareness and entrepreneurial orientation.

Piloting and refining the modified framework through stakeholder engagement

We piloted an early version of the modified framework to develop a survey for all CAPE programme participants. During this pilot stage, we sought feedback from the CAPE delivery team members across HEI and policy contexts in England. CAPE delivery team members are based at five collaborating universities with partners in the Parliamentary Office for Science and Technology (POST) and Government Office for Science (GO-Science), and Nesta (a British foundation that supports innovation). The HEI members include academics and professional services knowledge mobilisation staff, responsible for leading and coordinating CAPE activities. The delivery team comprised approximately 15–20 individuals (with some fluctuations according to individual availabilities).

We assessed appropriateness and utility, refined terminology, added domain elements and explored nuances. For example, stakeholders considered the multi-layered possibilities within the domain ‘capacity’, where some HEI or policy departments may demonstrate a belief that it is important to use research in policy, but this might not be the perception of the organisation as a whole. We also sought stakeholders’ views on the utility of the new domains, for example, the identification of facilitator expertise such as acting as a knowledge broker or intermediary; providing training, advice or guidance; facilitating engagement opportunities; creating engagement programmes; and sustainability of engagement that could be conceptualised at multiple levels: personally, in processes or through systems.

Testing against criteria for useful action framework

The modified framework fulfils the properties of a useful action framework [ 20 ]:

It has a clearly articulated purpose: development and evaluation of academic-policy engagement interventions through linear, relational and/or system approaches. It has identified loci for change, at the level of the individual, the organisation or system.

It has been informed by existing understandings, including conceptual work of the original SPIRIT framework, conceptual models identified from the literature, published empirical findings, understandings from consultation with stakeholders, and evaluation activities in CAPE.

It can be applied to the development, implementation and evaluation of targeted academic-policy engagement actions, the selection of points for intervention and identification of potential outcomes, including the work of sustaining them and unanticipated consequences.

It provides a structure to build knowledge by guiding the generation of hypotheses about mechanisms of action in academic-policy engagement interventions, or by adapting the framework further through application in practice.

The proliferation of frameworks to articulate processes of research translation reveals a need for their adaptation when applied in specific contexts. The majority of models in implementation science relate to translation of research into practice. By contrast, our focus was on engagement between academia and policy. There are a growing number of academic-policy engagement interventions but a lack of published evaluations [ 10 ].

Our framework modification provides an exemplar for others who are considering how to adapt existing conceptual frameworks to address new or expanded purposes. Field et al. identified the multiple, idiosyncratic ways that the Knowledge to Action Framework has been applied in practice, demonstrating its ‘informal’ adaptability to different healthcare settings and topics [ 35 ]. Others have reported on specific processes for framework refinement or extension. Wiltsey Stirman et al. adopted a framework that characterised forms of intervention modification, using a “pragmatic, multifaceted approach” ( [ 36 ], p.2). The authors later used the modified version as a foundation to build a further framework to encompass implementation strategies in a range of settings [ 21 ]. Oiumet et al. used the approach of borrowing from a different disciplinary field for framework adaptation, by using a model of absorptive capacity from management science to develop a conceptual framework for civil servants’ absorption of research knowledge [ 37 ].

We also took the approach of “adapting the tools we think with” ( [ 38 ], p.305) during our evaluation activities on the CAPE programme. Our conceptual modifications align with the literature on motivation and entrepreneurial orientation in determining policy-makers’ and researchers’ intentions to carry out engagement in addition to ‘usual’ roles [ 39 , 40 ]. Our framework offers an enabler for academic-policy engagement endeavours, by providing a structure for approaches beyond the linear transfer of information, emphasising the role of multidirectional relational activities, and the importance of their facilitation and maintenance. The framework emphasises the relationship between individuals’ and groups’ actions, and the social contexts in which these are embedded. It offers additional value by capturing the organisational and systems level factors that influence evidence-informed policymaking, incorporating the dynamic features of contexts shaping engagement and research use.

Conclusions

Our modifications extend the original SPIRIT framework’s focus on policy agencies’ use of research, to encompass dynamic academic-policy engagement at the levels of individuals, organisations and systems. Informed by the knowledge and experiences of policy professionals, HEI staff and knowledge mobilisers, it is designed to be meaningful and accessible for people working across varied contexts and functions in the evidence-policy ecosystem. It has potential applications in how academic-policy engagement interventions might be developed, evaluated, facilitated and improved, and it fulfils Redman et al.’s criteria as a useful action framework [ 20 ].

We are testing the ‘SPIRIT-Modified for Engagement’ framework (SPIRIT-ME) through our ongoing evaluation of academic-policy engagement activities. Further empirical research is needed to explore how the framework may capture ‘additionality’, that is, to identify what is achieved through engagement actions in addition to what would have happened anyway, including long-term changes in strategic behaviours or capabilities [ 41 , 42 , 43 ]. Application of the modified framework in practice will highlight its strengths and limitations, to inform further iterative development and adaptation.

Availability of data and materials

Not applicable.

Stewart R, Dayal H, Langer L, van Rooyen C. Transforming evidence for policy: do we have the evidence generation house in order? Humanit Soc Sci Commun. 2022;9(1):1–5.

Article   Google Scholar  

Sanderson I. Complexity, ‘practical rationality’ and evidence-based policy making. Policy Polit. 2006;34(1):115–32.

Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gülmezoglu M, et al. Using Qualitative Evidence in Decision Making for Health and Social Interventions: An Approach to Assess Confidence in Findings from Qualitative Evidence Syntheses (GRADE-CERQual). PLOS Med. 2015;12(10):e1001895.

Article   PubMed   PubMed Central   Google Scholar  

Bonell C, Meiksin R, Mays N, Petticrew M, McKee M. Defending evidence informed policy making from ideological attack. BMJ. 2018;10(362):k3827.

Hopkins A, Oliver K, Boaz A, Guillot-Wright S, Cairney P. Are research-policy engagement activities informed by policy theory and evidence? 7 challenges to the UK impact agenda. Policy Des Pract. 2021;4(3):341–56.

Google Scholar  

Head BW. Toward More “Evidence-Informed” Policy Making? Public Adm Rev. 2016;76(3):472–84.

Walker LA, Lawrence NS, Chambers CD, Wood M, Barnett J, Durrant H, et al. Supporting evidence-informed policy and scrutiny: A consultation of UK research professionals. PLoS ONE. 2019;14(3):e0214136.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Graham ID, Tetroe J, Group the KT. Planned action theories. In: Knowledge Translation in Health Care. John Wiley and Sons, Ltd; 2013. p. 277–87. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1002/9781118413555.ch26 Cited 2023 Nov 1

Davies HT, Powell AE, Nutley SM. Mobilising knowledge to improve UK health care: learning from other countries and other sectors – a multimethod mapping study. Southampton (UK): NIHR Journals Library; 2015. (Health Services and Delivery Research). Available from: http://www.ncbi.nlm.nih.gov/books/NBK299400/ Cited 2023 Nov 1

Oliver K, Hopkins A, Boaz A, Guillot-Wright S, Cairney P. What works to promote research-policy engagement? Evid Policy. 2022;18(4):691–713.

Nelson JP, Lindsay S, Bozeman B. The last 20 years of empirical research on government utilization of academic social science research: a state-of-the-art literature review. Adm Soc. 2023;28:00953997231172923.

Bell D. Technology, nature and society: the vicissitudes of three world views and the confusion of realms. Am Sch. 1973;42:385–404.

Milat AJ, Li B. Narrative review of frameworks for translating research evidence into policy and practice. Public Health Res Pract. 2017; Available from: https://apo.org.au/sites/default/files/resource-files/2017-02/apo-nid74420.pdf Cited 2023 Nov 1

Meyers DC, Durlak JA, Wandersman A. The quality implementation framework: a synthesis of critical steps in the implementation process. Am J Community Psychol. 2012;50(3–4):462–80.

Article   PubMed   Google Scholar  

Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7(1):37.

Glasgow RE, Battaglia C, McCreight M, Ayele RA, Rabin BA. Making implementation science more rapid: use of the RE-AIM framework for mid-course adaptations across five health services research projects in the veterans health administration. Front Public Health. 2020;8. Available from: https://www.frontiersin.org/articles/10.3389/fpubh.2020.00194 Cited 2023 Jun 13

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci IS. 2015 Apr 21 10. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4406164/ Cited 2020 May 4

Sheth A, Sinfield JV. An analytical framework to compare innovation strategies and identify simple rules. Technovation. 2022;1(115):102534.

Birken SA, Powell BJ, Shea CM, Haines ER, Alexis Kirk M, Leeman J, et al. Criteria for selecting implementation science theories and frameworks: results from an international survey. Implement Sci. 2017;12(1):124.

Redman S, Turner T, Davies H, Williamson A, Haynes A, Brennan S, et al. The SPIRIT Action Framework: A structured approach to selecting and testing strategies to increase the use of research in policy. Soc Sci Med. 2015;136:147–55.

Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. 2021;16(1):36.

CAPE. CAPE. 2021. CAPE Capabilities in Academic Policy Engagement. Available from: https://www.cape.ac.uk/ Cited 2021 Aug 3

CIPHER Investigators. Supporting policy in health with research: an intervention trial (SPIRIT)—protocol for a stepped wedge trial. BMJ Open. 2014;4(7):e005293.

Spaapen J, Van Drooge L. Introducing ‘productive interactions’ in social impact assessment. Res Eval. 2011;20(3):211–8.

Williams C, Pettman T, Goodwin-Smith I, Tefera YM, Hanifie S, Baldock K. Experiences of research-policy engagement in policymaking processes. Public Health Res Pract. 2023. Online early publication. https://doi.org/10.17061/phrp33232308 .

Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6(1):42.

Best A, Holmes B. Systems thinking, knowledge and action: towards better models and methods. Evid Policy J Res Debate Pract. 2010;6(2):145–59.

Edwards DM, Meagher LR. A framework to evaluate the impacts of research on policy and practice: A forestry pilot study. For Policy Econ. 2020;1(114):101975.

Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101(11):2059–67.

Borst RAJ, Wehrens R, Bal R, Kok MO. From sustainability to sustaining work: What do actors do to sustain knowledge translation platforms? Soc Sci Med. 2022;1(296):114735.

Zacka B. When the state meets the street: public service and moral agency. Harvard university press; 2017. Available from: https://books.google.co.uk/books?hl=en&lr=&id=3KdFDwAAQBAJ&oi=fnd&pg=PP1&dq=zacka+when+the+street&ots=x93YEHPKhl&sig=9yXKlQiFZ0XblHrbYKzvAMwNWT4 Cited 2023 Nov 28

Torrance H. The research excellence framework in the United Kingdom: processes, consequences, and incentives to engage. Qual Inq. 2020;26(7):771–9.

Rycroft-Malone J. The PARIHS framework—a framework for guiding the implementation of evidence-based practice. J Nurs Care Qual. 2004;19(4):297–304.

Stetler CB, Damschroder LJ, Helfrich CD, Hagedorn HJ. A guide for applying a revised version of the PARIHS framework for implementation. Implement Sci. 2011;6(1):99.

Field B, Booth A, Ilott I, Gerrish K. Using the knowledge to action framework in practice: a citation analysis and systematic review. Implement Sci. 2014;9(1):172.

Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. 2019;14(1):58.

Ouimet M, Landry R, Ziam S, Bédard PO. The absorption of research knowledge by public civil servants. Evid Policy. 2009;5(4):331–50.

Martin D, Spink MJ, Pereira PPG. Multiple bodies, political ontologies and the logic of care: an interview with Annemarie Mol. Interface - Comun Saúde Educ. 2018;22:295–305.

Sajadi HS, Majdzadeh R, Ehsani-Chimeh E, Yazdizadeh B, Nikooee S, Pourabbasi A, et al. Policy options to increase motivation for improving evidence-informed health policy-making in Iran. Health Res Policy Syst. 2021;19(1):91.

Athreye S, Sengupta A, Odetunde OJ. Academic entrepreneurial engagement with weak institutional support: roles of motivation, intention and perceptions. Stud High Educ. 2023;48(5):683–94.

Bamford D, Reid I, Forrester P, Dehe B, Bamford J, Papalexi M. An empirical investigation into UK university–industry collaboration: the development of an impact framework. J Technol Transf. 2023 Nov 13; Available from: https://doi.org/10.1007/s10961-023-10043-9 Cited 2023 Dec 20

McPherson AH, McDonald SM. Measuring the outcomes and impacts of innovation interventions assessing the role of additionality. Int J Technol Policy Manag. 2010;10(1–2):137–56.

Hind J. Additionality: a useful way to construct the counterfactual qualitatively? Eval J Australas. 2010;10(1):28–35.

Download references

Acknowledgements

We are very grateful to the CAPE Programme Delivery Group members, for many discussions throughout this work. Our thanks also go to the Sax Institute, Australia (where the original SPIRIT framework was developed), for reviewing and providing helpful feedback on the article. We also thank our reviewers who made very constructive suggestions, which have strengthened and clarified our article.

The evaluation of the CAPE programme, referred to in this report, was funded by Research England. The funding body had no role in the design of the study, analysis, interpretation or writing the manuscript.

Author information

Authors and affiliations.

Department of Health Services Research and Policy, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, 15-17 Tavistock Place, Kings Cross, London, WC1H 9SH, UK

Petra Mäkelä & Kathryn Oliver

Health and Social Care Workforce Research Unit, The Policy Institute, Virginia Woolf Building, Kings College London, 22 Kingsway, London, WC2B 6LE, UK

Annette Boaz

You can also search for this author in PubMed   Google Scholar

Contributions

PM conceptualised the modification of the framework reported in this work. All authors made substantial contributions to the design of the work. PM drafted the initial manuscript. AB and KO contributed to revisions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Petra Mäkelä .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval was granted for the overarching CAPE evaluation by the London School of Hygiene and Tropical Medicine Research Ethics Committee (reference 26347).

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Mäkelä, P., Boaz, A. & Oliver, K. A modified action framework to develop and evaluate academic-policy engagement interventions. Implementation Sci 19 , 31 (2024). https://doi.org/10.1186/s13012-024-01359-7

Download citation

Received : 09 January 2024

Accepted : 20 March 2024

Published : 12 April 2024

DOI : https://doi.org/10.1186/s13012-024-01359-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Evidence-informed policy
• Academic-policy engagement
• Framework modification

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 11 April 2024

A bibliometric analysis of knowledge mapping in Chinese education digitalization research from 2012 to 2022

• Rui Shi 1 , 2 &
• XiuLan Wan 1  

Humanities and Social Sciences Communications volume  11 , Article number:  505 ( 2024 ) Cite this article

Metrics details

• Science, technology and society
• Social science

The digital transformation of education should be continuously promoted to guarantee its sustainable development. Extensive research has been conducted in this field but has not comprehensively addressed Chinese education digitalization. To fill this research gap, discover the gaps between Chinese and international research on the digitization of education and provide well-founded, innovative ideas for future research, we perform a bibliometric analysis of knowledge mapping in Chinese education digitalization. WOS and CNKI databases were used to gather literature on Chinese education digitalization research from 2012 to 2022. CiteSpace was used to draw a knowledge map of Chinese education digitalization research through co-occurrence analysis of core authors, issuing institutions and regions and cluster analysis and burst terms analysis of keywords, combined with intensive manual studying of the literature. The results show the research status and hot spots of Chinese education digitalization research are divided into four dimensions: studies of lifelong education research in digital open universities and the online teaching transformation in higher education; studies of digital educational publications, the development and application of digital learning resources in vocational colleges and universities, and the equity of basic education resources in the digital context; studies on artificial intelligence technology empowering the digital transformation of education in China; and studies of digital integration of production and teaching in rural revitalization and improvement of digital literacy of university students and faculty. Future digital education research trends in China are likely to focus on the normalization of online education; the development of online education resources in the context of new infrastructure; “new technology plus education”; the impact of digital games on education; a more diversified digital divide in education; and digital rights, digital ethics, digital maturity and the Global Digital Education Development Index.

Similar content being viewed by others

Impact of artificial intelligence on human loss in decision making, laziness and safety in education

Sayed Fayaz Ahmad, Heesup Han, … Antonio Ariza-Montes

The impact of artificial intelligence on employment: the role of virtual agglomeration

Yang Shen & Xiuwu Zhang

Artificial intelligence and illusions of understanding in scientific research

Lisa Messeri & M. J. Crockett

Introduction

The Education 2030 Framework for Action, which was issued by UNESCO in 2015, emphasizes the Sustainable Development Goals for education and proposes a direction for the future digitalization of education (UNESCO 2015 ). As a response to the United Nations 2030 Global Goals for Sustainable Development, the Organization for Economic Cooperation and Development (OECD) launched the “Education 2030: Education and Skills for the Future” project in 2015 and published “OECD: The Future of Education and Skills-Education 2030” in 2018 (OECD 2019 ). All of the education policy documents issued by these important world organizations emphasize the need to strengthen research on the digitalization of education to achieve the vision of sustainable global education in 2030.

The Outline of the Fourteenth Five-Year Plan and Vision 2035 for National Economic and Social Development of the People’s Republic of China depicts the goals and tasks of digitalized economic and social development (Ministry of Education of the People’s Republic of China 2021 ). In this process, the Central Committee of the Communist Party of China (CPC) and the State Council issued the document “China Education Modernization 2035” in 2019, which clarifies that the goal and path to achieve education modernization is the implementation of digital education (Ministry of Education of the People’s Republic of China 2018 ).

As seen from the abovementioned policy documents, both the international community and countries in particular attach great importance to the digitization of education. Indeed, the digitization of education is an important revolution in the field of education in today’s digital era; moreover, the digitization of education is also an important means of guaranteeing the sustainable development of education in the future. The study of education digitization not only follows the development trends of the digital era so that education advances with the times and adapts to the needs of the digital society but also provides new solutions and methods for addressing some of the existing problems in education, such as the uneven distribution of educational resources, the poor quality of education, etc. Through the study of education digitization, the directions, reforms and trends of the future of education can be explored and improved. Therefore, the digitalization of education needs to be examined.

Literature review and research questions

Current status of research on “education digitalization”, current status of research on “international education digitalization”.

At present, the international literature on the digitalization of education has been more complete and richer from both the specific and the comprehensive perspectives.

A review of the literature shows that, from a specific perspective, international research on digitalization in education mainly focuses on topics such as research related to teachers’ and students’ digital literacy (Audrin and Audrin 2022 ), research on the impact of video games on education (Boyle et al. 2016 ), and research on equity in education under digital teaching (Kerras et al. 2020 ).

From a comprehensive perspective, numerous authors have provided systematic overviews of topics related to the “digitization of education”. For example, through a comprehensive econometric analysis of the WOS literature, we observe that the hot topics of international research on the digitization of higher education include the popularization and development of online education, the rise and development of massive open online courses (MOOCs), the construction and sharing of digital learning resources, the innovation and change in distance education, and the impact and effect of digital transformation on the field of higher education (Díaz-García et al. 2022 ). As another example, through the visual analysis of international online learning research, it is concluded that the main topics of international online education research include MOOC, flipped classrooms, COVID-19, computer-supported collaborative learning, the technology acceptance model, the community of inquiry, and distance learning (Shen et al. 2022 ). Furthermore, a literature review of international digital campus research has revealed that ICT plays a crucial role in the process of the digital transformation of campuses, revealing the factors that schools should consider to achieve effective and efficient digital change (Timotheou et al. 2023 ). These review papers provide a panoramic view of the extent of international research on the “digitization of education” and future research directions.

Current status of research on “Chinese education digitalization”

In recent years, due to the continuing development of digital technology, digital education has been involved at all school levels in China. Many studies have discussed how to realize the digital transformation of Chinese education. For example, studies have proposed that the development path of digital education transformation in China must be supported by a perfect “guarantee mechanism” as the basic conditions and then start from “things”, “people”, and “numbers” and aim at the “digital aspect” and the two major business application areas of “teaching” and “management” to realize the comprehensive digital transformation of education with high quality (Wu et al. 2022 ).

However, the current literature on the digitalization of education in China is still mainly from specific perspectives, focusing on various types of digitalization topics in specific areas, e.g., digital literacy (Huang 2015 ) and digital technology (Zhao 2021 ).

Moreover, no comprehensive and systematic compendium of research on the digitization of education in China has yet been conducted from a global perspective. Most of the relevant studies rely on the subjective opinions of researchers and lack objective bibliometric and quantitative analyses. Although there is much literature on the digitization of education in China, there is a gap in terms of panoramic studies that analyze the digitization of education in China from a comprehensive and objective perspective.

Research questions and research implications

Our study attempts to use CiteSpace to visualize and analyze the literature on education digitization. By comprehensively searching through the research on education digitization in China, we present a panoramic view of the current status of research on education digitization in China, research hotspots and future research trends and reveal the far-reaching impact of digital education on China’s education career.

This study can not only fill the gap of comprehensive research on the digitization of education in China but also reveal the differences or gaps between the research on the digitization of education in China and that in the international arena by comparing the current situation and hotspots of the research on this topic in the international arena horizontally to provide well-founded and innovative ideas for future research on the digitization of education in China.

Methodology

This study uses a combination of bibliometric analysis and manual reading to analyze the “Digitalization of Education Research in China, 2012–2022”. Specifically, this study seeks to uncover the current status, hotspots, and future research trends of educational digitization research in China during this period. Figures 1 and 2 show the process of the research.

The research processes 1.

The research processes 2.

Selection of databases

The first step of this study was to identify the databases used. Two databases were selected for this study, namely, the Web of Science (WOS) Core Collection and the China National Knowledge Infrastructure (CNKI).

To account for the possible emergence of foreign literature related to the “digitization of education in China”, the internationally recognized Web of Science Core Collection database was selected for this study. The database was chosen for the following two reasons:

WOS is a multidisciplinary large-scale database that includes information from hundreds of national and regional institutions worldwide and has the three most authoritative citation index databases (SCI-E, SSCI, A&HCI). CiteSpace software supports the import of literature from WOS.

For the Chinese literature related to the “digitization of education in China”, this study employed the China Knowledge Network (CNKI) database, which is the largest and most authoritative database in China, covering all types of literature in China. CiteSpace also supports the import of CNKI documents. Since China was chosen as the target country for educational digitization research for this study, CNKI is a very important database.

Data collection

After WOS and CNKI were selected as the databases for Chinese education digitization research for this study, a search for relevant literature in these two databases was conducted.

Literature search strategy

Literature search strategies for wos core collection.

The WOS Core Collection was searched with the theme of “Chinese education digitalization” or “digital education in China”. The period of 2012–2022 is a typical period for the rapid development of education digitization in China. Therefore, this study utilizes the literature from the databases within the time span of 2012.1.1–2022.12.31, and 452 works were retrieved.

Literature search strategy for CNKI

In the CNKI database, since almost all published authors are Chinese scholars, most of the literature on the topic of “education digitalization” in the database is research literature on “education digitalization in China”. Therefore, the keywords or themes of “education digitalization” or “digital education” were used to search the literature in the CNKI database within the period of 2012.1.1–2022.12.31, and 1963 works were retrieved.

Inclusion and exclusion criteria

Inclusion and exclusion criteria for wos core collection.

First, of the literature uncovered in the literature search strategy, the works that were not of the document type ARTICLE had to be excluded.

Second, on this basis, the categories were refined to “Education or Educational Research” or “ Psychology Multidisciplinary “ or “Humanities Multidisciplinary”, while other categories irrelevant to the education field like “environmental science “ were excluded, and a total of 96 articles were retrieved. Each of these articles is peer-reviewed and contains information such as title, authors, abstract, keywords, references and source publications.

Third, the 96 papers retrieved were manually read to determine whether they were indeed in line with the theme of “Digitization of Education in China” based on their titles, keywords and abstracts. Through manual screening, finally, a total of 44 relevant studies that met the above search strategy and fulfilled the inclusion criteria were finally obtained.

Inclusion and exclusion criteria for CNKI

First, works about education digitalization in other countries or international organizations were excluded. For example, a study titled “Digital Transformation of Education in the European Union: A Study of Policy Evolution, Key Initiatives, and Implications” was excluded. Since the percentage of this literature was very low, such studies were manually excluded.

Furthermore, any studies in nonthesis form were excluded.

Finally, due to the limitation of the type of literature that can be imported into CiteSpace, the relevant dissertations were further excluded on this basis, while peer-reviewed and publicly published core papers were selected and a total of 425 valid papers were obtained.

In this study, CiteSpace V.6.1. R6, a visualization and analysis software developed by Professor Chao-Mei Chen of Drexel University, USA, was used to process the data from CNKI.

CiteSpace is an effective tool for quickly accessing knowledge on specific topics, and its research includes the visualization of information and the analysis of knowledge graphs and atlases of scientific frontiers. In this study, the processing conditions were set as follows: the time range was 2012–2022, divided into 11 time zones; the term type was set as burst terms; and the node type was set as keyword.

Because CiteSpace has certain requirements on the imported papers and databases: the number of papers is generally 400–3000 (not including dissertations), and the papers in Chinese and English databases need to be imported separately. Therefore, according to the CiteSpace import requirements, this study chose to import 425 documents from CNKI into CiteSpace; the remaining 44 documents from WOS were not suitable for importing into CiteSpace for analysis because the number of articles was too small, and these 44 documents from WOS were manually studied.

The selected 425 documents were imported into CiteSpace, and then the keywords extracted from the imported files were subjected to co-occurrence analysis, cluster analysis and burst analysis. After the relevant knowledge map was obtained, combined with 44 documents from WOS database manually studied, the current status, hotspots and future research trends of education digitalization research in China during the period of 2012–2022 were examined through Excel integration and analysis of specific documents.

Results and discussions

Results and discussions of research status and hotspots.

A bibliometric analysis of research institutions and research authors using CiteSpace revealed that the research institutions and scholars of “Digitization of Education in China” are mainly Chinese scholars led by Prof. Zhu Zhiting at East China Normal University in Shanghai, China, as depicted in Figs. 3 and 4 .

Distribution of institutes.

Distribution of core author.

Then, the word frequency statistics, co-occurrence analysis and cluster analysis of keywords for the papers imported into CiteSpace can be organized in Figs. 5 and 6 .

The co-linear analysis of the literature related to the digitalization of education in China.

Cluster analysis of keywords.

Combined with manual studying of the 44 papers from WOS, the following conclusions may be made on the research status and hotspots of Chinese education digitalization.

According to The Education Informatization 2.0 Action Plan, an important document on the digitization of education issued by China’s Ministry of Education, outlines the elements of the digitization of education in the following four major areas:

Digital educational resources: These include digital teaching materials, books, courseware, cases, etc., as well as a variety of online courses, web courses and other resources.

Digital educational platforms: These include online education, distance education, mobile education, and smart education platforms, as well as a variety of online learning communities and online communication and interaction platforms.

Digital educational technology: This includes a variety of digital, information, and network technologies and other means, such as AI, big data, cloud computing and IoT.

Digital educational management: This includes the management of digital teaching and of assessing the digital literacy of teachers and students, as well as various digital education and teaching assessments and quality monitoring (Ministry of Education of the People’s Republic of China 2018 ).

On this basis, an analytical framework of the ELEMENTS of educational digitization was designed in our study. Based on a combination of the above charts and manual studying, the research status and hotspots of education digitization research in China during the period of 2012–2022 were organized, as showed in Fig. 7 .

Conclusions on the status and hotspots of Chinese education digitalization research.

Research on the platform dimension of Chinese education digitalization

Digital educational platforms are the carriers of digital education. Such platforms provide a place for the integration, sharing and exchange of resources for education digitization, and enable the effective management and efficient use of educational resources. Research on China’s education digital platforms can improve the teaching efficiency at all school levels and promote the sharing of teaching resources and educational equity.

A study of lifelong education in a digitized open university

Open universities constitute a new type of university with Chinese characteristics that mainly serves members of society and provides them with continuing education services (Yu et al. 2014 ). From 2012 to 2020, there has been much research on digital education in China’s open universities, which is closely related to a series of transformation and upgrading policies issued by China regarding the development of these universities during the period. In July 2012, National Open University was inaugurated, and pilot construction began to advance. In September 2020, China’s Ministry of Education issued the Comprehensive Reform Plan for the National Open University, and the digitalization of the open university reform was fully rolled out.

A digitized open university can not only build a lifelong learning service system for all people and thus promote the interconnection and sharing of various types of education but also enable the open university to better serve its base. For example, a digitized open university can build high-quality online course resources for nondegree education for rural areas, communities and industrial enterprises, focusing on the needs of new farmers, new citizens and elderly individuals, to bridge the digital divide between urban and rural areas and regions.

The literature published during this period mainly focuses on how to realize the digital transformation of China’s open universities. The strategies for this transformation can be summarized as follows: absorbing colleges and universities, research and consulting institutions, industrial enterprises and employers to participate in the construction of online course resources; introducing high-quality digital education resources from home and abroad to enrich the online resource library of the open university; perfecting the pricing of resources to form a sustainable and market-oriented resource supply and demand mechanism; cultivating special majors and courses, promoting the learning modes of online and offline blended learning and interdisciplinary integration, strengthening the management of the learning process, reforming the assessment and evaluation methods, and optimizing the quality control (Zhang 2021 ).

A transformative study of online teaching and learning in higher education

With the outbreak of the COVID-19 pandemic in 2020, China’s Ministry of Education proposed “stopping classes without stopping teaching, stopping classes without stopping learning”. Thus, online teaching and learning have become the norm (Khong et al. 2023 ). Therefore, in recent years, the current situation, characteristics, and strategies of online teaching and learning have been a hot topic for researchers in China, especially in higher education.

Online teaching has led to the following four transformations in Chinese higher education: educational resources have shifted from segmentation to sharing; student learning has shifted from linear to nonlinear; curricular reform has shifted from structured to unstructured; and educational technology has shifted from an auxiliary means to a deep integration with teaching (Xue and Guo 2020 ). As a result, the digitization of Chinese higher education has begun to show the following typical features: Chinese universities view digitization as characterized by instrumentality (e-campus construction and application) and modernization (continuity and efficiency of teaching innovation), which can also be seen in other countries. The unique goal of online higher education in China is to build a positive cyber-spirit and develop political and ideological education through digitalization (Xiao 2019 ).

Accordingly, strengthening online teaching in colleges and universities can be summarized as follows: (1) the need to strengthen the investment in infrastructure construction in central and western regions and to establish a mechanism for sharing higher education resources and a mechanism for recognizing course credits; (2) the need to establish the concept of lifelong learning, cultivate students’ lifelong learning ability, build a diversified talent cultivation system, and provide students with a more flexible learning system; (3) the need to change the narrow-minded thinking of specialized education, establish a more open sharing mechanism for course resources, and comprehensively update and improve the academic evaluation system; and (4) the need to strengthen the construction of resource platforms for online teaching in colleges and universities, reshape the university learning space, and comprehensively improve teachers’ informatization literacy and online teaching ability (Liu et al. 2015 ).

Research on the resource dimension of Chinese education digitalization

Digital educational resources are tools to provide of digital educational content and support personalized learning. This research mainly covers the study of digitalized teaching content, teaching materials, teaching courseware, etc. for various school levels. Research on digital educational resources can accelerate the popularization and sharing of educational resources, help provide students with convenient and diverse learning materials, and facilitate personalized learning.

Study on the digitization of educational publications

Since 2017, educational publishing, especially that of digital teaching materials, has been a research hotspot for scholars in the field of the digitization of education in China. Various scholars focus on the current situation of the digitization of educational publishing, the problems encountered and their reasons, coping strategies, and specific cases in which to develop their research on the digitization of educational publishing in China.

The COVID-19 pandemic of 2020 precipitated a boom in online education, and as a result, the field of educational publishing has correspondingly spawned a need for the digitalization of educational publishing. The most prominent development is that publishers have begun to provide many online education services, with content services as the main body. In addition, many publishers provide many recorded courses, audio-visual videos and other digital educational resources. Some publishers have even begun to try to launch live education (Chen and Fan 2015 ).

All these developments show that publishers have the ability to provide diversified educational services. Given the hotspot of online education during the pandemic, publishers should pay attention to several points when promoting the digitization of educational publishing: paying close attention to the construction of digital teaching materials and accurate content; expanding the construction of audio-visual content, focusing on creating systematic microvideo and online courses; and based on the content and the regional market, moderately expanding the online training industry, creating vertical platforms, and advancing the transformation from content resource providers to educational service providers (Luo 2018 ).

Other scholars have focused on the digitization of publishing in the field of higher education. The digital transformation of Chinese higher education publishing faces some development difficulties. For example, the homogenization of educational resources is serious, the dominant position of content resources has been impacted, the interactivity of products is weak, and the transformation of operation mode is difficult. In the face of this dilemma, Chinese higher education publishing units should actively explore the path of digital transformation, including developing service-oriented digital products, cultivating intelligent publishing service capabilities, exploring omni-channel marketing models, creating an all-media publishing format, and innovating the mechanism of convergent publishing (Zhou and Liu 2016 ).

Study of the development and application of digital learning resources in vocational institutions

In 2017, China’s Ministry of Education proposed the comprehensive implementation of reforms to vocational education. In 2018, the National Implementation Plan for Vocational Education Reform was officially promulgated. As seen from the above charts, from 2017 to the present, the study of the digital transformation of vocational education has been a research hotspot in the digitization of education in China.

Based on the integration and analysis of related literature, it was found that some scholars have sorted out the construction of digital teaching resource platforms for vocational education in China, classified the types of platforms, dissected the current problems in the use of digital teaching resources in China’s vocational education, and proposed corresponding solution strategies.

The scale of vocational education in China is enormous, with nearly 20,000 vocational and technical schools (colleges) at the middle and higher levels and approximately 640,000 adult education institutions. At present, China has established a considerable number of digital education and teaching resource platforms for vocational education at the national, provincial, local and school levels. The completed national-level websites specializing in vocational education include the China Vocational and Technical Education Network ( www.Chinazy.org ), the China Vocational and Adult Education Network ( www.cvae.com.cn ), and the National Digital Learning Resource Center. Provincial, municipal and regional vocational education networks have been set up and include Jiangsu Vocational Education Network ( www.jsve.edu.cn ), Nanjing Vocational and Social Education Network ( www.njzj.net ), and so on. Furthermore, many websites of secondary vocational schools and higher vocational schools have been established on a school basis. These websites provide all kinds of rich online education information and teaching resources at different levels for the audiences of vocational education in China (Wang, 2013 ).

The integration of the current literature on digital learning resources in Chinese vocational institutions reveals that the vast majority of online resources have the following problems. First, a unified national standard for the construction of resources on China’s vocational education digital platform is still lacking, which prevents better resource sharing on the platform. Second, most digital teaching resource platforms have a veneer webpage form with few intractable resources, which does not fully reflect the characteristics of digital learning and online learning. Third, the online resources of Chinese vocational colleges and universities do not highlight the advantages and resource characteristics of their vocational education (Jin and Rong 2022 ).

Some scholars have utilized questionnaire surveys to understand the actual digital teaching resource usage of teachers and students at vocational colleges and universities. Moreover, based on rationalized suggestions from frontline vocational education teachers and students, scholars have proposed strategies related to the development and application of digital learning resources in vocational colleges and universities: (1) strengthening the infrastructure and resource environment construction of digital teaching resources in vocational education; (2) integrating digital teaching resources into vocational education and sharing well; and (3) creating a network exchange platform for vocational education digital teaching resources. Nearly 10 years after the study of vocational education digitalization, in 2022, some scholars proposed a strategy that is more suitable for the development and application of digital learning resources in today’s vocational colleges and universities: (1) the construction of digital campuses should move from a model “with borders” to one “without borders”; (2) teaching reform should move from uniform standards to individualized teaching resources; (3) the construction and reform of digital teaching resources should move from uniform standards to individual customization; (4) professional construction should move from accompanying industries to leading industries; (5) resource development should move from innovation to sharing; (6) conceptual enhancement should move from the main body of teachers to participation of the whole staff; and (7) input guarantees should move from government-led to multi-participatory (Cheng 2016 ).

Study on the equity of elementary education resources in the context of digitalization

Digital educational resources are a powerful means of promoting the balanced development of education and provide an opportunity to narrow the regional, urban‒rural and interschool Digital Divide. The CiteSpace visual analysis shows that after the promulgation of the “double-decrease” policy in education, China has attempted to further expand the coverage area of basic education digitization (Wang and Li 2023 ). Therefore, the study of the digital transformation of basic education in promoting education equity and narrowing the gap in education quality has been one of the hot research topics among scholars in recent years.

In the relevant literature, a status quo survey on the equity of digital resources for basic education in urban and rural areas has been developed. For example, after the COVID-19 outbreak, the Chinese New Century Primary Mathematics Materials (NCPM) Committee developed a series of micro courses (referred to as NCPM micro courses), which were viewed by more than 25 million teachers and students across the country during the three-month social isolation period. A total of 132,740 data points on Chinese elementary school students’ viewing of the microlessons were collected, and the extent to which Chinese elementary school students recognized NCPM microlessons after social isolation was investigated. Based on the survey, it was concluded that primary school students in urban areas were generally more likely to recognize micro classes, while primary school students in rural areas were generally less likely to recognize micro classes. Moreover, other studies employed questionnaires and revealed that students from rural or migrant schools scored lower on all Internet inequality indicators (digital access, autonomy of use, social support, Internet use, and self-efficacy) and that they were at a disadvantage in terms of their Internet use status compared to their urban counterparts. In other words, rural and urban schools in China are currently at different stages of ICT development (Li and Ranieri 2013 ). Thus, rural schools should focus on improving ICT infrastructure and developing quality digital resources. The digital transformation of education will become an important tool for the high-quality development of rural education and even China’s basic education (Wu et al. 2019 ).

Accordingly, measures to improve the digital transformation of rural education and to reduce the inequality of basic education resources between urban and rural areas have been proposed in the literature. To realize the digital transformation of rural education, it is possible to upgrade and transform the infrastructure and environment of rural education, build a public service platform for “smart education” that facilitates interaction between teachers and students, and establish a database for rural education (Yang and Yu, 2014 ). To improve the efficiency of the digital transformation of rural education, the opportunity of the national education digitalization strategy needs to be seized to realize the parallel development of the three phases of computer-assisted teaching, “Internet plus Education” and “Artificial Intelligence plus Education” to improve the effectiveness of rural teachers’ use of digital teaching resources (Wang and Li 2023 ).

The relevant literature also focuses on the advantages and significance of education digitization for rural education and the realization of education equity, proposing that the digital transformation of rural education can gradually narrow the gap between urban and rural education and provide an effective way to promote their balanced development, thereby promoting the equity of compulsory education. In the process of education and teaching, the rational use of digital technology can promote equality in the starting point of education and provide a formal guarantee of equality in both the process and the outcome (Zhu and Hu 2022 ).

Research on the technological dimension of Chinese educational digitalization

Digital educational technology represents the driving force of digital education. Research on education empowered by advanced technologies such as big data analysis, cloud computing, and artificial intelligence can provide new possibilities and models for the future direction of education.

Study on artificial intelligence technology empowers digital transformation of education in China

Technology is the fundamental feature that distinguishes digital education from traditional education (Zhu and Hu 2022 ), and among the hot topics of “research on the digitalization of education in China”, many studies examine digital technology. As seen from the above chart, in earlier years, for example, in 2013, technological research on the digitization of education in China mainly focused on the impact of relatively elementary information technologies, such as the IoT and two-dimensional codes (QR codes), on education. Along with the further development of digitization technology, research on the integration of technology and education, represented by AI, has become a research hotspot from 2021 onward.

Some scholars have innovatively proposed the “empowerment theory” of the digital transformation of education, which means that the true meaning of the digital transformation of education is to give full play to the advantages of modern technology to promote the modernization of education, develop high-quality education, and empower educational innovation. Technology is used not only to support teaching but also to empower education, as digital technology is used to establish new educational services, implement new methods or form new models that transcend the boundaries and development patterns of traditional education (Zhu and Hu 2022 ).

Various scholars have proposed different views on how digital technologies represented by AI can “empower” the digital transformation of Chinese education. Among them, the viewpoint recognized by Chinese digital education researchers is that the development and application of new-generation digital technologies, such as AI, has had a disruptive impact on various industries. However, to realize the development of China’s education digitization, “AI plus Education” is not equal to the simple application of intelligent technology in education; instead, AI should be used as an endogenous variable for the overall change in education, supporting and leading its innovative development (Cao 2020 ).

Research on the management dimension of Chinese education digitalization

Digital educational management, which plays a planning and coordinating role in the development of digital education, serves as its guarantee. Research on the management of digital education in China includes studies on the formulation and implementation of relevant policies, as well as studies on evaluating and monitoring the effectiveness of the implementation of digital education and digital literacy. These studies can safeguard the environment of digital education in China and ensure the healthy and orderly development of digital education in China.

Study on the digitalized industry-education integration in the context of rural revitalization

The research on the digitization of Chinese education has typical Chinese characteristics.

In 2017, the Chinese government clearly noted in the work report that it would implement the strategy of “rural revitalization”. From the cluster analysis table, it can be seen that since 2017, “the deep integration of rural revitalization and vocational education in the context of digitization” has become a hot research topic connected to the digitization of education in China.

In vocational education in China, the integration of industry and education has always been emphasized. Therefore, in the study of the digital transformation of vocational education in China, some scholars have proposed and endorsed the transformation mode of “Internet + Vocational Education” and linked it to the “rural revitalization” proposed by the Chinese government in recent years. These scholars have suggested that in the context of rural revitalization, efforts should be made to build a digital education management mechanism for the in-depth integration and synergistic development of industry and education to empower and strengthen the high-quality development of rural industries and the realization of industrial prosperity and common prosperity for rural farmers (Jin and Rong 2022 ).

The digital integration of industry and education involves the use of digital technology to create the mechanism of “Internet + vocational education”, giving full play to the dualistic nature of education and industry in vocational education. At the theoretical level, the government, vocational colleges and universities, agricultural research institutes and other subjects can increase their research efforts to change the rural industry talent training mode through the form of scientific research projects, focusing on the construction of network learning platforms, the development of digital curriculum resources and other fields. Moreover, the logic and path of the practice of the digital transformation of vocational education can be systematically explored to help revitalize rural industry talent, and theoretical guidance for the improvement of the quality of the training of rural industry talent can be provided. At the practical level, vocational colleges and universities should strengthen cooperation with digital enterprises and scientific research institutions, implement digital cooperation projects in upgrading digital teaching resources and building “cloud classrooms”, and establish an “Internet + “ skills training system to support the development and prosperity of rural industries (Jin and Rong 2022 ).

Study on the digital literacy of teachers and students

In China’s education digitization research, many studies have examined improving teachers’ and students’ digital literacy. In addition to “digital literacy”, the terminology also includes “digital competence” and “digital capability”. Digital competence is becoming more important for our current information society and plays an essential role in the process of digital informal learning (He and Zhu 2017 ).

ICT and media have challenged traditional education and changed the way teachers think. Teachers’ digital literacy directly affects their teaching and plays an increasingly important role in education (Zhao et al. 2016 ). Some Chinese scholars, based on the current situation of Chinese higher education, have utilized a questionnaire survey and built upon the relevant studies of previous researchers to propose the novel idea that we should construct an index system for evaluating the digital literacy of teachers in Chinese universities and colleges. We can evaluate the digital literacy of college teachers on five dimensions and 18 specific indicators: digital technology use, digital information management, digital content creation, digital community building and digital security ability (Yang and Zhou 2019 ).

In addition to studies on teachers’ digital literacy, there are also many studies on students’ digital literacy. Relevant studies have mainly focused on the current situation, problems and strategies for enhancing the digital literacy of contemporary Chinese undergraduates. Some scholars have developed and validated a questionnaire to assess the digital skills of Chinese undergraduate students based on previous literature and the Chinese educational environment. The survey shows that Chinese undergraduates are still deficient in these digital literacy skills, such as the ability to access digital information, the ability to create digital content, the awareness of digital security, and the comprehensive ability to solve digital problems. Scholars in this field believe that digital literacy that includes critical thinking is an important source of creativity for current college students and should become an important goal of higher education and one of the most important standards for measuring the quality of college students. Therefore, the strategy to enhance the digital literacy of Chinese college students should be emphasized from the top of the national top-level design to the cooperation of specific colleges and universities so that multiple parties can work together to formulate a digital literacy framework for college students that meets the actual situation in China and to determine the specific educational standards of digital literacy (Ling 2020 ). Furthermore, a few studies have explored the development of digital literacy for younger students in China. Due to the intense debate about the benefits and risks of the early use of digital devices, 1953 parents of younger children from a province in central China were recruited for a dissertation experiment to examine their children’s digital literacy. The findings suggest significant within-group differences in early digital literacy and multimodal practices, which are mainly due to family SES and parental factors. Therefore, we should pay more attention to low SES families and parental education to narrow the digital divide among Chinese children (Dong et al. 2022 ).

The interrelationships and interactions of the various dimensions of Chinese educational digital transformation

Digital education resources, digital education platforms, digital education technologies, and digital education management are interconnected and interdependent; together, they act on digital education transformation, forming a dynamically developing ecosystem that promotes the digitalization of education in China.

Their comprehensive effect on the digitization of Chinese education is mainly reflected in the following three points.

This comprehensive effect is first reflected in synergy. The richness and quality of digital resources enhance the value and attractiveness of digital platforms; the innovation and application of digital technologies promote the functional upgrading and service optimization of digital platforms; and the effectiveness and foresight of digital management guarantee the healthy and sustainable development of the entire ecosystem of China’s digital education. Their synergistic effect enables the digitization of education in China to continuously adapt to the needs of development, meet the diversified needs of different user groups, and promote the fair, high-quality and innovative development of education in China.

Second, this combined effect significantly improves the user experience. The synergy of the four dimensions—integrating high-quality resources, building a powerful platform, applying advanced technology, and implementing effective management—has allowed the development of an efficient, convenient, and highly personalized learning environment for digital education in China. In such an environment, teachers and students can experience richer and more diverse content and enjoy more flexible and customized learning methods, which not only improves the effectiveness of education and teaching but also enhances the digital literacy of teachers and students.

Finally, this combined effect has become a powerful driving force for the continued innovation and development of digital education in China. The combined effect of these four dimensions can bring positive impacts to resource optimization, platform enhancement, technological innovation and management improvement for the digital development of education in China and promote China’s education to create a more digital, intelligent and personalized future.

Results and discussions for future evolutionary trends

The analysis of the future evolutionary trends of China’s educational digitization research in this study is based on three aspects: a burst term analysis using CiteSpace, the disparities between Chinese and international educational digitization research, and the insights gained from digital education industry reports. This discussion aims to provide a comprehensive understanding of the future directions of educational digitization in China, taking into account both domestic and international perspectives.

CiteSpace-based analysis of burst terms

Chaomei Chen defines research frontiers as a set of emergent dynamic concepts and potential research questions in CiteSpace, and these are represented by burst terms that were extracted based on literature titles (Title), abstracts (Abstract), keywords (Descriptor) and identifiers (Identifiers). To identify the frontiers of research on education digitization in China, this study utilized citation bursts to highlight the frontier terms and organize the burst terms by burst value, resulting in the top 20 burst terms with the highest burst values.

From the Fig. 8 , Online Education, Educational Publishing, Big Data, Smart Education, and Smart Campus are not relatively high in strength but also that they end in 2022. Since the selected time span of our literature is only up to 2022, research related to these burst words will likely continue to be a hot topic for future studies on the digitization of education in China.

Top 20 keywords with the strongest citation bursts.

First, research on the normalization of online education in the post-pandemic era is a future evolutionary trend based on the burst terms of CiteSpace. At the start of 2020, the sudden outbreak of the COVID-19 pandemic forced China to shut down schools, from elementary schools to colleges and universities. The pandemic caused both schools that were already using a blended model and those that were not fully prepared for online teaching to switch to online teaching at the same time. As a result, the amount of literature on the study of digitization of education in China has increased dramatically since 2020. However, the results of a nationwide questionnaire survey of frontline teachers and students on the implementation of online education indicated that most teachers and students were not ready to implement online education in the long term, as they believed that it was only a temporary coping strategy in the face of the COVID-19 pandemic and that online education failed to achieve the effect of teachers’ face-to-face lectures (Xie and Yang 2020 ). In December 2022, when the Chinese government reclassified COVID-19 as “Category B”, the pandemic prevention policy was discontinued. It is unclear whether teachers will be interested in continuing the online mode of education after the pandemic subsides. Although classes may revert to a face-to-face model when the COVID-19 situation improves, online teaching will play a critical role in the future of education. In addition, under China’s “Class B, Category B” regulatory requirements, Chinese schools in the post-pandemic era may occasionally shift to online teaching (Khong et al. 2023 ). Since the beginning of the COVID-19 pandemic, online education has become an increasingly important and indispensable form of teaching in Chinese education. Therefore, in the post-pandemic era, the normalization of online education will continue to be a major topic in future research on the digitization of Chinese education.

Second, research on the development of online educational resources in the context of the “new infrastructure” is another future evolutionary trend based on the CiteSpace burst terms. The concept of “new infrastructure” was first proposed in China in December 2018, and the construction of new infrastructure centered around new-generation information technology such as AI, big data, IoT, 5 G and other new-generation information technology has begun in various industries. With the continuous promotion of China’s new infrastructure process, the connotation of the new infrastructure is constantly being enriched. In April 2020, China’s Development and Reform Commission clarified the specific scope of the “new infrastructure” for the first time. That is, with the new development concept as the lead concept, driven by technological innovation, based on the information network, and oriented to the needs of high-quality development, this infrastructure system provides digital transformation, intelligent upgrading, and convergence and innovation services and includes information, convergence and innovation infrastructures (Zhu and Hu 2022 ). Against the backdrop of China’s new infrastructure, the new educational infrastructure has been developed at the right time. In the post-pandemic era, the change in the teaching paradigm of online-offline integration in the education industry has become the driving force of China’s new infrastructure for education, and the technological empowerment of 5 G + VR, education big data, AI and the booming development of online education have brought important opportunities for the development of a new technology-driven new infrastructure for education (Zhu et al. 2021 ). In 2021, the Ministry of Education and five other departments issued the Opinions on Vigorously Strengthening the Construction and Application of Online Educational and Teaching Resources for Primary and Secondary Schools, proposing that by 2025, a clearly positioned, interconnected, and shared online education platform system should essentially be formed, and that a disciplinary curriculum resource system covering various types of thematic education and versions of textbooks should be established for the use of online educational resources for teaching and learning to become the new normal.

Third, research on “new technology plus education” is also a future evolutionary trend based on the CiteSpace burst terms. In 2021, the “Metaverse” concept was widely used, and one by one, scholars in China’s education sector began to study the possibility of developing “Metaverse + Education”. Some scholars started by deconstructing the concept of “Metaverse”, clarifying its evolution and basic structure, and analyzing the opportunities and challenges it brings to education. The meta-universe is a higher-dimensional virtual space that transcends the real world and is characterized by an in-person immersive experience, cocreation, sharing and co-governance, the symbiosis of reality and reality, and real-time online interaction, which coincides with the development concepts and directions of future education (Li and Wang 2022 ). Metaverse-related technology has great potential for application in teaching scenarios such as contextualized teaching, personalized learning, game-based learning and teacher training. In the future, exploring the deep integration of metaverse and education will be very valuable (Wang and Zhang 2022 ). In addition to the concept of “Metaverse + Education” proposed in 2022, ChatGPT, an AI technology, has gained much attention since its launch in 2023. Ways in which to apply ChatGPT to digital education and better integrate AI technology with education have become a research trend for related researchers. The concept of “new technology plus education” will continue to be studied by more scholars in the future digitalization of education. Future research may focus on the following topics: (1) exploring teaching models and strategies based on new technologies in education; (2) determining the educational forms and main application scenarios of the “new technologies plus education” model; (3) defining the future challenges; (4) researching the construction of new educational infrastructure under the “new technology plus education” model; and (5) conducting ethical research on digital education under the new technology model.

Gap analysis of digital education based on Chinese and International educational systems

To better analyze the future research direction of digitization of education in China, this study not only examines the CiteSpace analysis of the burst terms of the literature on digitization of education in China but also makes comparative studies between Chinese digital education and international digital education. The comparative study mainly starts from two major aspects: the review literature, and the strategies and outcomes of the digitization of education.

To begin with, the study considers the review literature on the research of “international digitization of education” mentioned in the literature review section of the previous article. Thus, it is found that some hot topics of international research on digitization of education are not covered in the current research on digitization of education in China or are examined in less detail; however, these are the topics to which scholars of China’s education digitization research should pay attention in the future.

The first topic is research on the positive impact of digital games on education. In international settings, especially in Europe and the United States, where education is more advanced, more attention has been given to exploring the positive impact of digital games on education in recent years. For example, in a review of relevant literature published in Computers & Education, a leading international journal on education digitization, it was found that the international education community is increasingly interested in research on the positive impacts and outcomes of video games on education (Boyle et al. 2016 ). In addition, Humanities & Social Sciences Communications, a leading comprehensive journal in the field of humanities and social sciences, has included a survey on the impact of video games on teaching performance. The present study indicates that contrary to the stances of certain governments (e.g., those of China, Korea, or Vietnam) and media scholars who consider games to be violent or addictive, several critical aspects of games facilitate learning among gamers (Jung 2020 ). However, this topic has rarely been addressed in research on the digitization of education in China. Therefore, scholars in China can pay more attention to the research on the positive effect of digital games on education in the future.

The second topic is the research on diversification of the Digital Divide. Promoting the balanced development of education and realizing equity in education have always been the focal points of common concern worldwide. A comparison of international and Chinese papers published in recent years shows that although both are concerned with educational equity and the digital divide, Chinese research on this topic has mainly focused on the digital divide in education between urban and rural areas in China. However, international researchers, especially those from developed countries such as European nations and the United States, have a broader and more diversified perspective on the digital divide in education. They are concerned not only with the digital divide in education between countries and regions but also of the gaps between men and women and the differences in living conditions and in educational attainment (Kerras et al. 2020 ). Thus, In the future, in addition to researching the topic of equity in the importation of digital resources for urban and rural education, scholars in China can incorporate a more diverse study of the digital divide into the study of equity in the digitization of education in China.

In addition to comparing China and foreign countries from the perspective of disparities in hot spots of literature, this study also compares China’s education digitization strategies and outcomes with international education digitization strategies and outcomes (Wang and Yuan 2022 ), and finds that there are a number of differences between the two in terms of digital education transformation. In China, due to the vastness of the country and its uneven economic development, there are obvious differences in the development of infrastructure for the digital education transformation between urban and rural areas, as well as between regions. To address this challenge, the Chinese government has implemented a series of strategic initiatives, such as the Modern Distance Education Project for Rural Primary and Secondary Schools, which aims to improve the level of informatization of education in rural areas and narrow the digital divide. These efforts reflect the emphasis on infrastructure development in China’s education digitization strategy, especially in rural areas. Such strategies help ensure that students in all regions can enjoy the convenience and resources brought by digital education. In contrast, developed countries in Europe and the United States had an earlier start in digital education transformation and their digital education development has become relatively mature. These countries have a more balanced infrastructural development, with fewer differences between urban and rural areas and in development among regions. Therefore, their strategies focus more on improving the quality and depth of education digitization, such as focusing on citizens’ digital rights and interests, digital ethics, and the maturity of education digitization (Department for Education 2022 ). Their attention to these topics as hot spots reflects the pursuit of comprehensive and high-quality development of education in developed countries. For China, with the continuous transformation and upgrading of education digitization, the focus can also be gradually expanded to these aspects in the future. The exploration of digital rights and digital ethics involves an exploration of how to protect students’ privacy and data security while using digital technology to improve the quality of education; enhancing the maturity of education digitization implies the need to continuously improve the ecological environment of digital education, including optimizing teaching resources, improving teachers’ digital teaching ability, innovating teaching modes, etc., which will help to realize the sustainable development of education digitization in China.

Analysis based on industry reports

At the World Conference on Digital Education in 2023, China proposed building a higher education digital development index for the first time.

As an important measure of the digital development of higher education in the world, the Higher Education Digital Development Index (HEDI) is an important tool for monitoring and assessing the level of digital development of higher education in foreign countries. This index is helpful for mapping out the current situation of the digital construction and application of higher education in all countries and regions of the world, comparing the development gaps of countries in all regions, and studying and judging the future development of higher education. Many countries and international organizations have explored and implemented the construction of an education digital development index. These include the “Framework for Digital Transformation in Higher Education” carried out by the OECD, the “School Digital Maturity” survey implemented by the UK, the “National Digital Experience Survey” carried out by Ireland, the International Association of Universities (IAU), the “Digital Development Index for Higher Education” carried out by the United Kingdom and the “Global Survey on Digital Transformation of Higher Education Institutions” organized by the International Association of Universities (IAU) of the Research Group on the Report on the Digital Development of World Higher Education in 2023.

Currently, existing international explorations and practices focus on certain meso-level elements (e.g., digital maturity of higher education institutions) and microlevel elements (e.g., digital competence of faculty and students, as well as the state of digital infrastructure development in schools) (Xu and Zhang 2022 ).

China can approach this topic from a more macro level. At the latest World Digital Education Conference 2024, for the first time, China released the Global Digital Education Development Index (GDEI) and the Chinese Smart Education Development Report 2023 (Yangguang Net 2024 ). The report proposes to evaluate the development of digital education in each country from six evaluative dimensions: digital literacy enhancement, digital education system construction, digital education institutional innovation, digital education content reconstruction, digital teaching reform, and digital education governance upgrade, in order to identify the most suitable path for the development of digital education in each country.

In the future, China can continue to delve deeply into the Global Education Development Index, and through the evaluation of GDEI, provide Chinese wisdom, experience, and solutions for the world to monitor and assess the level of digital education development, and jointly build a new international education ecology that may be mutually learned, applied universally, and mutually promoted.

Research conclusions

In conclusion, based on a bibliometric analysis of knowledge mapping in Chinese education digitalization research from 2012 to 2022 and the study of specific related literature, it can be concluded that, at present, Chinese scholars, led by Zhu Zhiting of East China Normal University, have focused on the digitization of education in China.

In the period 2012–2022, the hot spots in the research on the digitization of education in China can be analyzed in the framework of digital elements of education in China, which can be divided into four dimensions: 1. Platform Dimension: Studies of the lifelong education research in digital open universities and the online teaching transformation in higher education; 2. Resource Dimension: Studies of digital educational publications, the development and application of digital learning resources in vocational colleges and universities, and the equity of basic education resources in the digital context; 3. Technological Dimension: Studies on AI technology empower digital transformation of education in China; and 4. Management Dimension: Studies of digital integration of production and teaching in rural revitalization and improvement of digital literacy of university students and faculty.

Future digital education research in China is likely to focus on the normalization of online education; the development of online education resources in the context of new infrastructure; “new technology plus education”; the impact of digital games on education; a more diversified digital divide in education; digital rights, digital ethics, digital maturity and the Global Digital Education Development Index.

Limitations and future prospects

In this bibliometric analysis, CiteSpace was used as the main tool combined with a manual reading method to evaluate papers on “Digitization of Education in China” based on the CNKI and WOS databases for the period from 2012 to 2022. However, despite the excellent accuracy of CiteSpace, this method has some limitations. First, due to the limitation of databases that can be included in CiteSpace, only WOS and the CNKI were selected as the databases for this study. Although the databases are well recognized authoritative databases both internationally and within China, studies in other databases may have been left out. Second, other publications, such as dissertations and policy documents on the digitization of education in China, were not included in CiteSpace for visualization and analysis due to the type of literature that could be included in CiteSpace analysis. This could also have an impact on a more comprehensive and precise understanding of research on the digitization of education in China. Finally, although keywords and topics related to education digitization were selected for the search in this study, some keywords were inevitably missed. In the future, the researcher can further expand the search paths and search strategies to collect more comprehensive and accurate literature to obtain a more comprehensive and systematic picture of the research on the digitization of education in China.

Data availability

The datasets analyzed during the current study are available in the supplementary file.

Audrin C, Audrin B (2022) Key factors in digital literacy in learning and education: a systematic literature review using text mining. Educ Inf Technol. 27(6):7395–7419. https://doi.org/10.1007/s10639-021-10832-5

Article   Google Scholar  

Boyle EA, Hainey T, Connolly TM, Gray G, Earp J, Ott M, Lim T, Ninaus M, Ribeiro C, Pereira J (2016) An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Comput Educ. 94:178–192. https://doi.org/10.1016/j.compedu.2015.11.003

Cao P (2020) Three realms of the reform of artificial intelligence in education. Educ Res 41(2):143–150

Google Scholar  

Chen L, Fan J (2015) Research on predicament and countermeasures of education publishing digital transformation. Editor Friend 6:10–13. https://doi.org/10.13786/j.cnki.cn14-1066/g2.2015.06.003

Cheng X (2016) Research on digital teaching resources for vocational education——based on 2005-2015 papers analysis. China Educ Technol 8:120–124. https://doi.org/10.3969/j.issn.1006-9860.2016.08.020

Department for Education (2022) Education technology: Exploring digital maturity in schools. CooperGibson Research. https://assets.publishing.service.gov.uk/media/623481bce90e0779a18d3f39/Exploring_digital_maturity_in_schools.pdf . Accessed 19 March 2024

Díaz-García V, Montero-Navarro A, Rodríguez-Sánchez JL, Gallego-Losada R (2022) Digitalization and digital transformation in higher education: A bibliometric analysis. Front Psychol 13:1081595. https://doi.org/10.3389/fpsyg.2022.1081595

Article   PubMed   PubMed Central   Google Scholar  

Dong C, Cao S, Li H (2022) Profiles and Predictors of Young Children’s Digital Literacy and Multimodal Practices in Central China. Early Educ Dev 33(6):1094–1115. https://doi.org/10.1080/10409289.2021.1930937

He T, Zhu C (2017) Digital informal learning among Chinese university students: the effects of digital competence and personal factors. Int J Educ Technol High Educ 14(1):44. https://doi.org/10.1186/s41239-017-0082-x

Huang Y (2015) Current situation analysis and training path of college students’ digital literacy. Ideol Theor Educ 3:82–85. https://doi.org/10.16075/j.cnki.cn31-1220/g4.2015.03.017

Jin P, Rong C (2022) Vocational education drives the development of rural industries under the background of rural revitalization: Mechanism construction and promotion paths. Educ Voc 22:18–25. https://doi.org/10.13615/j.cnki.1004-3985.2022.22.018

Jung CW (2020) The role of game genres and gamers’ communication networks in perceived learning. Palgrave Commun. 6(1):69. https://doi.org/10.1057/s41599-020-0439-y

Article   MathSciNet   Google Scholar  

Kerras H, Sánchez-Navarro JL, López-Becerra EI, de-Miguel Gomez MD (2020) The impact of the gender digital divide on sustainable development: comparative analysis between the European Union and the Maghreb. Sustainability 12(8):3347. https://doi.org/10.3390/su12083347

Khong H, Celik I, Le TTT, Lai VTT, Nguyen A, Bui H (2023) Examining teachers’ behavioural intention for online teaching after COVID-19 pandemic: A large-scale survey. Educ Inf Technol 28(5):5999–6026. https://doi.org/10.1007/s10639-022-11417-6

Li H, Wang W (2022) Metaverse+Education: A new state of educational development in the future. Mod Distance Educ Res 1:47–56. https://doi.org/10.13927/j.cnki.yuan.20220110.002

Li Y, Ranieri M (2013) Educational and social correlates of the digital divide for rural and urban children: A study on primary school students in a provincial city of China. Comput Educ 60(1):197–209. https://doi.org/10.1016/j.compedu.2012.08.001

Ling Z (2020) Current situation, problems and educational path of college student’s digital literacy in China. Inf Stud-Theory Appl 43(7):43–47+53. https://doi.org/10.16353/j.cnki.1000-7490.2020.07.008

Liu H, Nie J, Wu Y (2015) The construction of university digital teaching resources based on education informationization: Ideas, strategy and routek. Mod Educ Manag 9:89–94. https://doi.org/10.16697/j.cnki.xdjygl.2015.09.017

Luo W (2018) Exploration of digital transformation path of educational publishing. View Publ 16:47–49. https://doi.org/10.16491/j.cnki.cn45-1216/g2.2018.16.014

Ministry of Education of the People’s Republic of China (2018) Education Informationization 2.0 Action Plan. https://www.moe.gov.cn/srcsite/A16/s3342/201804/t20180425_334188.html . Accessed 30 November 2023

Ministry of Education of the People’s Republic of China (2021) The Outline of the Fourteenth Five-Year Plan and Vision 2035 for National Economic and Social Development of the People’s Republic of China. http://www.moe.gov.cn/jyb_xwfb/s6052/moe_838/202103/t20210315_519738.html . Accessed 30 November 2023

OECD (2019) OECD future of education and skills 2030 concept note. OECD Publishing. https://www.oecd.org/education/2030-project/ . Accessed 30 November 2023

Shen Y, Huang L, Wu X (2022) Visualization analysis on the research topic and hotspot of online learning by using CiteSpace-Based on the Web of Science core collection (2004-2022). Front Psychol 13:1059858. https://doi.org/10.3389/fpsyg.2022.1059858

Timotheou S, Miliou O, Dimitriadis Y, Sobrino SV, Giannoutsou N, Cachia R, Monés AM, Ioannou A (2023) Impacts of digital technologies on education and factors influencing schools’ digital capacity and transformation: A literature review. Educ Inf Technol (Dordr.) 28(6):6695–6726. https://doi.org/10.1007/s10639-022-11431-8

Article   PubMed   Google Scholar  

UNESCO (2015) Education 2030 Framework for Action to be formally adopted and launched UNESCO. http://www.unesco.org/new/en/education/themes/leading-the-future-of-education/2030-education-framework-for-action/ . Accessed 30 November 2023

Wang J, Zhang H (2022) Meta Universe: The singularity and realistic challenge of future education. Inn Mong Soc Sci. 43(4):193–200. https://doi.org/10.14137/j.cnki.issn1003-5281.2022.04.025

Wang S, Yuan Y (2022) Experience and strategy analysis of digital transformation of international education. People Educ. Z3:50–53

Wang T, Li Z (2023) Value orientation and practical direction of digital transformation of rural education. Chongqing High Educ Res 11(4):14–22. https://doi.org/10.15998/j.cnki.issn1673-8012.2023.04.002

Wang X (2013) Research on the construction of vocational education digital resource platform. Voc Techn Educ Forum 22:10–13. https://doi.org/10.3969/j.issn.1001-7518.2013.22.003

Wu D, Li C-C, Zhou W-T, Tsai C-C, Lu C (2019) Relationship between ICT supporting conditions and ICT application in Chinese urban and rural basic education. Asia Pac Educ Rev 20(1):147–157. https://doi.org/10.1007/s12564-018-9568-z

Wu J, Li H, Wei X (2022) Digital transformation of education: International perspective,development needs and implementation strategy. Distance Educ China 7:21–27+58+79. https://doi.org/10.13541/j.cnki.chinade.2022.07.009

Xiao J (2019) Digital transformation in higher education: critiquing the five-year development plans (2016-2020) of 75 Chinese universities. Distance Educ 40(4):515–533. https://doi.org/10.1080/01587919.2019.1680272

Xie Y, Yang X (2020) The digital transformation of higher education personnel training model. Guangxi Soc Sci 2:185–188. https://doi.org/10.3969/j.issn.1004-6917.2020.02.030

Xu X, Zhang C (2022) Factors and ways of digitized higher education reform in our country. High Educ Res China 7:31–35. https://doi.org/10.16298/j.cnki.1004-3667.2022.07.06

Xue C, Guo Y (2020) Reflections and Recommendations on the Reform of Online Teaching Reform in Universities. J East China Norm Univ (Educ. Sci. Ed.) 38(7):65–74. https://doi.org/10.16382/j.cnki.1000-5560.2020.07.005

Yang S, Zhou Z (2019) Research on the evaluation index construction of college teachers’ digital literacy. J Mod Inf 39(3):59–68+100. https://doi.org/10.3969/j.issn.1008-0821.2019.03.007

Article   CAS   Google Scholar  

Yang X, Yu S (2014) On the transformation and upgrading of digital education in China. Educ Res 35(5):113–120

CAS   Google Scholar  

Yangguang Net (2024) The global digital education development index and the China smart education development report (2023) were released in Shanghai. https://baijiahao.baidu.com/s?id=1789600115176031611&wfr=baike . Accessed 6 February 2024

Yu P, Qian D, Zhu Z (2014) Research on the classification,structure and standards of digital lifelong education resources. Mod Distance Educ Res 4:47–55. https://doi.org/10.3969/j.issn.1009-5195.2014.04.006

Zhang J (2021) Comprehensive reform of open universities and innovative development of lifelong education under the background of digitization. J Distance Educ 39(6):3–8. https://doi.org/10.15881/j.cnki.cn33-1304/g4.2021.06.001

Zhao J (2021) Teacher burden in the Age of Technology: A new perspective for understanding the digital transformation of education. Educ Res 42(11):151–159

ADS   CAS   Google Scholar  

Zhao P, Kynäshlahti H, Sintonen S (2016) A qualitative analysis of the digital literacy of arts education teachers in Chinese junior high and high schools. J Librariansh Inf Sci. 50(1):77–87. https://doi.org/10.1177/0961000616658341

Zhou J, Liu X (2016) Exploration and thinking of digital transformation of higher education press. Publ Res 9:44–46. https://doi.org/10.19393/j.cnki.cn11-1537/g2.2016.09.015

Zhu Z, Xu Q, Wu Y (2021) Building new infrastructure for educational informatization: standards and implementation. Distance Educ China 10:1–11+76. https://doi.org/10.13541/j.cnki.chinade.2021.10.001

Zhu ZT, Hu J (2022) The logic of practice and opportunities for digital transformation in education. E-Educ Res 43(1):5–15. https://doi.org/10.13811/j.cnki.eer.2022.01.001

Download references

Author information

Authors and affiliations.

International and Comparative Educational Institute, School of Education, Zhejiang Normal University, Jinhua, 321004, China

Rui Shi & XiuLan Wan

College of International Cultural Communication and International Education, Communication University of Zhejiang, Hangzhou, 310015, China

You can also search for this author in PubMed   Google Scholar

Contributions

RS had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Concept design and drafting of the manuscript: RS Data search: RS Statistical analysis: RS, XW. Acquisition, analysis, or interpretation of data: RS Supervision: XW. All authors reviewed the manuscript. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Rui Shi .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

This article does not contain any studies with human participants performed by the author.

Informed consent

Additional information.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Institutions table, rui shi certificate 2, authors mapping, cluster mapping, keywords mapping, institutions mapping, authors table, burst terms derive from citespace, cluster table, keywords for clustering algorithms, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Shi, R., Wan, X. A bibliometric analysis of knowledge mapping in Chinese education digitalization research from 2012 to 2022. Humanit Soc Sci Commun 11 , 505 (2024). https://doi.org/10.1057/s41599-024-03010-8

Download citation

Received : 30 May 2023

Accepted : 02 April 2024

Published : 11 April 2024

DOI : https://doi.org/10.1057/s41599-024-03010-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Frameworks for Determining Research Gaps During Systematic Reviews

Methods Future Research Needs Reports, No. 2

Investigators: Karen A Robinson , PhD, Ian J Saldanha , MBBS, MPH, and Naomi A Mckoy , MS.

Affiliations

• Copyright and Permissions

Structured Abstract

Research objective:.

Systematic reviews, in addition to summarizing the evidence, generally also discuss needs for future research. However, in contrast to the methods of the systematic review, future needs are not identified systematically. There is limited literature describing organizing principles or frameworks for determining research gaps. We developed and pilot-tested a framework for the identification of research gaps from systematic reviews.

Study Design:

We reviewed the research gaps identification practices of organizations involved with evidence synthesis. We contacted: (i) evidence-based practice centers (EPCs) (n=12) associated with the Agency for Healthcare Research and Quality ( AHRQ ) in the US and Canada, and (ii) other organizations around the world (n=64) that conduct systematic reviews, cost-effectiveness analyses, or technology assessments. Based on the responses, we developed a framework for identifying research gaps. We obtained feedback from two technical experts at our institution and pilot-tested this framework on two randomly selected EPC evidence reports. We also developed a simple, user-friendly worksheet with instructions to facilitate the use of the framework by investigators during or after a systematic review.

Population Studied:

Not Applicable.

Principal Findings:

Four (33.3%) EPCs and 3 (8.1%) of the other organizations reported currently using an explicit framework to determine research gaps. We did not identify one framework that captured all elements needed to determine and characterize research gaps. Variations of the PICO (population, intervention, comparison, outcomes) framework were most common. It is also important to classify the reason(s) for the gap to help determine how to address the gap. Therefore, we propose a framework that includes both the characterization of the gap using PICOS elements (also including setting) and the identification of the reason(s) why the gap exists. The framework allows investigators to classify reasons for the existence of a research gap as: (a) insufficient or imprecise information, (b) biased information; (c) inconsistency or unknown consistency, and (d) not the right information. We mapped each of these reasons to concepts from three commonly used evidence grading systems: the Grading of Recommendations Assessment, Development and Evaluation ( GRADE ); the United States Preventive Services Task Force ( USPSTF ); and the Strength of Evidence (SOE) used by EPCs. This allows leveraging of work already being completed during evidence grading. During pilot-testing, we identified challenges including difficulty in applying the framework for completed systematic reviews and differences in the specificity of research gaps abstracted by different users. These could be tackled with a priori discussions amongst investigators. Further testing should determine if these challenges are ameliorated if the framework is used during a systematic review.

Conclusions:

We developed a framework to identify and characterize research gaps from systematic reviews. The framework provides for the classification of where and why the current evidence falls short.

Implications for Policy, Delivery, or Practice:

In synthesizing evidence, systematic reviews inform health-care decisions for patients, policymakers, and clinicians. Systematic reviews can also be invaluable for identifying research gaps, thus helping develop research agendas. This potential impact of systematic reviews has not been realized. Our framework provides for systematically identifying and characterizing research gaps from systematic reviews. This explicit identification of research gaps will help determine the type of research needed to address the goals of comparative effectiveness research.

• Collapse All
• Acknowledgments
• Technical Expert Panel
• Peer Reviewers
• Executive Summary
• Step 1. Focused Literature Review
• Step 2. Review of Current Practices of Evidence-based Practice Centers (EPCs)
• Step 3. Review of Current Practices of Organizations Involved With Evidence Synthesis
• Step 4. Development of Framework
• Step 5. Pilot Test of Framework
• Step 6. Refinement and Finalization of Framework
• Limitations and Future Research
• Conclusions
• Abbreviations
• Appendix A. AHRQ’s Seven Questions About the Development, Prioritization, and Presentation of Research Needs
• Appendix B. Step 2: Review of Current Practices of Evidence-based Practice Centers (EPCs)—Data Abstraction Form for Audit of Evidence Reports
• Appendix C. Step 2: Review of Current Practices of Evidence-based Practice Centers (EPCs)—Responses Obtained From EPCs
• Appendix D. Step 3: Review of Current Practices of Organizations Involved With Evidence Synthesis—List of Contacted Organizations
• Appendix E. Step 3: Review of Current Practices of Organizations Involved With Evidence Synthesis—Responses Obtained and Final Determinations of Formal Processes
• Appendix F. Step 4: Development of Framework—Instructions for Research Gaps Abstraction Worksheet

Suggested citation:

Robinson KA, Saldanha IJ, Mckoy NA. Frameworks for determining research gaps during systematic reviews. Methods Future Research Needs Report No. 2. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. HHSA 290-2007-10061-I.) AHRQ Publication No. 11-EHC043-EF. Rockville, MD: Agency for Healthcare Research and Quality. June 2011. Available at: www.effectivehealthcare.ahrq.gov/reports/final.cfm .

This report is based on research conducted by the Johns Hopkins University Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2007-10061-I). The findings and conclusions in this document are those of the author(s), who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

The information in this report is intended to help health care researchers and funders of research make well-informed decisions in designing and funding research and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of scientific judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical research and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances.

This report may be used, in whole or in part, as the basis for research design or funding opportunity announcements. AHRQ or the U.S. Department of Health and Human Services endorsement of such derivative products or actions may not be stated or implied.

None of the investigators has any affiliations or financial involvement that conflicts with the material presented in this report.

540 Gaither Road, Rockville, MD 20850; www ​.ahrq.gov

• Cite this Page Robinson KA, Saldanha IJ, Mckoy NA. Frameworks for Determining Research Gaps During Systematic Reviews [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2011 Jun. (Methods Future Research Needs Reports, No. 2.)
• PDF version of this title (561K)
• Disable Glossary Links

Other titles in this collection

• AHRQ Methods Future Research Needs Series

Related information

• NLM Catalog Related NLM Catalog Entries

Similar articles in PubMed

• Review Development of a framework to identify research gaps from systematic reviews. [J Clin Epidemiol. 2011] Review Development of a framework to identify research gaps from systematic reviews. Robinson KA, Saldanha IJ, McKoy NA. J Clin Epidemiol. 2011 Dec; 64(12):1325-30. Epub 2011 Sep 19.
• The future of Cochrane Neonatal. [Early Hum Dev. 2020] The future of Cochrane Neonatal. Soll RF, Ovelman C, McGuire W. Early Hum Dev. 2020 Nov; 150:105191. Epub 2020 Sep 12.
• Review Framework for Determining Research Gaps During Systematic Review: Evaluation [ 2013] Review Framework for Determining Research Gaps During Systematic Review: Evaluation Robinson KA, Akinyede O, Dutta T, Sawin VI, Li T, Spencer MR, Turkelson CM, Weston C. 2013 Feb
• Review Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update. [Methods Guide for Effectivenes...] Review Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update. Berkman ND, Lohr KN, Ansari M, McDonagh M, Balk E, Whitlock E, Reston J, Bass E, Butler M, Gartlehner G, et al. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. 2008
• Review Understanding Health-Systems' Use of and Need for Evidence To Inform Decisionmaking [ 2017] Review Understanding Health-Systems' Use of and Need for Evidence To Inform Decisionmaking White CM, Sanders Schmidler GD, Butler M, Wang Z, Robinson K, Mitchell MD, Berkman N, Henderson J, Fiordalisi C, Hartling L, et al. 2017 Oct

Recent Activity

• Frameworks for Determining Research Gaps During Systematic Reviews Frameworks for Determining Research Gaps During Systematic Reviews

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Main Navigation

• Accept offer and enrol
• Current Students

Personalise your experience

Did you mean..., diploma of arts and social sciences, art/science collaboration wins waterhouse natural science art prize, unit of study hlth4002 health and human sciences thesis project i (2025).

Future students: T: 1800 626 481 E: Email your enquiry here

Current students: Contact: Faculty of Health

Students studying at an education collaboration: Please contact your relevant institution

updated - DO NOT REMOVE THIS LINE 6:05 AM on Fri, 12 April

Show me unit information for year

Unit snapshot, credit points, faculty & college.

Faculty of Health

Pre-requisites

Must have completed or be currently enrolled in HLTH4001 - Literature Review

Unit description

Supports students during the project development phase of the research process. Students will comprehensively review the scientific literature to identify an evidence gap and design a suitable research project. 

Unit content

Students will be working under the supervision of expert academics to complete the initial stage of their research project.

Availabilities

Learning outcomes.

Unit Learning Outcomes express learning achievement in terms of what a student should know, understand and be able to do on completion of a unit. These outcomes are aligned with the graduate attributes . The unit learning outcomes and graduate attributes are also the basis of evaluating prior learning.

On completion of this unit, students should be able to:

design a project that addresses a research question using theory and methods

plan the timely progression of a research project

synthesise scientific evidence to identify a knowledge gap in health and human sciences

formulate a research question using a knowledge gap

Teaching and assessment

Online (dual term), prescribed learning resources, dual term 2.

• Prescribed text information is not currently available.
• Prescribed resources/equipment information is not currently available.

Prescribed Learning Resources may change in future Teaching Periods.

Fee information

Commonwealth Supported courses For information regarding Student Contribution Amounts please visit the Student Contribution Amounts .

Fee paying courses For postgraduate or undergraduate full-fee paying courses please check Domestic Postgraduate Fees OR Domestic Undergraduate Fees .

International

Please check the international course and fee list to determine the relevant fees.

Courses that offer this unit

Bachelor of health and human sciences (honours) (2025), bachelor of health and human sciences (honours) (2024), any questions we'd love to help.