in research frontier

Share this page

Check against delivery

Dear Ladies and Gentlemen,

I have been asked to speak to you today about the importance of frontier research.

In some ways it is strange to be asked to talk to you about this subject at this event.

I expect that most of you here have dedicated your lives and careers to pursuing frontier research. Certainly that is the case for many of you that I know personally.

The aim of the European Academy of Sciences is to promote science and technology and their essential roles in fostering social and economic development.

So I do not think that I need to convince you about this.

The issue is that we rely on the support of others in order to be able to carry out our work.

It is vital that politicians and policy makers and citizens and society as a whole understand the importance of frontier research.

And there are powerful arguments for supporting frontier research, which I will get to. But, unfortunately there is no magic formula which is guaranteed to persuade politicians and policy makers to support frontier research. So we must be patient and make our case over and over again.

Indeed, it is important not only to make the right arguments. But also to engage with the right decision makers at the right time.

There will always be those involved in the policy process that are more interested in making a career than improving the world. For them, it’s not of interest to look beyond the short term to see the benefits of investing in frontier research.

Instead we must engage with those decision makers who dedicate their careers to leaving the world a better place than they found it. Those who take a long-term view. Those who enter public service to do something and not just to be someone. And thankfully they exist, and some of them care about and understand the importance of fundamental research. 

But let us recognise that our leaders and decision makers have difficult choices to make. The problem for any government is that they face almost unlimited demands with limited resources.

And investing in research is not like investing in anything else. It is easy for citizens to see what you are getting if you build a new road or a hospital or raise pensions. But it is not so easy to see what you are getting if you invest in research.

That is why sometimes we can be tempted to phrase our proposals as if we were not asking for investment in basic research at all. Instead we might be tempted to argue that we need funding to cure cancer or address climate change or land somebody on Mars or create the next Google.

These things are all possible. And I understand this approach. But I do not think it is necessary to argue like this for the value of frontier research.

For a start, we should be wary of over-promising and under-delivering. In the long run this can actually undermine support for research.

In reality what we are getting when we invest in frontier research is an increased understanding of the world, whether it be the physical world, the living world or the social world.

So our biggest problem is that people find this hard to grasp. An increase in understanding? What is the value in that?

But there are compelling reasons why increasing our understanding of the world is one of the best investments we can make.

Firstly, if I build, for example, a new bridge, some people will benefit from it as it will shorten their travel time. But if I have a new idea or a deeper understanding, then everyone in the world can benefit.

Paul Romer received the Nobel Prize in 2018 [1] for analysing the economic consequences of this insight. What he realised is that physical and human capital are rival goods: if a particular machine, or a trained engineer, is used in one factory, the same machine or engineer cannot be used at the same time in another factory.

Ideas, on the other hand, are non-rival goods. One person or firm using an idea does not preclude others from using it too. And this, in a nutshell, is why investment in frontier research can reap huge benefits far beyond any other type of investment. It is like creating a bridge that can be used anywhere, at any time.

So, the fact that ideas can be used over and over again is amazing in itself, but the good news does not stop there. Because secondly, investing in new knowledge has another tremendous advantage: the possibility of combining each new understanding we acquire with all other knowledge we already have.

Paul Romer calls this “combinatorial explosion”. And this is what has driven the huge rise in living standards that we have seen over the last two centuries. Put simply, the more knowledge we acquire, the more useful combinations become possible. And this is why solutions can come from unexpected places. New findings in one area can open up new opportunities in different areas. Research advances on a broad front, an unpredictable front. Solutions can come from unexpected places. Putting all our resources into priority areas can therefore paradoxically lower our chances of achieving progress, even in those very areas. So the slow accumulation of knowledge delivers compounding results and does not depreciate as other “investments” do.

Thirdly, science has another amazing quality which makes it a great investment. The great thing about science is that we can perform a whole series of trial and error experiments and if we discover an improvement, we can retain it and discard the rest. “ Critically we have the option, not the obligation to keep the result, which allows us to retain the upper bound and be unaffected by adverse outcomes” . Imagine again our bridge. It has to work. Otherwise the investment is completely wasted. This is not the case for investment in new knowledge. Here a “failure” can be a positive result.

But we have still not finished with the good news: the process of steadily accumulating knowledge in itself is the best way to train the highly skilled knowledge workers which our economies increasingly need.

The channels by which frontier research feeds into the economy (or makes an “impact”) are many and diverse. It is not just about the occasional breakthrough. Basic research increases the stock of useful knowledge, both the kind which is written down (e.g. scientific publications) and the kind that people carry around in their heads (e.g. skills, knowhow and experience). It trains skilled graduates and researchers in solving complex problems, produces new scientific instruments and methodologies, creates international peer networks for transmitting the latest knowledge and can even raise new questions about societal values and choices.

And you do not have to take my own word for this.

Countless studies have looked into the return on investment of funding frontier research and found that it is very high. When Mario Monti was asked to lead a high level expert group into the future EU budget he identified two areas that had the highest EU added value and these were research and security.

But this is nothing new.

For over 200 years economists have been studying the classical factors of production: land; labour; and capital. But, starting with Robert Solow (who won a Nobel Prize for this work), economists in the 1960s and 70s came to realise that at most, only half of the historical growth could be explained by the known factors. The rest could only be explained by positing a new factor of production: technological progress.

Nobody now disputes this claim. The issue is therefore how best to support technological progress. And here again there is a high level of consensus. Firstly, it is accepted that technological progress requires both frontier or curiosity-driven research and applied research. Secondly, it is accepted that governments need to fund frontier research. That is, because the applications of such research cannot be foreseen and there is possibly a long time-lag between fundamental discoveries and their exploitation. This means the private sector does not have the right incentives to fund it.

And again very few now dispute this form of “division of labour”. According to the OECD’s latest innovation strategy from 2015, “ public investment in scientific research is widely recognised as an essential feature of effective national innovation systems. Public research plays a key role in innovation systems by providing new knowledge and pushing the knowledge frontier. Universities and public research institutions often undertake longer-term, higher-risk research and complement the activities of the private sector. Although the volume of public R&D is less than 30% of the total OECD R&D, universities and public research institutes perform more than three-quarters of total basic research.”

So next time you have to explain the importance of funding frontier research, maybe you should ask:

• What other type of investment generates the same level of returns?
• What other type of investment compounds in value and does not depreciate?
• What other type of investment produces value even when it produces negative results?
• What other type of investment trains the people we need for the knowledge economy and allows us to access knowledge discovered elsewhere? 

The European Research Council is based on this understanding. The ERC supports excellent scientists from anywhere in the world, of any age and from any field of research - including the social sciences and humanities. There are no predetermined targets or quotas. The ERC provides substantial, long-term funding of up to 3.5 million euros for up to five years. The only conditions are that ERC funded researchers must be based in Europe and willing to be adventurous and to take risks in their research.

The philosophy of the ERC rests on the idea that researchers know best the most promising research areas to explore. We do not ask our researchers to tell us what societal problem they are going to solve or what impact it will have. Our belief is that, without understanding there can be no real solution to problems.

So we are absolutely not saying we don’t care if the work that we fund has any impact. We are saying that giving freedom to researchers is the best way to get the most impact.

The ERC has just published its own analysis of all projects it funded under Horizon 2020 [2]. A series of fact sheets show the diversity of the funded research with projects in many emerging areas of science. But this also showed that 34% of the analysed ERC projects are likely to contribute to health policies, including in cancer, brain and human mind research. One in ten projects addressed problems linked to the digital transition, half of which were in the area of artificial intelligence. And 14% were found to be relevant to climate policies and green solutions.

So not only do we see that ERC grantees push the frontiers of knowledge, but the study also highlights that this knowledge is actively contributing to political priorities. Nobody told grantees to go in that direction.

This report refutes the view that you have to tell researchers what to do because otherwise they’ll never get down to practical matters and urgent problems. Nothing is further from the truth! So my message to all research policy makers is: trust researchers and give them the means to pursue their best ideas! That’s the best investment in our future.

In his 1945 report to the President of the United States, Vannevar Bush called for an expansion of government support for science, and the creation of the National Science Foundation. Famously he stated: “ Scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown. Freedom of inquiry must be preserved under any plan for Government support of science.”

The fact is that by understanding the world we can change the world.

It is clear then that researchers must have the freedom to explore and understand the world as it is. That is why funding cannot be short-sighted. To maintain a healthy research system, it is right to invest in long-term curiosity-driven research. Some consider this approach to be idealistic. But I consider this approach to be pragmatic and necessary in order for science to have its maximum impact for the benefit of society. And while I am President of the ERC I will continue to make this argument for as long as it takes to be heard.c

[1] https://www.nobelprize.org/uploads/2018/10/popular-economicsciencesprize2018.pdf

[2] https://erc.europa.eu/news/mapping-ERC-frontier-research

Title Press contact

Madeleine Drielsma Press and Communication adviser T: +32 2 298 76 31

Research Topics

• For Potential Research Topic Contributor
• For Topic Editors
• Troubleshooting RT
• {0} More...

I have submitted an article already to the Research Topic, why am I still getting reminders?

Possible reasons and solutions if reminders to submit a manuscript are still received by the author after the actual submission.

Can Topic Editors submit to their Research Topics?

Clarification on what the topic editor can do about authoring in their own Research Topic and how topic editors should handle manuscripts of their co-topic editors

How can I go to my Research Topic's management tool?

Instructions on how the topic editor can find his Research Topic's management page in the My Frontiers interface

How can I search for contributors

Instructions on how a topic editor can use the Frontiers system to obtain contributor suggestions for his Research Topic

What are Research Topics?

Frontiers’ Research Topics are peer-reviewed article collections around themes of cutting-edge research. Defined, managed, and led by renowned researchers, they unite the world’s leading experts around the hottest topics in research, stimulating collaboration and accelerating science. Managed and disseminated on Frontiers’ custom Open...

What are the essential steps of hosting a Research Topic?

Please see this page https://www.frontiersin.org/about/research-topics for details on steps involved in proposing and hosting a Frontiers Research Topic.

How do Research Topic deadlines work?

Clarification regarding the deadlines of Research Topics

Is there an upper limit to the number of manuscripts accepted for publication in a Research Topic?

No, there is no upper limit to the number of manuscripts. Some research topics have received more than 50 papers!

How is the publication fee calculated for Research Topic articles ?

Clarification regarding fees for manuscripts submitted to a Research Topic

I can't accept to participate to a Research Topic (different email address)

What to do if the address in the Research Topic's call for participation does not match my Frontiers account

Related Topics

About Frontiers   |   Contact   |   Terms & Conditions

© 2007 - 2022 Frontiers Media SA. All Rights Reserved

• FRONTIERS Residency Program Guide

FRONTIERS is the science journalism initiative funded by the European Research Council that offers grants for journalists from all over the world seeking to develop their professional skills in covering complex scientific topics by spending 3 to 5 months in residence in a European research institution performing frontier research in any discipline, including social sciences and humanities.

The FRONTIERS project is scheduled to run from 2023 to 2027. Throughout this period, up to 40 science journalists will have the opportunity to spend time with research teams and pursue their own reporting ideas, in total independence, at an institution of their choice.

FRONTIERS journalists in residence primarily focus on ‘Frontier research’ and engage with inquiries that reside at the cutting edge of available knowledge. Often characterized as high-risk/high-reward endeavors, ‘Frontier research’ may be complex to elucidate and particularly challenging to present to the public in a balanced and responsible way.

The Residency program guide provides essential information and resources for preparing a successful application. In between calls, this guide may be updated based on the valuable feedback and insights provided by project participants all along the project duration.

For technical information about the Financial Support for Third Parties (FSTP) mechanism for the FRONTIERS grants, visit this page.

Contributions from participants will help reshape and refine this guide, ensuring that it remains a living document that evolves alongside this collective journey. Suggestions for improvement can be sent to the FRONTIERS contact point at [email protected] .

Program’s main objectives

In its commitment to supporting independent science journalism, FRONTIERS aspires to play a pivotal role in supporting the professional development of science journalists.

This is of paramount importance in Europe, serving as a crucial bridge between the scientific community and the broader public. In a rapidly evolving world where science and technology play an ever-increasing role in our daily lives, the role of independent science journalists is more vital than ever. Their ability to accurately interpret and communicate complex scientific findings is essential in fostering a well-informed public, capable of making informed decisions about personal choices and scientific policies.

In Europe, with its rich diversity of cultures and languages, the challenge for science journalists is even greater. They must not only stay abreast of the latest scientific developments but also be adept at tailoring this information to a varied audience. Professional development opportunities, such as the FRONTIERS project, enable journalists to hone their skills in critical thinking, clear communication, and ethical reporting. Moreover, as misinformation and disinformation become increasingly prevalent and some important scientific topics – including social sciences and humanities – do not reach the wider audience of citizens, the role of trained and independent science journalists is invaluable. They are instrumental in building public trust in scientific institutions and in the scientific method itself. By investing in their professional growth, Europe ensures that its citizens have access to reliable and accurate scientific reporting, which is essential for a resilient, healthy and democratic society. This investment not only benefits the field of journalism but also enhances public understanding and engagement with science, ultimately contributing to more informed policymaking and a more scientifically literate society.

What we offer

The FRONTIERS project offers fellowship residencies to journalists interested in spending 3 to 5 months in one or more European research institutions to increase their knowledge in a specific ‘Frontier research’ field of science or to carry out research for their own production (science journalism projects, books, or any other professional goal), in a totally independent way.

The FRONTIERS’ bottom-up approach allows journalists to apply for residencies in any European research institution covering any field of frontier science, including social sciences and humanities. Within the 3-5 months of the residency, journalists can ask to spend shorter periods of time in one or more other research institutions to further enrich their experience and improve their knowledge in the selected field of frontier research. 

During the project, training and networking activities will be developed, connecting stakeholders, disseminating good practices in science journalism, and creating bridges between journalists, researchers, and institutions. Selected fellows are expected to take part in these activities, before, during and after their residency.

The program design ensures the value of the initiative for both the research institution and the journalist while maintaining the journalists’ independence and credibility.  

Eligibility criteria

Journalists.

Nationality/Country of residence: While primarily targeting residents in Europe and associated countries, the FRONTIERS program will also consider applications from science journalists of all nationalities residing elsewhere, who are willing to spend time in European research institutions and better understand the European scientific landscape.

Professional criteria: Eligible participants include science journalists, defined as reporters, writers, editors, producers, illustrators, filmmakers, and photojournalists working across various media, including self-managed social media channels so long as they produce independent journalistic content. Journalistic content can have any format, from voice to video, text and pictures as mixed formats. The FRONTIERS consortium and Advisory Board are aware that in some contexts the boundaries between science journalism and science communication are not always clearly defined, and will not rule out applications by candidates with mixed profiles and careers, provided that at the time of the application they are recognised as journalists in their professional context.

Commitment: During the residency, selected applicants are expected to focus fully on their project, and to refrain from outside professional work. Applicants who are selected to become fellows are expected to actively participate in training and networking activities organized within the FRONTIERS project framework, and to take part in communication and dissemination activities (such as participation in a short video about their experiences during the residencies, and takeover of social media accounts for a short period of time). They are also expected to provide feedback on the residency as requested by the FRONTIERS staff, during and after the residency. 

Multiple applications: Journalists may submit a single application per call. Should more than one application exist, only the last one to be submitted will be considered. Applicants who have not been selected in one call may apply again in the next call. Science journalists selected for a residency will not be allowed to apply again, even if they are for some reason unable to complete the residency.

Seniority/career level: FRONTIERS grants differentiate among three levels of career, based on the professional seniority (see Scholarship and taxation section below):

• early-career : up to 5 years of professional experience;
• mid-career : 6-9 years of experience;
• established: ten or more years of experience.

Resolution of controversies: Fellows are also expected to report immediately to the FRONTIERS team ( [email protected] ) all controversies that should arise with the host institution during or after the residency. 

For any questions or further details, the FRONTIERS manager can reach out to the FRONTIERS Coordination and Support Office via e-mail ( [email protected] ).

Host institutions

Premise: host institutions may support more than one journalist’s application in the same call. However, since diversity of host institutions will be encouraged, it is unlikely that one host institution will host two FRONTIERS fellows. Host institutions will be able to host more than one fellow selected in different calls.  

Location: host institutions should be a legal entity based in the EU Member State or in a country associated with the EU’s Horizon Europe Programme by the call deadline.

Focus: institutions must host one or more research groups who are performing, or have performed in the past, frontier research in any field. Hosting current or past ERC grantees is a plus but is not required. 

Requirements: host institutions have to commit in writing to fulfil the basic requirements listed below, in order to help science journalists in residence have a fruitful and enriching experience in carrying out their project in total independence, in a welcoming and safe working environment. Basic requirements include: 

• Badge or other forms of credentials granting access to the relevant premises; 
• Access to the cafeteria/canteen and other shared areas under the same condition as senior research staff; 
• A working space with access to wi-fi; 
• Access to the institution’s library and electronic resources (such as books, scientific journals and databases); 
• Access to all events and activities organized for the institution’s staff.  

Contribution: host institutions are expected to support to the best of their possibilities the science journalist in residence before and during the residency, and to discuss with them all possible ways to organize lectures, seminars, workshops, round tables on science journalism and science communication for the institution’s staff or for the wider public.

Independence: host institutions are required to respect and cherish the independence of the science journalist in residence. Science journalists in residence are not expected and will not provide any support for institutional communication, and are not expected to provide coverage of the host institution’s activities, neither during nor after the residency, except for fulfilling their project and their own goals. 

Multiple applications: host institutions are allowed to support more than one application in each call, but must inform candidates about concurrent applications, warning them that only one fellowship can be awarded to each institution in each call. Institutions that have hosted a science journalist in residence are allowed to support one or more applications in the following calls, knowing that the evaluation process is inspired by inclusion and balance, and will give priority to institutions that have not participated yet.

Management: the host institution will identify a ‘FRONTIERS manager’, who will be responsible for the management of all aspects of the residency. The FRONTIERS manager could be the scientist hosting the journalist, his/her laboratory or department head, a person from the communication office, etc. The FRONTIERS manager can be supported by a Deputy FRONTIERS manager.

Feedback: the FRONTIERS manager identified by the host institution is expected to provide feedback on the residency as requested by the FRONTIERS staff during and after the residency, via e-mail, phone and/or other communication tools.

Resolution of controversies: host institutions are expected to report immediately to the FRONTIERS team ( [email protected] ) all controversies that should arise with the science journalist in residence during or after the residency. 

For any questions or further details, contact the FRONTIERS Coordination and Support Office via e-mail ( [email protected] ).

Role of the FRONTIERS Coordination and Support Office (CSO)

The FRONTIERS consortium supports both science journalists and host institutions with a centralized Coordination and Support Office (CSO) both in preparing the application itself and in managing possible controversies during and after the fellowships.

The duties of the CSO include:

• the development and maintenance of a database of possible host institutions and researchers, listing the institutions that already expressed their interest in being included. The institutions will be included in the database if they submit their request via the online form on the website https://frontiers.media;
• the matchmaking service for journalists who are searching for a suitable research institution that fits with the goal of their project, if they are unable to find one on their own;
• the support to the fellows, the hosting researchers and the host institution.

Role of the Advisory Board

The FRONTIERS Advisory Board plays a pivotal role in ensuring the integrity and independence of journalistic practices within these settings. 

This board, composed of experts from both the journalistic and scientific communities, contributes to establishing clear guidelines and protocols that safeguard the journalists from any undue influence or pressure from their host institutions. By doing so, they preserve the essential objectivity and critical perspective that journalists bring to the reporting of scientific endeavors. 

Furthermore, the Advisory Board is responsible for overseeing a rigorous and fair evaluation process for selecting journalist candidates. This process involves assessing each candidate’s journalistic credentials, commitment to unbiased reporting, and ability to effectively communicate complex scientific concepts to the public. 

Through these responsibilities, the Advisory Board ensures that the fellowship program not only fosters a rich exchange between journalism and science but also upholds the highest standards of journalistic independence and excellence.

The composition of the Advisory Board is available at: https://frontiers.media/about/advisory-board.

How to apply

Journalists wishing to apply for a FRONTIERS fellowship must identify and contact an eligible research institution that is willing to host them and accept the terms described in this guide. To initiate the application procedure, potential applicants should find the application form available on the FRONTIERS website and provide the required information, uploading all necessary documentation, which includes the proposed project for their residency.

Applicants who have not yet identified a host institution or researcher can search the database of potential host institutions available on the FRONTIERS website and explore possible matches by contacting the provided contact points at the chosen institution, or the FRONTIERS support office, which will provide assistance. 

Note that only applications that fulfil all specified criteria and include the required documentation will be considered for further evaluation.

The FRONTIERS team is available to provide support and guidance for the application process. Please feel free to contact us at [email protected] for any questions related to this topic. 

Structure of the application form

• Applicant Journalist Name 
• Applicant Journalist Nationality
• Applicant Journalist Email
• Applicant Journalist Phone Number
• Career level application (early career, mid-career, established) – Please see the section above, “Eligibility Criteria”, to determine the level of your application 
• Host institution Name
• Host institution Country
• Name of FRONTIERS Manager at Host Institution 
• Email of FRONTIERS Manager at Host Institution
• Scientific Domain (Physical Sciences and Engineering, Life Sciences, Social Sciences and Humanities, Multidisciplinary);
• Support documents (upload):
• Residency project proposal: a PDF file detailing the project to be developed during the residency in the host institution.
• Work samples: in a single PDF file, provide up to 10 relevant work samples from the last two years. Choose samples that best illustrate your interests and abilities. For audio and video samples, you may provide links, ensuring they are open. Please include a translation for any work not produced in English. 
• Recommendation letter(s): in a single PDF, provide up to 3 recommendation letters by people familiar with your work. 
• Commitment letter by the science journalist: provide a commitment letter, based on this template signed by the applicant journalist.
• Commitment letter by the host institution: provide a commitment letter, based on this template , signed by a representative on the letterhead of the candidate host institution. 

Structure of the Journalist’s project

The project proposal presented by the applicant journalist should have the following structure, in this order

• CV of the applicant in narrative form highlighting the professional career, scientific background, spoken languages, personal interests, and any other detail the applicant considers important for a successful application (up to 500 words).
• Details on the host institution, the hosting researcher, and the contact person in the institution.
• Letter of intent: the applicant should describe in 300 words maximum why he/she/they are applying to the FRONTIERS program.
• Project description (up to 500 words): this section should include the topic(s) of interest; a description of the activities that will take place during the fellowship (active participation in the lab life, journal clubs etc; interviews; personal research; attendance/organization of courses at the host institution, etc).
• Impact (up to 200 words): expected impact of the project on the journalist’s career; expected societal impact of the residency (related to the topic, to the involved stakeholders, etc).
• Dissemination ( not mandatory ): if the journalist is planning to produce books, videos, articles, social media posts, documentaries, long-form, story series, multimedia, audio, installation etc, the dissemination section of the proposal (up to 200 words) will be considered by the evaluation committee.
• Budget: a tentative budget of the project, including life expenses, will help the committee to provide a reasonable economic support for the selected applicants. Scholarship and taxation below.

Evaluation procedure

Within the framework of the FRONTIERS project, the selection procedure relies on several key criteria, each of which plays a pivotal role in determining successful applicants. These criteria encompass the quality of the project, the strength of the candidate’s resume, the expected impact of the project, ensuring no more than one journalist per host institution in each call, striving for diversity in residency in terms of gender, nationality, and geographical distribution, and giving priority to applicants who have already established strong connections with scientists and their institutions.

Applications compete with other applications within the same career stage category, meaning that early-career journalists will not be competing with experienced or established journalists. 

The FRONTIERS project promotes equality and fairness principles. In order to have reasonably equal participation in terms of gender (journalists and scientists), geographical distribution across Europe and scientific areas covered by the residency program, the selection procedure is designed to include two steps. 

Step 1 of the evaluation

Step 1 of the evaluation ranks all applications based on the criteria listed below. All the applications that have received a positive evaluation pass on to the second step.

Evaluation Criteria for Step 1 

The evaluation criteria focus on the excellence and potential impact of the applications in the following dimensions, as presented in the project submitted by the applicant:

Residency criteria (max 40 points)

• Objectives and scope (up to 10 points)
• Foreseen contribution to:
• The diffusion of research results (up to 10 points)
• The journalists’ career (up to 10 points)
• Increasing trust in science journalism and in science (up to 10 points)

Journalist criteria (max 40 points)

Journalist’s track record

• Expertise and experience and their relevance to the field to be covered in the project (up to 10 points)
• Journalistic publications: stories for reporters/writers or role in journalistic projects for producers/editors (up to 10 points)
• Commitment to journalism, in terms of involvement in networking and mutual support with other science journalists, nationally and internationally (up to 10 points)
• Previous participation in science journalism events (professional conferences, festivals, workshops) (up to 5 points)
• Previous interactions with researchers (such as in-depth interviews, periods spent in research facilities, and living with researchers) (up to 5 points)

Research criteria (max 20 points)

Frontier science

• Groundbreaking and cutting-edge research work (up to 10 points)
• Value for society (up to 10 points)

Step 2 of the evaluation

Step 2 of the evaluation then proceeds to compose a balanced list of up to ten candidates, starting from the top-scoring applications, with a final composition of candidates equally distributed between young, experienced and established journalists, and in terms of gender (journalists and scientists), geographical distribution and scientific areas covered by the residency program.

The 10 pre-selected candidates will be invited to receive financial support. Should one or more of the pre-selected candidates waive the support, the selection committee will try to identify an applicant suitable for integrating the respective list while preserving as much as possible the balance.

Evaluation and selection committee

The evaluation and selection are managed by a committee composed of 3 members of the Advisory Board, that will rotate among the different calls, and one representative of each of the four FRONTIERS partners. Committee members will be invited to disclose all potential conflicts of interest that should arise, and the committee as a whole will deal with them appropriately.

Applicants must provide an email address and a phone number through which they can be contacted (if needed) both by the FRONTIERS team and the FRONTIERS manager at the host institution.

The FRONTIERS team will be available to answer questions (at [email protected] ) related to different issues:

• administrative (e.g., payments, unexpected costs, etc.).  
• logistics (e.g., lack of access, working conditions, etc). 
• educational/training (e.g., attendance to courses etc).
• reporting controversies (e.g., disagreements on the scope of the residency).

Scholarship and taxation

Selected fellows will be entitled to receive a monthly payment covering travel, accommodation, any potential taxation (applied on the individual and/or the institution) and eligible daily expenses up to 4,000 euro (early-career) 5,000 euro (mid-career), or 6,000 euro (established), based on EU rules.

These grants are final and will not be increased under any circumstances.

Rights and duties 

Rights and duties of the journalist in residence.

For prospective participants in the FRONTIERS project, a set of rights and duties are in place to guide their involvement. These criteria include being an active journalist and committing to reside in close proximity to the host institution throughout the entire residency period. It is essential to spend time at the host institution as planned in the project, actively engage with researchers and other professionals at the institution and participate in various social and cultural events organized by the host institution.

FRONTIERS fellows should be prepared to participate in training sessions organized by the FRONTIERS project and collaborate with the host institution(s) on activities that promote discussion on topics related to science and the media, such as seminars, workshops, and round table discussions.

FRONTIERS fellows are expected to work alongside the consortium in raising awareness on frontier science and independent science journalism, while helping to spread the word about the FRONTIERS residencies among science journalists. The collaboration may include sharing testimonies or pictures to be used on FRONTIERS communication channels. Fellows will also be invited to take over one of FRONTIERS’ social media accounts for periods of one week. Some suggestions will be provided but total freedom will be ensured.

Additionally, FRONTIERS fellows need to be responsive to questionnaires and interviews conducted by the FRONTIERS team.

In the event of disputes with the researcher or host institution on matters concerning the residency, a concerted effort to find mutually acceptable solutions is essential. Participants should promptly report any relevant incidents that might impede the completion of the residency, and they should be available for any necessary follow-up.

FRONTIERS fellows must maintain accurate records of eligible expenses, following the guidelines provided by the FRONTIERS consortium. Before commencing the residency, participants will be asked to sign a letter of commitment, signifying their agreement to abide by these ‘rules of engagement’

Rights and duties of the host institution

Host institutions play a crucial role in welcoming FRONTIERS fellows and integrating them into the institution by providing them with access as requested by their project. A strong support from the institution is essential in ensuring journalistic independence, allowing them to pursue their projects with autonomy.

Host institutions are supposed to engage FRONTIERS fellows by inviting them to participate in activities that foster discussion on the intersections of science and the media. This could involve their involvement in seminars, workshops, and round table discussions. Additionally, host institutions should consider giving them the opportunity to present a seminar on a topic of their choice, as their unique perspectives can enrich the discourse.

As part of this collaborative effort, host institutions should also be prepared to provide responses to questionnaires and interviews conducted by the FRONTIERS team, which helps in assessing the impact of the program.

Furthermore, we invite host institutions to collaborate with the FRONTIERS team to document the FRONTIERS residency, namely through the institution’s Communication/Press Office. This collaboration may include writing a profile of the journalist for the FRONTIERS website, allowing a broader audience to understand their background and aspirations, or providing content for social media. Collecting footage to document their residency may be a valuable way to capture the experience and share it with a wider audience. Further information will be provided before the start of the residency.

Last update: 25 January 2024 Changed text from “Should more than one application exist, only the first one to be submitted will be considered” to “Should more than one application exist, only the last one to be submitted will be considered”.

How much would you like to donate?

Research Frontiers

Solving challenges emerging in a rapidly changing world >>>.

The university has invested in major research initiatives, bringing together diverse expertise that transcends traditional discipline boundaries. In partnership with industry, government, and foundations, these focus areas address emerging challenges and opportunities that seek to improve the human condition and create a better world for all. 

These strategically focused areas are Virginia Tech Research Frontiers.

Championing Research

“Our goal is to win more large grants in these research areas,” said Dan Sui, senior vice president for research and innovation. “Such awards will progress the university’s expenditures, reputation, and ranking, and accelerate President Sands’ goal to position Virginia Tech as a top global land grant university."

Virginia Tech Research Frontiers

The Artificial Intelligence Frontier

Harnessing artificial intelligence for intelligence augmentation

Building on expertise in artificial intelligence and data science, systems engineering, neuroscience, human factors, robotics, immersive visualization, and education, among others, to accelerate human-technology partnerships toward seamless augmentation, ethically and sustainably.

The Health Frontier

Leading one health to achieve whole health

Changing the focus on disease and symptoms to one of whole health, integrating intersections of animal, environment, and human health and building in communities and systems to empower multifaceted well-being.

The Security Frontier

Innovating for secure and resilient communities

Ensuring communities are prepared to face global threats, from climate change to cybersecurity to national defense through advances in preparation, defense, mitigation, and recovery.

The Quantum Frontier

Advancing the quantum leap

Using an unparalleled transdisciplinary focus to accelerate the integration of quantum technologies across society, realizing unprecedented computing and communication capabilities and restructuring our social framework.

Virginia Tech NATIONAL SECURITY INSTITUTE FORMED

“As chairman of the Senate’s Select Committee on Intelligence, I am pleased that Virginia Tech is strategically organizing and prioritizing its national security research and workforce efforts,” said U.S. Sen. Mark Warner. “Given the university’s nearly $50 million Department of Defense research portfolio, and its strategic locations in Northern Virginia near key national intelligence agencies and the Pentagon, this purposeful focusing of Virginia Tech’s efforts in national security is welcome news. Virginia Tech’s new National Security Institute will help our nation develop new security-related technological advancements while helping train the future generations of intelligence leaders.”

Northrop Grumman commits $12.5 million toward quantum research and education

Virginia Tech faculty, students, and collaborators are working together at the intersection of quantum disciplines to drive quantum mechanics from the lab into real world applications. The new partnership with Northrop Grumman positions the university to dramatically increase the scope of its transdisciplinary quantum-related research and teaching.

Machine Learning Research Partnership with Amazon

As part of the Amazon - Virginia Tech Initiative for Efficient and Robust Machine Learning initiative, Virginia Tech faculty members will be invited to submit machine learning sponsored research projects for Amazon that work to revolutionize the way the world uses and understands this field of modern technology. Research project proposals may include support for students and staff.

The Miraculous, Imperfect Mechanics of Pregnancy

Raffaella De Vita's STRETCH Lab studies pregnancy's constant remodeling of reproductive organs and structures — and how those biomechanical changes may contribute to injuries and disorders that affect millions of women.

Page 1 of 25 | 100 Results

Research Topics

Frontiers' Research Topics are collaborative hubs built around an emerging theme. ​ ​ Defined, managed, and led by renowned researchers, they bring together leading experts around a shared area of interest – stimulating collaboration and accelerating science.

Managed and disseminated on Frontiers' custom open science platform, topics are free to access and highly visible. They maximize the impact of your research by getting it out into the community so more people can read and discuss it.

Shape the future of your field

Become a guest editor for a topic around your own area of research. Benefit from increased impact and discoverability, a dedicated platform and support team, and rigorous peer review for every paper.

Suggest your topic

As a guest editor you will:

edit a topic around your chosen area

grow your network and collaborate with leading researchers around the world

track impact in real-time with advanced impact metrics

ensure quality through rigorous, transparent and fast peer review

increase visibility with the possibility of a freely shareable ebook.

More impact

Thanks to Frontiers' platform, anyone and everyone around the world will have free and unrestricted online access to your Research Topic.

Each topic has its own page and can be cross-listed from multiple Frontiers' journals. Plus you can track your impact in real-time with our advanced impact metrics.

More collaboration

Collaborate with colleagues from around the world to propose a topic. We encourage early career researchers to team up with more senior colleagues.

Grow your network by inviting key experts to contribute to your topic and ensure the quality of every article with our collaborative peer review – a unique online forum with real-time interactions.

More support

Managing a Research Topic is simple with our innovative technology and professional support team. We'll support you through every step of the process, from defining your topic and inviting contributors to promoting your topic and tracking impact.

With Frontiers you also retain the copyright of your work by publishing under the CC-BY license .

Submit your topic

Suggest your topic, or ask for more information, by contacting our editorial team .

We welcome teams of guest editors to propose Research Topics. Early career researchers are encouraged to team up with senior colleagues.

Publication fees

Open access provides free, unrestricted online access to research for everyone in the world. Frontiers is a gold open access publisher. This means that we maintain high quality services through article processing charges (APCs). Articles accepted for publication following peer-review by our external editors will incur a publishing fee.

Agricultural frontiers and environment: a systematic literature review and research agenda for Emerging Countries

• Open access
• Published: 25 October 2023

Cite this article

You have full access to this open access article

• Arthur Pereira Sales   ORCID: orcid.org/0000-0002-2000-6325 1  

755 Accesses

Explore all metrics

Despite having the largest land and water reserves for agriculture on the planet, intensive agricultural production in emerging countries has stimulated research around the world, especially due to the numerous environmental impacts caused by the expansion of agricultural frontiers. Motivated to analyze the literature on the transformations brought about by the development of intensive agriculture since the middle of the twentieth century, this study analyzes the main studies on the interference of agricultural frontiers on the environment in emerging countries over the last 30 years (1993–2022). To do so, the Systematic Literature Review methodology was used, with the CIMO planning approach and the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) reporting guideline. The analysis initially included 14,366 scientific articles from a wide range of subjects in the social and natural sciences, available on the Web of Science (Clarivate Analytics), ScienceDirect (Elsevier), and Google Scholar databases. One of the most significant findings of this study is that there is no specific framework to analyze the relationship between the agricultural frontier and the environment in developing countries; however, literature has mainly been concerned with measuring the impact of intensive agriculture on natural resources, as well as verifying how local socio-economic factors and/or public policies affect populations’ behavior regarding this relationship between the environment and agricultural production. The data also revealed that Brazil is the “country of origin” of the literature on agricultural frontiers and the environment, especially due to studies on the Amazon rainforest, followed somewhat distantly by studies on South America in general and the island regions of Indonesia and Malaysia. There is also a lack of studies on European economies in transition, emerging African countries and Russia, or on the agri-environmental impact of the demand for food in populous countries such as India and China. Finally, in addition to country-specific suggestions, this systematic literature review suggests directions and implications for future research.

Similar content being viewed by others

Impact of agricultural land use in Central Asia: a review

Ahmad Hamidov, Katharina Helming & Dagmar Balla

The Future of New Forms of Urban Agriculture: A Call for Evidence

Introduction

Avoid common mistakes on your manuscript.

1 Introduction

The total weight of grains produced in the world adds up to over 3 billion tons, but one of the characteristics of global agricultural production is the concentration of a few products and countries. The three main commodities (corn, rice, and soybeans, respectively) account for two-thirds of all grains produced in the world, especially in emerging countries, in which China, the United States, Brazil, and India, for example, accumulate 54% of all global grains. In addition, with the largest land area in the world, Russia is a leading producer of wheat, barley, and sunflower, with 124 million hectares of the country’s area under cultivation. Indonesia and Malaysia together produce around 95% of the world’s palm oil, while Argentina, Bolivia, and Paraguay, as well as excelling in soybean production, are Latin America’s leading beef exporters, alongside Brazil (FAO, 2021 ).

The main drivers of the expansion of agricultural frontiers in emerging countries are the current global demand for agricultural products, the flexibility of environmental regulations, foreign investments, the reduction of production costs, as well as poverty, energy, and food dependency, which consequently aggravate environmental problems (Gibbs et al., 2010 ; Feintrenie, 2014 ; Villela et al., 2014 ; Avagyan, 2018 ; Ibrahim et al., 2022 ; Jahanger et al., 2022 ; Makhdum et al., 2022 ; Usman & Balsalobre-Lorente, 2022 ; Usman et al., 2023 ).

Increasing rates of deforestation, polluting gas emissions, food waste, loss of animal biodiversity, and water pollution, among others, are some of the problems arising from unsustainable agricultural production in emerging countries (Avagyan, 2010 , 2017 and 2021 ; Adegbeye et al., 2020 ). Another problem to be mentioned is the increasing use of fertilizers and pesticides (Avagyan, 2018 ). According to Schreinemachers and Tipraqsa ( 2012 ), there has been an increase in production and consumption in countries such as Brazil, Mexico, Uruguay, Cameroon, and Malaysia, due to the visible changes in the soil, while there is no concern about environmental issues and population’s food security.

Such environmental problems could be addressed by using wood waste and agricultural residues to produce board, binderless board, and paper, or else by converting these organic residues to clean fuels and/or petrochemical substitutes via pyrolysis. Organic waste may be also converted chemically—by hydrolysis—into different types of sugar, which may be fermented to generate bioethanol. Moreover, such residues may be used for composting (Fahmy, 1982 ; Fahmy et al., 1982 ; Mobarak et al., 1982 ; Mobarak, Fahmy, & Schweers, 1982a , 1982b ; El-Shinnawy et al., 1983 ; Mobarak, 1983 ; Fahmy & Mobarak, 2013 ; Fahmy et al., 2017 and 2020 ). Another way to attempt to minimize environmental problems, as well as the storage and production of clean energy through simple and economically sustainable methods, is to adopt the use of new technologies, such as nanotechnology (Zinatloo-Ajabshir & Salavati-Niasari, 2019 ; Zinatloo-Ajabshir et al., 2019 , 2020 and 2022 ; Etemadi et al., 2021 ; Tabatabaeinejad et al., 2021 ; Zinatloo-Ajabshir & Mousavi-Kamazani, 2021 ; Heidari-Asil et al., 2022 ; Zonarsaghar et al., 2022 ).

For this study, the agricultural frontier is defined as an expression indicating the advance of intensive agricultural production over the environment; however, as the term “environment” itself is subjective, analyzing and measuring the impact of intensive agricultural production in locations as diverse as emerging countries can have many aspects and connections. Thus, the following questions arise: how can we synthesize the studies that link agricultural frontiers and the environment in emerging countries? Do the studies show that there is an equal interest in this topic among emerging countries? What would be the main indications and suggestions for future work?

To answer these questions, this article aims to identify and analyze the main research on the interference of agricultural frontiers in the environment in emerging countries, through a critical discussion of the theories used to follow the transformations that have occurred in the last thirty years (1993–2022). The choice of the period of analysis, as well as the justification for this work, was due to the curiosity and importance of analyzing the literature that involves the environmental impacts promoted by the development of intensive agriculture in the mid-twentieth century while presenting a recent discussion on sustainable agriculture in emerging countries and serving as a basis for future work. Moreover, as the “Agricultural Frontier” is a theme that is intrinsic not only to environmental problems, but also to psychological, social, and cultural aspects, it lacks a better-qualified debate. Thus, the choice to use the Systematic Literature Review is because it is an adequate method to evaluate and synthesize the best evidence on a given subject, and, in this particular study, it will work as a gap to fill the lack of an in-depth discussion on studies dealing with the environmental impacts of agricultural frontiers in emerging countries, since, to date, there is no systematic literature review which addresses such discussion.

2 Methodology

2.1 systematic literature review method.

According to James et al. ( 2021 ), a systematic review does a thorough search of the literature, evaluates the data found, and synthesizes the best evidence on a specific research question, to provide accurate and evidence-based information from the study. Therefore, to analyze the studies that encompass the theme of agricultural frontiers and the environment in emerging countries, this study used the Systematic Literature Review (SLR) methodology.

Initially, to plan this SLR, the CIMO approach (Denyer & Tranfield, 2009 ) was used as a search for scope and understanding of context “C,” intervention “I,” mechanisms “M,” and “O” outputs that surround the search, that is, this phase includes planning the research questions and defining the scope of the study. After understanding the planning process, it is necessary to adopt a protocol to be used in the selection of articles. This study adopted the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P), which consists of organizing and analyzing the results of previous studies, and identifying the main questions and problems addressed in the research (Moher et al., 2015 ).

Thus, this SLR consists of 7 procedures Footnote 1 : 1: search for possible articles following selected search queries based on expert recommendations; 2: search for potential papers by other sources; 3: implementation of inclusion and exclusion criteria; 4: analysis of duplicate papers; 5: selection of papers for first reading (title, keywords, and abstract); 6: selection of papers for full reading; 7: analysis of the synthesis.

2.2 Source of analysis and data

The Systematic Literature Review was conducted from 01/August/2022 to 30/November/2022 with articles published in the last 30 years (1993–2022). Data analysis was carried out as follows: Initially, 14,366 scientific articles from the most diverse disciplines of the social and natural sciences were observed through search queries on the environment and agricultural frontiers made available in electronic databases of the Web of Science (Clarivate Analytics), ScienceDirect (Elsevier) and Google Scholar platforms. The key terms used in the database search were: “agriculture* Footnote 2 or livestock or farming and frontier AND environment*.”

Then, articles that, even if not on the three main research platforms cited, are relevant to the subject, including articles of recommendations or already known, were verified. From there on, the inclusion and exclusion criteria were implemented, in which only scientific articles written in English, published between 1993 and 2022, with a citation number Footnote 3 of > 50 or with an average of 10 citations per year, and which had emerging countries as the area of study were analyzed. The key terms for the emergent countries analysis were more specific, namely: “Emerging countries OR Developing countries OR Latin America OR Transitional economies OR BRIC* OR Brazil OR China OR India OR Russia OR Malaysia OR Indonesia OR Argentina OR Mexico OR Turkey OR Poland OR Hungary OR Croatia OR South Africa OR Morocco OR Egypt.”

Following the inclusion and exclusion criteria, as well as the search queries, 138 articles were analyzed for possible duplicates using a reference management software—EndNote. From then on, the analyses were more carefully refined, in which 108 articles were selected for an initial reading of the title, keywords, and abstract, and, after a cutout of 42 papers, 66 articles were fully read. Finally, after reading them entirely, only 6 articles were excluded and 60 were ready for the literature review. The entire process is illustrated in Fig.  1 .

Source: own elaboration

Results of the scoping search—PRISMA flow diagram.

3 Geographical distribution and main topics

In total, 15 countries/regions are addressed in the selected articles about the relationship between the agricultural frontier and the environment, being Brazil is the most explored country, corresponding to more than 33% of the sample. In the second position are the articles whose area of study is South America in general (6 articles), followed closely by the island region of Indonesia/Malaysia with 5 articles, and Mexico, Indonesia (alone), and China with 4 articles each. Papers addressing Emerging Countries, in general, were only 2. The geographical distribution of the selected papers is illustrated in Table 1 .

In line with the methodology adopted in the articles, it was observed that the influence of agricultural frontiers can address several factors and be assessed in different ways, therefore, regarding the characteristics of the methodological process, most of the studies were empirical (46), while only 14 had a theoretical approach. The empirical articles used different research models and methods, including satellite mapping analysis (15 articles), linear regression models (10 articles), simulation models (8 articles), logit models (6), analysis of variance (3), probabilistic models (2 articles), case studies and mathematical model (with 1 article each). On the other hand, theoretical articles focused on discussing the topic at hand through the descriptive method (14 articles).

Although the articles had specific objectives, their main focus was to analyze the interference of agricultural production on nature, assessing this impact on the various environmental factors or analyzing the public policies and socio-economic aspects involved in the process. As illustrated in Fig.  2 , most authors were concerned with studying public policies for preservation or regeneration in agricultural frontiers (15 articles). The environmental impact on land use was the second most studied topic (10), followed closely by articles that had deforestation as their focus (7 articles). The interference of socio-economic factors in the relationship between the environment and agricultural frontiers or vice versa was studied in 5 articles, followed by studies that addressed the environmental impact on animal biodiversity (4 articles), on the measurement of greenhouse gas emissions (3), while only 1 article analyzed this environmental impact on water.

Key topics for environmental impacts on agricultural frontiers.

In addition, as “environment” is a subjective and broad concept, some articles (15 in total) did not have only one specific point, i.e., they addressed two or more themes in the analysis. Some articles jointly analyzed the interference of agricultural frontiers on land and water use (1 article), greenhouse gas emissions and land use (1), interference of socio-economic factors and water use (1 article), deforestation and animal biodiversity (1), deforestation and socio-economic factors (1), as well as deforestation and preservation or regeneration policies (3 articles). When the article had more than two themes as its main focus, it was defined as a general approach paper, and 7 papers as such were studied.

4 Results and discussion

The analysis of studies on agricultural frontiers in emerging countries brings a broad context, especially due to the distinct characteristics between these countries. Whether it is the number of inhabitants, the landform, the climate, or the politics, studying emerging countries means dealing with a diverse range of aspects and issues. Thus, Sect.  4.1 presents an overview of the main research conducted in emerging countries, emphasizing what the authors are concerned with studying in each country.

4.1 Key Studies in Emerging Countries

Considered the source of biodiversity in the world, the data reveal that Brazil is the home country of studies on the environmental impacts promoted by intensive production, especially because of the Amazon Rainforest (Nepstad et al., 2001 , 2006 and 2008 ; Mertens et al., 2002 ; Soares-Filho et al., 2002 and 2004; Rodrigues et al., 2009 ; Pacheco, 2009 ; Macedo et al., 2012 ; Schiesari et al., 2013 ; Verburg et al., 2014 ; Ochoa-Quintero et al., 2015 ; Nobre et al., 2016 ). The paradigms addressed by studies on the Brazilian Amazon are vast and range from loss of animal biodiversity, measurement of deforestation, and land use to analysis of forest preservation policies.

Nepstad et al. ( 2001 ) and Soares-Filho et al. ( 2004 ) studied the impacts of road paving on deforestation. Rodrigues et al. ( 2009 ) analyzed how human development levels interfere with deforestation in the region. In turn, Mertens et al. ( 2002 ) measured the deforestation of the Amazon due to cattle ranching, and Ochoa-Quintero et al. ( 2015 ) the loss of native species caused by environmental degradation. Regarding the implementation or effectiveness of public policies, Nepstad et al. ( 2006 ), for example, compared inhabited and uninhabited reserves in the process of slowing the conversion of land to agriculture. Pacheco ( 2009 ) studied the impact of land reform and agrarian structures on deforestation in the region. Nepstad et al. ( 2008 ) analyzed synergistic trends in Amazonian economies, forests, and climate. Verburg et al. ( 2014 ), in turn, tried to reconcile conservation policies with commodity prices. Nobre et al. ( 2016 ) proposed a new sustainable development paradigm for land use and climate change.

In addition to the Amazon Rainforest, studies—fewer in number—have addressed other Brazilian biomes of global importance, such as the Cerrado and the Atlantic Forest, or else have studied the country in general. Studies on the Cerrado Biome have focused on land use and water reuse (Spera et al., 2016 ), the expansion of soybean production and its impacts (Rausch et al., 2019 ), as well as trying to optimize the agricultural profit with freshwater quality and biodiversity (Kennedy et al., 2016 ). In turn, Umetsu and Pardini ( 2007 ) studied changes in small mammal habitats due to human interference in the Atlantic Forest. Regarding studies with a more general focus, Barretto et al. ( 2013 ) analyzed agricultural intensification and land use patterns in Brazil. Picoli et al. ( 2018 ) mapped the expansion of crops and land changes due to pasture intensification in the country’s agricultural frontiers, and Da Silva Junior et al. ( 2020 ) checked persistent fires in Brazilian biomes and to what extent this would or not follow the 2015 Paris Agreement.

However, the literature on agricultural frontiers and the environment does not only “live” in Brazil. The data revealed that Indonesia and Malaysia are also prominent in this topic, as they face major agricultural expansion due to palm oil production (Koh & Wilcove, 2008 ; McCarthy & Cramb, 2009 ; Koh et al., 2011 ; Wicke et al., 2011 ; Carlson et al., 2012 , 2013 and 2018 ; Miettinen et al., 2012 ; Busch et al., 2015 ). In addition, articles that address Emerging Countries together feature discussions on the environmental impact of commodity production and exports (Henders et al., 2015 ) and smallholder farmers’ decisions on deforestation in forest areas (Babigumira et al., 2014 ). In turn, studies that focus on the South American region address the intensification of agricultural products in the Chaco, formed by the territories of Paraguay, Bolivia, Argentina, and Brazil (Baumann et al., 2017 ; Fehlenberg et al., 2017 ; Le Polain de Waroux et al., 2018 ), and in research on the Río de la Plata, which covers an area that passes through Brazil, Argentina, Bolivia, Paraguay and Uruguay (Baeza & Paruelo, 2020 ).

With China as a study area, some authors have been concerned with analyzing land use variations in the country (Chen et al., 2014 ; Lin & Ho, 2003 ), as well as measuring the efficiency of agricultural production (Deng & Gibson, 2019 ) and water use (Wang et al., 2019 ). In turn, articles about Mexico were more “duelistic” and addressed trade-offs between ecological reserves versus archeological-ecotourist zones (Turner Ii et al., 2001 ), economic benefit for irrigation versus negative effects on groundwater (Raquel et al., 2007 ), and between community-based forest management versus protected areas (Ellis & Porter-Bolland, 2008 ).

Articles from Argentina have predominantly studied the Argentinean Chaco, focusing on the expansion of agriculture and its impacts on deforestation (Gasparri & Grau, 2009 ) and animal biodiversity (Mastrangelo & Gavin, 2012 ), as well as the controlling factors of this expansion (Volante et al., 2016 ). On the other hand, studies on India have addressed the environmental consequences of agriculture during the Green Revolution (Singh, 2000 ), the environmental impacts produced by human interference in watersheds (Rao & Pant, 2001 ), the presence of big cats in agricultural areas (Athreya et al., 2013 ).

To date, studies have shown that agricultural frontiers, through intensive production, impact the environment in emerging countries. However, the extent of these impacts may vary according to the natural resources affected, as well as country-specific factors, namely: public policies, regulations, and incentives, among others. To provide more recent results and serve as a basis for the discussion in the next section of this study, Table 2 presents the literature on agricultural frontier and environment in emerging countries published in the last ten years (2013 to 2022).

4.2 The connections between agricultural frontiers and the environment

The literature has focused on analyzing the impact of intensive agriculture on the still available natural resources, as well as the processes that can help preserve and regenerate the environment. Therefore, the interaction between agricultural frontiers and the environment in Emerging Countries will be analyzed from here on out in two ways: The Extended Industrial Agriculture Focus, in which the literature focuses on the measurement, analysis, and interpretation of the impacts that natural resources (water, soil, air, fauna, flora, etc.) suffer due to the advance of agricultural production; and the Socio-Economic-Ecological Focus, when the studies are concerned about verifying/measuring how local socio-economic factors and/or public policies affect the behavior of the population in this relationship between the environment and the agricultural frontiers.

4.2.1 Extended industrial agriculture focus

Many studies showed that there are strong environmental impacts due to the intensification of agricultural frontiers in Emerging Countries, in which the Extended Industrial Agriculture Focus was present in the most varied natural resources: land, fauna, flora, air, and water. To reach this conclusion, the authors used research methods such as Argumentative/Narrative text, Satellite-based maps, and Linear, Probabilistic, and Simulation Models.

The subjectivity and breadth of the term “environment,” already discussed, causes some researchers to address more than one natural resource in the same study. In this context, the data revealed that the articles that analyze two natural resources mainly address the environmental impact promoted by agricultural production on land use or flora (deforestation) in addition to a second resource. Studying the impact of agricultural frontiers on Flora and Biodiversity in the Brazilian Amazon, for instance, Ochoa-Quintero et al. ( 2015 ) concluded that due to deforestation, environments with 30 to 40% forest cover harbored lower numbers of mammals and birds. Furthermore, predictions for 2030 indicated that under the same devastation scenario, only 22% of Amazonian landscapes would be able to harbor at least 75% of these species.

In a study of the impact of agricultural production on land use and greenhouse gas emissions in Emerging Countries, Henders et al. ( 2015 ) found that alterations in land use and carbon fluxes from 2000 to 2011 were mainly due to exports of beef, palm oil, and soybean. Furthermore, in an analysis of land and water use in agricultural production in the Brazilian Cerrado, Spera et al. ( 2016 ) found that the increase in the agricultural area from 2003 to 2013 caused a decrease in the amount of water recycled into the atmosphere.

In turn, articles that analyze two or more natural resources (defined here as the “general approach”) provide a broader picture of the impacts caused in a given region. In a study on India, for example, Rao and Pant ( 2001 ) concluded that agricultural and extractive activities, together with population growth, caused the decline of vegetation cover in the central Himalayan region between 1963 and 1996, which subsequently stimulated soil and water loss in the Sadiyagad watershed region.

In a study involving Malaysia and Indonesia, Koh et al. ( 2011 ) state that 6% of all tropical peatlands in the region were used for palm oil production, which consequently caused the emission of over 4.5 million Mg of carbon per year, and the loss of 140 million g of biomass carbon, in addition to destroying the biodiversity of the region. In the same vein, Carlson et al. ( 2012 ) state that the impact of intensive palm oil production caused a 4% reduction in forest cover from 1989 to 2008 in Indonesia alone, due to the deforestation of 40% of peat lands in the country, in addition to possibly leveraging deforestation, according to projections, on regional lands and community lands.

Articles that point out the impacts of agricultural frontiers on a variety of natural resources have also been seen in studies for Eurasia, Africa, and Brazil. Researching the regions from Western Ukraine to Eastern China and from Southern Russia to Turkmenistan, Horion et al. ( 2016 ) found that rainfall use efficiency decreased due to the fall of the Soviet Union in 1991 and the abandonment of agricultural land thereafter, but natural resources in the region were also impacted by anthropogenic effects such as grazing intensity, increased salinization, and changes in irrigation practices.

In a study for South Africa, Jewitt et al. ( 2015 ) found that the coastal province of KwaZulu-Natal had over 7% of its natural habitat devastated between 2005 and 2011 due to the intensification of agricultural production and the construction of mines and dams, which resulted in land use transformation and generated losses of endemic biodiversity. For Brazil, analyzing the expansion of soybean production in the Cerrado Biome, Rausch et al. ( 2019 ) stated that this was responsible for converting 22% of the biome from 2003 to 2014, with most of the deforestation occurring within legal limits. For them, one way to try to decrease degradation would be to encourage policies coming from the private sector that restrict deforestation carried out by soybean producers.

After presenting the studies that analyze the impact of agricultural frontiers on more than one natural resource with an Extended Industrial Agricultural Focus, this Systematic Literature Review brings the articles that focus on only one of the resources.

4.2.1.1 Land use

Land use was the most studied topic among those covering the Extended Industrial Agricultural Focus in Emerging Countries. To quantify the impact between the agricultural frontier and land use in Brazil, Barretto et al. ( 2013 ), for instance, used the OLS model and found that the intensification of land use promoted a decrease in pastures and crops in the consolidated regions of agriculture, but this same intensification caused an increase in agricultural land in the agricultural frontiers, i.e., in areas where land management practices differ from those already established. In addition, in an analysis of the Río de la Plata Footnote 4 region, Baeza and Paruelo ( 2020 ) found that the increase in the agricultural area, mainly on the banks of the Uruguay River and in the western part of the Pampa Interior, also caused a decrease in pastures in the region. Still in this context, Graesser et al. ( 2015 ) emphasize, in their study for Latin America, the importance of distinguishing between pastures and crops when analyzing land use efficiency in agricultural production, since they are two distinct agricultural systems and bring different consequences to the soil.

With a more theoretical bias, Wicke et al. ( 2011 ) used data from Indonesia and Thailand from 1975 to 2005 and concluded that, despite the precariousness of the data, the studies showed that the impact of palm oil production on land use change was intense, which generated the loss of forest cover of 40 million hectares (Mha) of land in Indonesia and almost 5 Mha in Malaysia. In turn, in studies on China, Lin and Ho ( 2003 ), through data from the 1996 land survey, stated that there was a large loss of agricultural land in the country mainly due to the rapid process of urbanization, rural industrialization, and restructuring of the agricultural process; furthermore, combining Zelinsky’s hypothesis of the mobility transition model and the theory of land use transition, Chen et al. ( 2014 ) stated that it is necessary to consider the process of rural out-migration when studying land use change in China.

Some authors have pointed out ways to try to minimize the environmental impact of agricultural production on land use. Picoli et al. ( 2018 ) concluded, in an analysis of the state of Mato Grosso—Brazil, that the increase in double cropping systems saved the amount of land used for agricultural production. Still focusing on Brazil, Kennedy et al. ( 2016 ) stated, through a probabilistic model, that optimal outcomes between land use for agriculture and the environment occur when land use meets environmental preservation and biodiversity regulations. In the same line of reasoning, in a study on Russia and Ukraine, Smith et al. ( 2007 ) stated that soil organic carbon loss due to climate change in these two countries is imminent, but this loss will be lower when environmental considerations are met and outweigh the others.

4.2.1.2 Flora

The data showed that the flora of emerging countries is currently being devastated due to agricultural production, in which a large part of this deforestation is attributed to soy and livestock production. Fehlenberg et al. ( 2017 ) concluded that deforestation was directly driven by soybean cultivation in the Argentine Chaco, while cattle ranching increased deforestation rates in Argentina, Bolivia, and Paraguay. In a study of the Brazilian Amazon, Mertens et al. ( 2002 ) saw that cattle production evolved and promoted greater trade flows, thus increasing deforestation and fire outbreaks in the region. Furthermore, Müller et al. ( 2012 ) stated that in Bolivia, deforestation is largely driven by intensive agriculture, followed by cattle ranching and smaller-scale agriculture, in which fertile soil, favorable climate, and local and export markets are factors stimulating these impacts.

In a study for the Argentinean Chaco, Gasparri et al. ( 2009 ) stated that 1.4 million hectares of dry forest were cleared between 1972 and 2007 due to the global demand for soy during the 1980s and 1990s. On the other hand, in a study of the Brazilian Amazon, Macedo et al. ( 2012 ) believed that the relationship between deforestation and soybean production could be inversely proportional in topic forests once land and efficient land use policies are in place.

In addition to the intensive production of soy and cattle ranching, other factors are driving deforestation on the agricultural frontiers of emerging countries. According to Nepstad et al. ( 2001 ), investment in paving and building roads tends to increase deforestation rates in the Amazon Rainforest. Moving to microeconomic analysis, Pacheco ( 2006 ) concluded that deforestation in Bolivia intensified when the economic model of import substitution industrialization was changed to a more liberal model.

4.2.1.3 Fauna

The loss or extinction of animal biodiversity due to advancing agricultural frontiers in emerging countries has also been addressed in the literature. Koh and Wilcove ( 2008 ) noted, for example, that oil palm cultivation through the conversion of primary and secondary forests caused a decrease in the number of birds and butterflies in Malaysia and Indonesia. Accordingly, in a study on the Argentinian Chaco, Mastrangelo and Gavin ( 2012 ) stated that in areas of cattle production, there are far fewer bird species compared to areas of intact forest.

Some studies have concluded that there have also been migratory processes of animals due to the advance of intensive agricultural production in emerging countries. In a survey of the Atlantic Forest of Brazil, Umetsu and Pardini ( 2007 ) found that the destruction of native vegetation increased the number of invasive species in the Morro Grande Reserve. In a study conducted in India, Athreya et al. ( 2013 ) stated that intensive agriculture has led to large wild carnivores being seen in areas previously inhabited only by humans.

4.2.1.4 Air

To verify the impacts that agricultural frontiers could cause in the atmosphere, the authors quantified the emissions of polluting gases arising from intensive production and simulated scenarios of the behavior of this pollution over time. In a study for the South American Chaco, Baumann et al. ( 2017 ) used satellite data and found that pasture and crop intensification was responsible for decimating 20% of the Chaco Forest between 1985 and 2013, which subsequently caused substantial emissions of 824 Tg of carbon.

Studying the region of Malaysia and Indonesia, Miettinen et al. ( 2012 ) found that the devastation of peatlands from oil palm cultivation emitted 230,310 Mt CO2e into the atmosphere. Furthermore, the authors projected that following these numbers, there would be a conversion of 69 Mha of peatlands by 2020, causing the annual carbon increase in the two countries to be between 380 and 920 Mt CO2e. Along the same lines, Carlson et al. ( 2013 ) stated that between 2000 and 2010 oil palm production grew more than 270% in the Kalimantan region of Indonesia, with projections for 2020 indicating that this region alone would contribute to about 20% of the CO2 emissions of the entire country.

4.2.1.5 Water

The analysis showed that only one study discusses the environmental impact of agricultural frontiers on water. Raquel et al. ( 2007 ) used Game Theory to verify the optimal decision between increasing agricultural production using irrigation or decreasing environmental effects on groundwater in the Alto Rio Lerma Irrigation District—Mexico. The authors stated that irrigation used for agricultural production can considerably decrease groundwater in the region, in which the optimal consumption decision depends on the relative importance given to irrigation and overall water use. Considering only the environmental impacts, the Pareto optimum would be to extract about 370 million cubic meters of water per year.

4.2.2 Socio-economic-ecological focus

To analyze the articles with a Socio-Economic-Ecological Focus, the authors also presented data on the environmental impacts caused by agricultural frontiers. However, they essentially focused on listing possible solutions for preservation and/or regeneration, as well as analyzing how the socio-economic aspects of the population can stimulate or reduce environmental degradation. First, the literature on possible solutions to environmental problems is discussed and then the influence of socio-economic aspects is presented.

In a study on Mexico, Ellis and Porter-Bolland ( 2008 ) showed the importance of protection areas in the forest preservation process, since they visualized that deforestation was higher in regions with community-based forest management when compared to protected areas. In turn, taking the Argentine Chaco as a study area, Volante et al. ( 2016 ) showed that some forest laws, such as the “Native Forest Law,” are still insufficient to restrict deforestation and transformation of the region, so they believe that changes in implementation and enforcement strategies in the law itself or the insertion of alternative incentives, such as the European Union’s biofuel import standards, may be solutions to attempt to reverse this situation.

Some studies point to the understanding of heterogeneity between regions as a key factor for the adoption of environmental protection measures in emerging countries. In a study for Latin America, Pacheco et al. ( 2010 ) state that although Reducing Emissions from Deforestation and Forest Degradation (REDD) is an important mechanism to preserve and conserve tropical forests, the implementation of public policies is hampered by socio-economic and land use heterogeneity in the region. In a discussion on South American biomes, Nolte et al. ( 2017 ) state that the Cerrado, Chaco, and Chiquitano regions have lower carbon stocks and biodiversity, but have greater agricultural importance, a higher number of private properties and greater compliance with forestry regulations on private lands when compared to the Amazon; subsequently, policies aimed to combat deforestation in South America must consider the specialties and subjectivities of each agricultural frontier. According to Gasparri and De Waroux ( 2015 ), despite the diversity among countries, there is a coupling of soybean and cattle production frontiers in South America, which are the main drivers of deforestation in the region, so it is necessary to adopt models that analyze the coupling between geographic locations and productive sectors.

The literature on Brazil, unsurprisingly, has focused on the Amazon Rainforest. Some of the solutions for forest preservation and/or regeneration analyzed and suggested by the authors were: The creation of ecological parks and preservation of indigenous reserves (Nepstad et al., 2006 ), and the continuation and enforcement of the Forest Code to stem deforestation (Verburg et al., 2014 ); regulated and controlled use of fire by landowners, increased environmental performance in commodity markets, and incentives in the carbon market (Nepstad et al., 2008 ); and the encouragement of biological, digital, and materials technologies to promote sustainable land use development and climate change (Nobre et al., 2016 ).

The studies for Southeast Asia (Indonesia and Malaysia) presented the main “drivers” for palm oil cultivation in the region and tried to find solutions to mitigate the environmental problems arising from this production. For McCarthy and Cramb ( 2009 ), the shift from social government to neoliberalism facilitated the devastation of forests by agricultural frontiers in Malaysia and Indonesia, in which subsistence farming by indigenous peoples and smallholders gave way to mechanized agriculture. According to Carlson et al. (2015), agricultural concessions are the main stimulus for deforestation in Indonesia, so reducing these licenses, as well as encouraging carbon emission reduction policies, are important steps to try to reverse the process of environmental degradation in the country. In addition, Carlson et al. ( 2018 ) state that, although it does not solve all environmental problems, RSPO certification is an important mechanism to aid conservation, as it significantly reduced deforestation in Indonesia from 2001 to 2015.

The discussion on palm oil has extended to Southeast Asia, where, in a study of India and China, Wilcove and Koh ( 2010 ) concluded that there are some ways to try to minimize the problems arising from palm oil production, but it should be noted that “boycott” policies will not work for this region, so promoting competitiveness through incentives for REDD, for example, will be more effective. However, the environmental impacts of agricultural frontiers in Asia have not been restricted to palm oil alone. In a study in India, Singh ( 2000 ) found that intensive agricultural production during the Green Revolution led to soil degradation and alteration, and water pollution, so to help restore degraded areas, it is necessary to increase and diversify biomass productivity, as well as focus on the effectiveness of moisture conservation and water harvesting policies, nutrient management and land use planning, and recharge of groundwater reservoirs.

Moving to the literature that discusses the influence of socio-economic factors, some articles have shown that environmental degradation rates in emerging countries are reduced when there are aspects of socio-economic development among the population, whether intellectual or financial. Studies for the Brazilian Amazon have exemplified this relationship very well. Rodrigues et al. ( 2009 ) concluded that literacy, expectation, and relative standards of living corroborate with the hypothesis of the Environmental Kuznets Curve, i.e., these aspects are inversely proportional to environmental degradation at higher stages of development. Also, Schiesari et al. ( 2013 ) asserted that small-scale farmers with higher levels of education and technical support tend to use fewer pesticides and/or resources that degrade the environment less.

Along these lines, in a study of the Shandong region in China, Deng et al. ( 2019 ) concluded that land productivity is concentrated in cities far from the economic or provincial center, but eco-efficiency is higher in areas of developed cities, eco-tourism, or belonging to the coastal and mountainous economic zone. On the other hand, although fewer in number, the literature has shown that the environmental impacts of agricultural production are also caused by the “most developed part” of the population. An example of this was demonstrated in the study by Babigumira et al. ( 2014 ) who, using 24 emerging countries as a study area, concluded that farmers with high and middle incomes tend to deforest more than smallholder farmers who are considered poor and lack market knowledge.

Going beyond this direct discussion between environmental degradation and the economic and social levels of the population, some studies point to other ways in which socio-economic factors interfere with the relationship between agricultural frontiers and the environment. In a study on Russia and Ukraine, Meyfroidt et al. ( 2016 ) argue that one of the main problems Europe faces today is the abandonment of agricultural land so factors such as young labor and an increasing rural population become essential to leverage the re-cultivation of this land and consequently stimulate the creation of new agricultural frontiers in the region. In a study on the South American Chaco, Le Polain de Waroux et al. ( 2018 ) found that while the expansion of agricultural frontiers has responded well to the use of new technologies and infrastructure and rising prices, the dynamics of these frontiers are shaped by the existence of abnormal economic rents and the presence of a limited number of actors (commodity producers, speculators, rentiers, etc.) able to capture and influence this whole process.

5 Research agenda

Considered the “home country” of the discussion between intensive agricultural production and the environment, the literature on Brazil has mainly focused on studies of the Amazon rainforest. Regarding these studies, it was noted that some of the possible “solutions” to reverse the process of environmental degradation have been implemented in the region for some time, such as the expansion of protected areas, national and foreign financial incentives, national public policies, etc. However, the literature still lacks studies that seek to measure the effectiveness (joint or not) of such measures or even to see if these measures have the same effect within the limits of the Amazon Rainforest, since the region covers nine states in Brazil alone, besides belonging to six different countries. Furthermore, considering the research criteria, other Brazilian biomes of worldwide importance were almost “forgotten” among the authors, such as the Cerrado, Atlantic Forest, Caatinga, Pampas, and Pantanal. The Cerrado, for example, comprises the most recent agricultural frontier in Brazil (Matopiba) and I believe it deserves special attention, especially due to the current rates of deforestation in the region, as it contains unique biodiversity and large natural aquifers.

Research on Indonesia and Malaysia has shown that the intensive production of palm oil has brought countless environmental impacts to the region, and there are already studies in the literature that measure the effectiveness of some certificates that help preserve the environment, such as the RSPO. However, even with the destruction of tropical forests and the adoption of programs and incentives with an environmental focus, the production of palm oil continues “full steam ahead,” as well as the degradation rates. With this, what would the next steps be? According to Pirker et al. ( 2016 ), there are only 17% of the area in the world left suitable for the expansion of palm oil, which, besides being scarce, are areas with little accessibility. Thus, I suggest, in addition to research that seeks to verify spatially where these scarce areas are and if there is a change in the variability of this percentage, studies that analyze more concisely if the process of deforestation and changes in soil, for example, accompany the expansion of palm oil. Are there indications that the soil in Indonesia and Malaysia may be going through a process of desertification? If so, how can this process be reversed?

It was also seen that in populous countries like India and China, the form of analyzing the impacts of agricultural frontiers on the environment was more connected to a possible scarcity of land and food. While areas are needed for cultivation and food production, space is also needed to meet the growing population levels. Therefore, I suggest research to discuss how to meet the current demand for food with minimal impact on the environment. Are the management of soil and already degraded areas important in this process? Besides, because it covers large territories, the question remains: is there migration of people due to intensive production and/or degraded areas in these countries? Studies that visualize how soil changes and deforestation promoted by the expansion of agricultural frontiers stimulate migration in these countries will be highlighted.

Looking more generally, it was noted that emerging countries are heterogeneous, but they carry within themselves a strong political-structural dependence, that is, they are mostly guided by internal and external stimuli. When it comes to issues such as agricultural production and the environment, the scenario is no different, in which many measures for and against the devastation promoted by the agricultural frontiers are motivated by the markets and especially by the governments in force. One suggestion is to analyze how emerging countries deal with the relationship between agricultural frontiers and the environment considering the form of government of these countries. Is there a consensus among them? Are these measures spatially dependent or just tied to the local government? Furthermore, although there have been studies at the national level, the interference of foreign markets in the agricultural production and preservation policies of a group of emerging countries, for instance, has not yet been visualized. Is there a strong interference between the markets of the emerging countries themselves or is there a greater influence of external markets? Are these influences more toward an Extended Industrial Agriculture bias or a Socio-Economic-Ecological Focus?

In addition, while it is necessary to stimulate agricultural production in emerging countries to promote economic growth, it is also necessary to know, visualize, and measure how the expansion of agricultural frontiers affects the environment and try to minimize these impacts as much as possible. This raises the question: can there be sustainable development in countries that have agricultural production as the main means of economic growth, as is the case in most emerging countries? To what extent will the environment support the expansion of agricultural frontiers? These are dilemmas to solve, almost a virtuous cycle. It is already clear that natural resources are increasingly scarce, so I would put the adoption—increasingly—of multi-component forecast models, such as System Dynamics Modeling. And, from there on, to adopt the necessary measures to balance intensive agricultural production and environmental impacts in a more incisive way.

6 Conclusions

This study analyzed, through a Systematic Literature Review, the main research on the interference of agricultural frontiers on the environment in emerging countries, discussing the theories that have accompanied the environmental and agricultural transformations that have taken place in these countries over the last thirty years (1993–2022).

One of the most significant findings of this literature review is that there is no specific framework to synthesize studies on agricultural frontiers and the environment in developing countries, i.e., to study the environmental impacts arising from intensive agricultural production in regions as heterogeneous as emerging countries, authors have adopted different approaches, theories, and methodologies. However, it is noticeable that much of the literature has been mainly concerned with analyzing and measuring the impact of agriculture on natural resources, which I have called the Expanded Industrial Agriculture Approach, as well as verifying and measuring how local socio-economic factors and/or public policies affect the behavior of the population in this relationship between the environment and agricultural frontiers, which I have defined as the Socio-economic-Ecological Approach. This is the most appropriate way (at the moment) to summarize how the literature on agricultural frontiers and the environment in emerging countries is being conducted.

In the analysis with the Expanded Industrial Agriculture Focus, it was found that intensive agriculture degrades the most varied natural resources, but the articles brought discussions essentially on the environmental impact on flora (through the analysis of deforestation rates), air (through the measurement of polluting gases) and changes in land use, thus generating the need for further research into the impacts of agricultural production on water, for example. In turn, the articles with a Socio-Economic-Ecological Focus brought possible solutions to environmental problems in emerging countries, such as the creation of ecological parks and increase in forest protection areas, public policies that take into account the specificity of each region, implementation and enforcement of stricter environmental laws, encouragement of biological, digital and material technologies, increase in environmental performance in commodity markets and incentives in the carbon market. In addition, research has shown that the existence of some socio-economic aspects among the population, such as literacy, expectation, and high standard of living, tends to decrease the rates of environmental degradation in these countries.

The data also revealed that there is no equal interest in research on agricultural frontiers and the environment among emerging countries, with most studies discussing the impacts of intensive agricultural production in Brazil, followed somewhat distantly by studies on South America in general and the island region of Indonesia and Malaysia. Unsurprisingly, research on Brazil has been almost entirely on the world’s largest tropical rainforest, the Amazon Rainforest, thus leaving a large gap in the literature on other Brazilian biomes of global importance, such as the Cerrado, the Atlantic Forest, the Caatinga, the Pampas and the Pantanal. Moreover, the literature still lacks research on European economies in transition, emerging African countries, and Russia, or on the agricultural environmental impact of the huge demand for food in populous countries such as India and China.

Therefore, in addition to country-specific suggestions, future research on agricultural frontiers and the environment should not only propose solutions but also measure the effectiveness of proposals aimed at reducing/reversing the degradation process in emerging countries in general, considering the interference of markets, types of government and the high heterogeneity among them. Moreover, as this is a delicate debate with multiple components involved, research should adopt more dynamic forecasting models to seek a balance between intensive agricultural production and environmental impacts. There is also a need for studies that bring more recent discussions, such as agricultural digitalization, migration of agro-industrial poles, nanotechnology, and circular economy, among others.

Data availability

The final set of data supporting the conclusions of this study is available from the corresponding author upon reasonable request.

Procedures 1 to 3 are part of the planning stage and, from procedure 4 onwards, the procedures are part of the PRISMA protocol.

Asterisks next to keywords indicate that the exact spelling of the word was included in the search, e.g., agriculture, agricultural; environment, environmental; BRIC, BRICS.

The citation number was not an exclusion criterion for inclusion when additional articles (other sources) were selected in Step 2.

The Rio de la Plata forms a natural border between the South American countries of Uruguay and Argentina.

Adegbeye, M. J., Reddy, P. R. K., Obaisi, A. I., Elghandour, M. M. M. Y., Oyebamiji, K. J., Salem, A. Z. M., & Camacho-Díaz, L. M. (2020). Sustainable agriculture options for production, greenhouse gasses and pollution alleviation, and nutrient recycling in emerging and transitional nations—An overview. Journal of Cleaner Production, 242 , 118319. https://doi.org/10.1016/j.jclepro.2019.118319

Article   CAS   Google Scholar  

Athreya, V., Odden, M., Linnell, J. D., Krishnaswamy, J., & Karanth, U. (2013). Big cats in our backyards: Persistence of large carnivores in a human dominated landscape in India. PLoS ONE, 8 (3), e57872. https://doi.org/10.1371/journal.pone.0057872

Avagyan, A. B. (2018). Algae to energy and sustainable development. Technologies, resources, economics and system analyses. New design of global environmental policy and live conserve industry. Amazon , ISBN-13, 978–1718722552.

Avagyan, A. B. (2010). New design & build biological system addressed to global environment management and sustainable development through including microalgae and their biomass in production and bio cycles. Journal of Environmental Protection, 1 , 183–200. https://doi.org/10.4236/jep.2010.12023

Avagyan, A. B. (2017). Environmental building policy by the use of microalgae and decreasing of risks for Canadian oil sand sector development. Environmental Science and Pollution Research, 24 (25), 20241–20253. https://doi.org/10.1007/s11356-017-9864-x

Avagyan, A. B. (2021). Theory of bioenergy accumulation and transformation: application to evolution, energy, sustainable development, climate change, manufacturing, agriculture, military activity and pandemic challenges. Athens J Sci, 8 (1), 57–80. https://doi.org/10.30958/ajs.8-1-4

Article   Google Scholar  

Babigumira, R., Angelsen, A., Buis, M., Bauch, S., Sunderland, T., & Wunder, S. (2014). Forest clearing in rural livelihoods: Household-level global-comparative evidence. World Development, 64 , S67–S79. https://doi.org/10.1016/j.worlddev.2014.03.002

Baeza, S., & Paruelo, J. M. (2020). Land use/land cover change (2000–2014) in the Rio de la Plata grasslands: An analysis based on MODIS NDVI time series. Remote Sensing, 12 (3), 381. https://doi.org/10.3390/rs12030381

Barretto, A. G., Berndes, G., Sparovek, G., & Wirsenius, S. (2013). Agricultural intensification in Brazil and its effects on land-use patterns: An analysis of the 1975–2006 period. Global Change Biology, 19 (6), 1804–1815. https://doi.org/10.1111/gcb.12174

Baumann, M., Gasparri, I., Piquer-Rodríguez, M., Gavier Pizarro, G., Griffiths, P., Hostert, P., & Kuemmerle, T. (2017). Carbon emissions from agricultural expansion and intensification in the Chaco. Global Change Biology, 23 (5), 1902–1916. https://doi.org/10.1111/gcb.13521

Busch, J., Ferretti-Gallon, K., Engelmann, J., Wright, M., Austin, K. G., Stolle, F., & Baccini, A. (2015). Reductions in emissions from deforestation from Indonesia’s moratorium on new oil palm, timber, and logging concessions. Proceedings of the National Academy of Sciences, 112 (5), 1328–1333. https://doi.org/10.1073/pnas.141251411

Carlson, K. M., Curran, L. M., Asner, G. P., Pittman, A. M., Trigg, S. N., & Marion Adeney, J. (2013). Carbon emissions from forest conversion by Kalimantan oil palm plantations. Nature Climate Change, 3 (3), 283–287. https://doi.org/10.1038/nclimate1702

Carlson, K. M., Curran, L. M., Ratnasari, D., Pittman, A. M., Soares-Filho, B. S., Asner, G. P., & Rodrigues, H. O. (2012). Committed carbon emissions, deforestation, and community land conversion from oil palm plantation expansion in West Kalimantan, Indonesia. Proceedings of the National Academy of Sciences, 109 (19), 7559–7564. https://doi.org/10.1073/pnas.120045210

Carlson, K. M., Heilmayr, R., Gibbs, H. K., Noojipady, P., Burns, D. N., Morton, D. C., & Kremen, C. (2018). Effect of oil palm sustainability certification on deforestation and fire in Indonesia. Proceedings of the National Academy of Sciences, 115 (1), 121–126. https://doi.org/10.1073/pnas.170472811

Chen, R., Ye, C., Cai, Y., Xing, X., & Chen, Q. (2014). The impact of rural out-migration on land use transition in China: Past, present and trend. Land Use Policy, 40 , 101–110. https://doi.org/10.1016/j.landusepol.2013.10.003

da Silva Junior, C. A., Teodoro, P. E., Delgado, R. C., Teodoro, L. P. R., Lima, M., de Andréa Pantaleão, A., & Facco, C. U. (2020). Persistent fire foci in all biomes undermine the Paris Agreement in Brazil. Scientific Reports, 10 (1), 16246. https://doi.org/10.1038/s41598-020-72571-w

Deng, X., & Gibson, J. (2019). Improving eco-efficiency for the sustainable agricultural production: A case study in Shandong, China. Technological Forecasting and Social Change, 144 , 394–400. https://doi.org/10.1016/j.techfore.2018.01.027

Denyer, D., & Tranfield, D. (2009). Producing a systematic review. D.A. Buchanan, A. Bryman (Eds.), The Sage handbook of organizational research methods , Sage Publications, Thousand Oaks, CA (2009), pp. 671–689.

Ellis, E. A., & Porter-Bolland, L. (2008). Is community-based forest management more effective than protected areas?: A comparison of land use/land cover change in two neighboring study areas of the Central Yucatan Peninsula. Mexico. Forest Ecology and Management, 256 (11), 1971–1983. https://doi.org/10.1016/j.foreco.2008.07.036

El-Shinnawy, N. A., Heikal, S., & Fahmy, Y. (1983). Saccharification of cotton bolls by Concentrated sulphuric acid. Research and Industry, 28 (2), 123–126.

CAS   Google Scholar  

Etemadi, H., Afsharkia, S., Zinatloo-Ajabshir, S., & Shokri, E. (2021). Effect of alumina nanoparticles on the antifouling properties of polycarbonate-polyurethane blend ultrafiltration membrane for water treatment. Polymer Engineering & Science, 61 (9), 2364–2375. https://doi.org/10.1002/pen.25764

Fahmy, T. Y., Fahmy, Y., Mobarak, F., El-Sakhawy, M., & Abou-Zeid, R. E. (2020). Biomass pyrolysis: Past, present, and future. Environment, Development and Sustainability, 22 , 17–32. https://doi.org/10.1007/s10668-018-0200-5

Fahmy, T. Y., Fahmy, Y., Mobarak, F., El-Sakhawy, M., & Fadl, M. (2017). Agricultural residues (wastes) For manufacture of paper, board, and miscellaneous products: Background overview and future prospects. International Journal of ChemTech Research, 10 (2), 424–448. https://doi.org/10.5281/zenodo.546735

Fahmy, T. Y., & Mobarak, F. (2013). Advanced binderless board-like green nanocomposites from undebarked cotton stalks and mechanism of self-bonding. Cellulose, 20 , 1453–1457. https://doi.org/10.1007/s10570-013-9911-9

Fahmy, Y. (1982). Pyrolysis of agricultural residues. I. Prospects of lignocellulose pyrolysis for producing chemicals and energy sources. Cellulose Chemistry and Technology, 16 , 347–355.

Fahmy, Y., Mobarak, F., & Schweers, W. (1982). Pyrolysis of agricultural residues. II. Yield and chemical composition of tars and oils produced from cotton stalks, and Assessment of lignin structure. Cellulose Chemistry and Technology, 16 , 453–459.

FAO, FAOSTAT statistical database (2021). Available in https://www.fao.org/faostat/en/#data . Accessed in 12 of January 2022.

Fehlenberg, V., Baumann, M., Gasparri, N. I., Piquer-Rodriguez, M., Gavier-Pizarro, G., & Kuemmerle, T. (2017). The role of soybean production as an underlying driver of deforestation in the South American Chaco. Global Environmental Change, 45 , 24–34. https://doi.org/10.1016/j.gloenvcha.2017.05.001

Feintrenie, L. (2014). Agro-industrial plantations in Central Africa, risks and opportunities. Biodiversity and Conservation, 23 (6), 1577–1589. https://doi.org/10.1007/s10531-014-0687-5

Gasparri, N. I., & de Waroux, Y. L. P. (2015). The coupling of South American soybean and cattle production frontiers: New challenges for conservation policy and land change science. Conservation Letters, 8 (4), 290–298. https://doi.org/10.1111/conl.12121

Gasparri, N. I., & Grau, H. R. (2009). Deforestation and fragmentation of Chaco dry forest in NW Argentina (1972–2007). Forest Ecology and Management, 258 (6), 913–921. https://doi.org/10.1016/j.foreco.2009.02.024

Gibbs, H. K., Ruesch, A. S., Achard, F., Clayton, M. K., Holmgren, P., Ramankutty, N., & Foley, J. A. (2010). Tropical forests were the primary sources of new agricultural land in the 1980s and 1990s. Proceedings of the National Academy of Sciences, 107 (38), 16732–16737. https://doi.org/10.1073/pnas.0910275107

Graesser, J., Aide, T. M., Grau, H. R., & Ramankutty, N. (2015). Cropland/pastureland dynamics and the slowdown of deforestation in Latin America. Environmental Research Letters, 10 (3), 034017. https://doi.org/10.1088/1748-9326/10/3/034017

Heidari-Asil, S. A., Zinatloo-Ajabshir, S., Alshamsi, H. A., Al-Nayili, A., Yousif, Q. A., & Salavati-Niasari, M. (2022). Magnetically recyclable ZnCo 2 O 4 /Co 3 O 4 nano-photocatalyst: Green combustion preparation, characterization and its application for enhanced degradation of contaminated water under sunlight. International Journal of Hydrogen Energy, 47 (38), 16852–16861. https://doi.org/10.1016/j.ijhydene.2022.03.157

Henders, S., Persson, U. M., & Kastner, T. (2015). Trading forests: Land-use change and carbon emissions embodied in production and exports of forest-risk commodities. Environmental Research Letters, 10 (12), 125012. https://doi.org/10.1088/1748-9326/10/12/125012

Horion, S., Prishchepov, A. V., Verbesselt, J., de Beurs, K., Tagesson, T., & Fensholt, R. (2016). Revealing turning points in ecosystem functioning over the Northern Eurasian agricultural frontier. Global Change Biology, 22 (8), 2801–2817. https://doi.org/10.1111/gcb.13267

Ibrahim, R. L., Ajide, K. B., Usman, M., & Kousar, R. (2022). Heterogeneous effects of renewable energy and structural change on environmental pollution in Africa: Do natural resources and environmental technologies reduce pressure on the environment? Renewable Energy, 200 , 244–256. https://doi.org/10.1016/j.renene.2022.09.134

Jahanger, A., Usman, M., & Ahmad, P. (2022). Investigating the effects of natural resources and institutional quality on CO 2 emissions during globalization mode in developing countries. International Journal of Environmental Science and Technology, 32 , 1–20. https://doi.org/10.1007/s13762-022-04638-2

James, M. S., Marrissa, C., Mark, S., & Anthea, B. (2021). Systematic approaches to a successful literature review. Systematic Approaches to a Successful Literature Review, 42 , 1–100.

Google Scholar  

Jewitt, D., Goodman, P. S., Erasmus, B. F., O’Connor, T. G., & Witkowski, E. T. (2015). Systematic land-cover change in KwaZulu-Natal, South Africa: Implications for biodiversity. South African Journal of Science, 111 (9–10), 01–09. https://doi.org/10.17159/sajs.2015/20150019

Kennedy, C. M., Hawthorne, P. L., Miteva, D. A., Baumgarten, L., Sochi, K., Matsumoto, M., & Kiesecker, J. (2016). Optimizing land use decision-making to sustain Brazilian agricultural profits, biodiversity and ecosystem services. Biological Conservation, 204 , 221–230. https://doi.org/10.1016/j.biocon.2016.10.039

Koh, L. P., Miettinen, J., Liew, S. C., & Ghazoul, J. (2011). Remotely sensed evidence of tropical peatland conversion to oil palm. Proceedings of the National Academy of Sciences, 108 (12), 5127–5132. https://doi.org/10.1073/pnas.1018776108

Koh, L. P., & Wilcove, D. S. (2008). Is oil palm agriculture really destroying tropical biodiversity? Conservation Letters, 1 (2), 60–64. https://doi.org/10.1111/j.1755-263X.2008.00011.x

Lin, G. C., & Ho, S. P. (2003). China’s land resources and land-use change: Insights from the 1996 land survey. Land Use Policy, 20 (2), 87–107. https://doi.org/10.1016/S0264-8377(03)00007-3

Macedo, M. N., DeFries, R. S., Morton, D. C., Stickler, C. M., Galford, G. L., & Shimabukuro, Y. E. (2012). Decoupling of deforestation and soy production in the southern Amazon during the late 2000s. Proceedings of the National Academy of Sciences, 109 (4), 1341–1346. https://doi.org/10.1073/pnas.1111374109

Makhdum, M. S. A., Usman, M., Kousar, R., Cifuentes-Faura, J., Radulescu, M., & Balsalobre-Lorente, D. (2022). How do institutional quality, natural resources, renewable energy, and financial development reduce ecological footprint without hindering economic growth trajectory? Evidence from China. Sustainability, 14 (21), 13910. https://doi.org/10.3390/su142113910

Mastrangelo, M. E., & Gavin, M. C. (2012). Trade-offs between cattle production and bird conservation in an agricultural frontier of the Gran Chaco of Argentina. Conservation Biology, 26 (6), 1040–1051. https://doi.org/10.1111/j.1523-1739.2012.01904.x

McCarthy, J. F., & Cramb, R. A. (2009). Policy narratives, landholder engagement, and oil palm expansion on the Malaysian and Indonesian frontiers. Geographical Journal, 175 (2), 112–123. https://doi.org/10.1111/j.1475-4959.2009.00322.x

Mertens, B., Poccard-Chapuis, R., Piketty, M. G., Lacques, A. E., & Venturieri, A. (2002). Crossing spatial analyses and livestock economics to understand deforestation processes in the Brazilian Amazon: The case of Sao Felix do Xingu in South Para. Agricultural Economics, 27 (3), 269–294. https://doi.org/10.1111/j.1574-0862.2002.tb00121.x

Meyfroidt, P., Schierhorn, F., Prishchepov, A. V., Müller, D., & Kuemmerle, T. (2016). Drivers, constraints and trade-offs associated with recultivating abandoned cropland in Russia, Ukraine and Kazakhstan. Global Environmental Change, 37 , 1–15. https://doi.org/10.1016/j.gloenvcha.2016.01.003

Miettinen, J., Hooijer, A., Shi, C., Tollenaar, D., Vernimmen, R., Liew, S. C., & Page, S. E. (2012). Extent of industrial plantations on S outheast A sian peatlands in 2010 with analysis of historical expansion and future projections. GCB Bioenergy, 4 (6), 908–918. https://doi.org/10.1111/j.1757-1707.2012.01172.x

Mobarak, F. (1983). Rapid continuous pyrolysis of cotton stalks for charcoal production. Holzforschung . https://doi.org/10.1515/hfsg.1983.37.5.251

Mobarak, F., Fahmy, Y., & Augustin, H. (1982). Binderless lignocellulose composite from bagasse and mechanism of self-bonding. Holzforschung . https://doi.org/10.1515/hfsg.1982.36.3.131

Mobarak, F., Fahmy, Y., & Schweers, W. (1982b). Production of phenols and charcoal from bagasse by a rapid continuous pyrolysis process. Wood Science and Technology, 16 , 59–66. https://doi.org/10.1007/BF00351374

Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., & Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4 (1), 1–9. https://doi.org/10.1186/2046-4053-4-1

Müller, R., Müller, D., Schierhorn, F., Gerold, G., & Pacheco, P. (2012). Proximate causes of deforestation in the Bolivian lowlands: An analysis of spatial dynamics. Regional Environmental Change, 12 , 445–459. https://doi.org/10.1007/s10113-011-0259-0

Nepstad, D., Carvalho, G., Barros, A. C., Alencar, A., Capobianco, J. P., Bishop, J., & Prins, E. (2001). Road paving, fire regime feedbacks, and the future of Amazon forests. Forest ecology and management, 154 (3), 395–407. https://doi.org/10.1016/S0378-1127(01)00511-4

Nepstad, D., Schwartzman, S., Bamberger, B., Santilli, M., Ray, D., Schlesinger, P., & Rolla, A. (2006). Inhibition of Amazon deforestation and fire by parks and indigenous lands. Conservation Biology, 20 (1), 65–73. https://doi.org/10.1111/j.1523-1739.2006.00351.x

Nepstad, D. C., Stickler, C. M., Filho, B. S., & Merry, F. (2008). Interactions among Amazon land use, forests and climate: Prospects for a near-term forest tipping point. Philosophical Transactions of the Royal Society B: Biological Sciences, 363 (1498), 1737–1746. https://doi.org/10.1098/rstb.2007.0036

Nobre, C. A., Sampaio, G., Borma, L. S., Castilla-Rubio, J. C., Silva, J. S., & Cardoso, M. (2016). Land-use and climate change risks in the Amazon and the need of a novel sustainable development paradigm. Proceedings of the National Academy of Sciences, 113 (39), 10759–10768. https://doi.org/10.1073/pnas.1605516113

Nolte, C., de Waroux, Y. L. P., Munger, J., Reis, T. N., & Lambin, E. F. (2017). Conditions influencing the adoption of effective anti-deforestation policies in South America’s commodity frontiers. Global Environmental Change, 43 , 1–14. https://doi.org/10.1016/j.gloenvcha.2017.01.001

Ochoa-Quintero, J. M., Gardner, T. A., Rosa, I., de Barros Ferraz, S. F., & Sutherland, W. J. (2015). Thresholds of species loss in Amazonian deforestation frontier landscapes. Conservation Biology, 29 (2), 440–451. https://doi.org/10.1111/cobi.12446

Pacheco, P. (2006). Agricultural expansion and deforestation in lowland Bolivia: The import substitution versus the structural adjustment model. Land Use Policy, 23 (3), 205–225. https://doi.org/10.1016/j.landusepol.2004.09.004

Pacheco, P. (2009). Agrarian reform in the Brazilian Amazon: Its implications for land distribution and deforestation. World Development, 37 (8), 1337–1347. https://doi.org/10.1016/j.worlddev.2008.08.019

Pacheco, P., Aguilar-Støen, M., Börner, J., Etter, A., Putzel, L., & Vera Díaz, M. D. C. (2010). Landscape transformation in tropical Latin America: Assessing trends and policy implications for REDD+. Forests, 2 (1), 1–29. https://doi.org/10.3390/f2010001

Picoli, M. C. A., Camara, G., Sanches, I., Simões, R., Carvalho, A., Maciel, A., & Almeida, C. (2018). Big earth observation time series analysis for monitoring Brazilian agriculture. ISPRS journal of photogrammetry and remote sensing, 145 , 328–339. https://doi.org/10.1016/j.isprsjprs.2018.08.007

Pirker, J., Mosnier, A., Kraxner, F., Havlík, P., & Obersteiner, M. (2016). What are the limits to oil palm expansion? Global Environmental Change, 40 , 73–81. https://doi.org/10.1016/j.gloenvcha.2016.06.007

Polain, Le., de Waroux, Y., Baumann, M., Gasparri, N. I., Gavier-Pizarro, G., Godar, J., Kuemmerle, T., & Meyfroidt, P. (2018). Rents, actors, and the expansion of commodity frontiers in the Gran Chaco. Annals of the American Association of Geographers, 108 (1), 204–225. https://doi.org/10.1080/24694452.2017.1360761

Rao, K. S., & Pant, R. (2001). Land use dynamics and landscape change pattern in a typical micro watershed in the mid elevation zone of central Himalaya, India. Agriculture, Ecosystems & Environment, 86 (2), 113–124. https://doi.org/10.1016/S0167-8809(00)00274-7

Raquel, S., Ferenc, S., Emery, C., Jr., & Abraham, R. (2007). Application of game theory for a groundwater conflict in Mexico. Journal of Environmental Management, 84 (4), 560–571. https://doi.org/10.1016/j.jenvman.2006.07.011

Rausch, L. L., Gibbs, H. K., Schelly, I., Brandão, A., Jr., Morton, D. C., Filho, A. C., & Meyer, D. (2019). Soy expansion in Brazil’s Cerrado. Conservation Letters, 12 (6), e12671. https://doi.org/10.1111/conl.12671

Rodrigues, A. S., Ewers, R. M., Parry, L., Souza, C., Jr., Veríssimo, A., & Balmford, A. (2009). Boom-and-bust development patterns across the Amazon deforestation frontier. Science, 324 (5933), 1435–1437. https://doi.org/10.1126/science.1174002

Schiesari, L., Waichman, A., Brock, T., Adams, C., & Grillitsch, B. (2013). Pesticide use and biodiversity conservation in the Amazonian agricultural frontier. Philosophical Transactions of the Royal Society b: Biological Sciences, 368 (1619), 20120378. https://doi.org/10.1098/rstb.2012.0378

Schreinemachers, P., & Tipraqsa, P. (2012). Agricultural pesticides and land use intensification in high, middle and low income countries. Food Policy, 37 (6), 616–626. https://doi.org/10.1016/j.foodpol.2012.06.003

Singh, R. B. (2000). Environmental consequences of agricultural development: A case study from the green revolution state of Haryana, India. Agriculture, Ecosystems & Environment, 82 (1–3), 97–103. https://doi.org/10.1016/S0167-8809(00)00219-X

Smith, J. O., Smith, P., Wattenbach, M., Gottschalk, P. I. A., Romanenkov, V. A., Shevtsova, L. K., & Lisovoi, N. V. (2007). Projected changes in the organic carbon stocks of cropland mineral soils of European Russia and the Ukraine, 1990–2070. Global Change Biology, 13 (2), 342–356. https://doi.org/10.1111/j.1365-2486.2006.01297.x

Soares-Filho, B., Alencar, A., Nepstad, D., Cerqueira, G., Vera Diaz, M. D. C., Rivero, S., & Voll, E. (2004). Simulating the response of land-cover changes to road paving and governance along a major Amazon highway: The Santarém-Cuiabá corridor. Global Change Biology, 10 (5), 745–764. https://doi.org/10.1111/j.1529-8817.2003.00769.x

Soares-Filho, B. S., Cerqueira, G. C., & Pennachin, C. L. (2002). DINAMICA—A stochastic cellular automata model designed to simulate the landscape dynamics in an Amazonian colonization frontier. Ecological Modelling, 154 (3), 217–235. https://doi.org/10.1016/S0304-3800(02)00059-5

Spera, S. A., Galford, G. L., Coe, M. T., Macedo, M. N., & Mustard, J. F. (2016). Land-use change affects water recycling in Brazil’s last agricultural frontier. Global Change Biology, 22 (10), 3405–3413. https://doi.org/10.1111/gcb.13298

Tabatabaeinejad, S. M., Zinatloo-Ajabshir, S., Amiri, O., & Salavati-Niasari, M. (2021). Magnetic Lu 2 Cu 2 O 5 -based ceramic nanostructured materials fabricated by a simple and green approach for an effective photocatalytic degradation of organic contamination. RSC Advances, 11 (63), 40100–40111. https://doi.org/10.1039/D1RA06101A

Turner Ii, B. L., Villar, S. C., Foster, D., Geoghegan, J., Keys, E., Klepeis, P., & Vance, C. (2001). Deforestation in the southern Yucatán peninsular region: an integrative approach. Forest Ecology and Management, 154 (3), 353–370. https://doi.org/10.1016/S0378-1127(01)00508-4

Umetsu, F., & Pardini, R. (2007). Small mammals in a mosaic of forest remnants and anthropogenic habitats—Evaluating matrix quality in an Atlantic Forest landscape. Landscape Ecology, 22 , 517–530. https://doi.org/10.1007/s10980-006-9041-y

Usman, M., & Balsalobre-Lorente, D. (2022). Environmental concern in the era of industrialization: Can financial development, renewable energy and natural resources alleviate some load? Energy Policy, 162 , 112780. https://doi.org/10.1016/j.enpol.2022.112780

Usman, M., Balsalobre-Lorente, D., Jahanger, A., & Ahmad, P. (2023). Are Mercosur economies going green or going away? An empirical investigation of the association between technological innovations, energy use, natural resources and GHG emissions. Gondwana Research, 113 , 53–70. https://doi.org/10.1016/j.gr.2022.10.018

Verburg, R., Rodrigues Filho, S., Lindoso, D., Debortoli, N., Litre, G., & Bursztyn, M. (2014). The impact of commodity price and conservation policy scenarios on deforestation and agricultural land use in a frontier area within the Amazon. Land Use Policy, 37 , 14–26. https://doi.org/10.1016/j.landusepol.2012.10.003

Villela, A. A., D’Alembert, B. J., Rosa, L. P., & Freitas, M. V. (2014). Status and prospects of oil palm in the Brazilian Amazon. Biomass and Bioenergy, 67 , 270–278. https://doi.org/10.1016/j.biombioe.2014.05.005

Volante, J. N., Mosciaro, M. J., Gavier-Pizarro, G. I., & Paruelo, J. M. (2016). Agricultural expansion in the semiarid chaco: Poorly selective contagious advance. Land Use Policy, 55 , 154–165. https://doi.org/10.1016/j.landusepol.2016.03.025

Wang, F., Yu, C., Xiong, L., & Chang, Y. (2019). How can agricultural water use efficiency be promoted in China? A spatial-temporal analysis. Resources, Conservation and Recycling, 145 , 411–418. https://doi.org/10.1016/j.resconrec.2019.03.017

Wicke, B., Sikkema, R., Dornburg, V., & Faaij, A. (2011). Exploring land use changes and the role of palm oil production in Indonesia and Malaysia. Land Use Policy, 28 (1), 193–206. https://doi.org/10.1016/j.landusepol.2010.06.001

Wilcove, D. S., & Koh, L. P. (2010). Addressing the threats to biodiversity from oil-palm agriculture. Biodiversity and Conservation, 19 , 999–1007. https://doi.org/10.1007/s10531-009-9760-x

Zinatloo-Ajabshir, S., Heidari-Asil, S. A., & Salavati-Niasari, M. (2022). Rapid and green combustion synthesis of nanocomposites based on Zn–Co–O nanostructures as photocatalysts for enhanced degradation of acid brown 14 contaminant under sunlight. Separation and Purification Technology, 280 , 119841. https://doi.org/10.1016/j.seppur.2021.119841

Zinatloo-Ajabshir, S., Morassaei, M. S., Amiri, O., & Salavati-Niasari, M. (2020). Green synthesis of dysprosium stannate nanoparticles using Ficus carica extract as photocatalyst for the degradation of organic pollutants under visible irradiation. Ceramics International, 46 (5), 6095–6107. https://doi.org/10.1016/j.ceramint.2019.11.072

Zinatloo-Ajabshir, S., Morassaei, M. S., & Salavati-Niasari, M. (2019). Eco-friendly synthesis of Nd 2 Sn 2 O 7 –based nanostructure materials using grape juice as green fuel as photocatalyst for the degradation of erythrosine. Composites Part b: Engineering, 167 , 643–653. https://doi.org/10.1016/j.compositesb.2019.03.045

Zinatloo-Ajabshir, S., & Mousavi-Kamazani, M. (2021). Recent advances in nanostructured Sn−Ln mixed-metal oxides as sunlight-activated nanophotocatalyst for high-efficient removal of environmental pollutants. Ceramics International, 47 (17), 23702–23724. https://doi.org/10.1016/j.ceramint.2021.05.155

Zinatloo-Ajabshir, S., & Salavati-Niasari, M. (2019). Preparation of magnetically retrievable CoFe 2 O 4 @ SiO 2 @ Dy 2 Ce 2 O 7 nanocomposites as novel photocatalyst for highly efficient degradation of organic contaminants. Composites Part B: Engineering, 174 , 106930. https://doi.org/10.1016/j.compositesb.2019.106930

Zonarsaghar, A., Mousavi-Kamazani, M., & Zinatloo-Ajabshir, S. (2022). Sonochemical synthesis of CeVO 4 nanoparticles for electrochemical hydrogen storage. International Journal of Hydrogen Energy, 47 (8), 5403–5417. https://doi.org/10.1016/j.ijhydene.2021.11.183

Download references

Open access funding provided by University of Pécs.

Author information

Authors and affiliations.

Faculty of Business and Economics, University of Pécs, Pécs, 7622, Hungary

Arthur Pereira Sales

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Arthur Pereira Sales .

Ethics declarations

Conflict of interest.

The author states that there is no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Sales, A.P. Agricultural frontiers and environment: a systematic literature review and research agenda for Emerging Countries. Environ Dev Sustain (2023). https://doi.org/10.1007/s10668-023-04030-1

Download citation

Received : 22 March 2023

Accepted : 10 October 2023

Published : 25 October 2023

DOI : https://doi.org/10.1007/s10668-023-04030-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Emerging countries
• Literature review
• Agricultural frontiers
• Environment
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science

Kevin j. boudreau.

London Business School, London NW1 4SA, United Kingdom; and Harvard Business School, Boston, Massachusetts 02163

Eva C. Guinan

Dana-Farber/Harvard Cancer Center, Boston, Massachusetts 02215

Karim R. Lakhani

Harvard Business School, Boston, Massachusetts 02163

Christoph Riedl

D’Amore-McKim School of Business, Northeastern University, Boston, Massachusetts 02115

Selecting among alternative projects is a core management task in all innovating organizations. In this paper, we focus on the evaluation of frontier scientific research projects. We argue that the “intellectual distance” between the knowledge embodied in research proposals and an evaluator’s own expertise systematically relates to the evaluations given. To estimate relationships, we designed and executed a grant proposal process at a leading research university in which we randomized the assignment of evaluators and proposals to generate 2,130 evaluator–proposal pairs. We find that evaluators systematically give lower scores to research proposals that are closer to their own areas of expertise and to those that are highly novel. The patterns are consistent with biases associated with boundedly rational evaluation of new ideas. The patterns are inconsistent with intellectual distance simply contributing “noise” or being associated with private interests of evaluators. We discuss implications for policy, managerial intervention, and allocation of resources in the ongoing accumulation of scientific knowledge.

1. Introduction

A fundamental challenge that all organizations engaged in scientific and technological innovation face is how to allocate resources across alternative project proposals (e.g., Astebro and Elhedhli 2006 , Hallen 2008 ). Senior managers and scientific researchers alike devote significant time and effort to evaluating and selecting projects. Stevens and Burley (1997) find that executives have to manage, on average, more than 3,000 ideas to secure one commercial success. In science, tens of thousands of experts are involved in the annual evaluation of more than 89,000 research applications by the National Institutes of Health (NIH) and National Science Foundation (NSF) ( Stephan 2012 , Li 2015 ). The challenge of evaluating ideas has only grown with increasing use of “ideation,” platform-based contests, crowdsourcing, and crowdfunding as a means of generating a large number of proposals ( Agrawal et al. 2014 , Piezunka and Dahlander 2014 , Mollick and Nanda 2015 ). A common approach to evaluating innovative projects is to refer to experts with deep domain knowledge to assess quality of proposed projects, i.e., peer review ( Chubin and Hackett 1990 , Lamont 2009 ). In the United States, for example, academic research, which is the feedstock for many subsequent commercial innovations, depends on expert peer review to allocate more than $40 billion of research funds every year in engineering, medicine, science, and technology ( Xie and Killewald 2012 ). Contrary to the popular notion of a “marketplace for ideas,” in which the best ideas simply rise to the top, resource allocation in academic science is shaped in important ways by supporting institutions and processes ( Kuhn 1962 , Merton 1968 , Dasgupta and David 1994 , Stephan 2012 ). In this paper, we investigate how “intellectual distance”—the degree of overlap and relatedness between evaluators’ knowledge or expertise and the knowledge embodied in research proposals—plays a role in systematically shaping evaluation outcomes and consequent resource allocation in scientific peer review.

The evaluation and funding process for leading-edge scientific and technological projects is highly competitive. In the United States, for example, the NIH funds fewer than one in six applications, and for the NSF, it is one in four. Between one-third and one-half of rejected project proposals and their associated research lines are subsequently discontinued by their authors ( Chubin and Hackett 1990 ). Although rejected proposals might simply be of lower quality and deserve to be stopped, tremendous unexplained variation and seeming “noise” is the single most regular feature of scientific peer evaluations. Interrater reliability in funding decisions is routinely found to be very low (e.g., Rothwell and Martyn 2000 , Bornmann and Daniel 2008 , Jackson et al. 2011 ), with concordance sometimes “barely beyond chance” ( Kravitz et al. 2010 , p. 1) and “perilously close to rates found for Rorschach inkblot tests” ( Lee 2012 , p. 862). Variance among reviewers is sometimes greater than variance between submissions ( Cole et al. 1981 ). Beyond the fact of low interrater reliability, there is yet little agreement about underlying causes. Past research has argued that expert evaluation of research proposals may be shaped by any number of factors beyond the “true” quality of research, including researcher and evaluator characteristics, ties between researchers and their evaluators, proposal formats, and evaluation procedures. (See Marsh et al. 2008 and Lee et al. 2013 for comprehensive reviews and syntheses of the relevant findings.)

In this paper, we investigate whether the intellectual distance and relative positions in “knowledge space” between evaluators’ knowledge and the knowledge embodied in research proposals has systematic effects on evaluations. We motivate and study two specific conceptions of distance: intellectual distance per se and novelty, or departures from the established body of research.

We consider three theoretical perspectives and associated mechanisms through which positions in knowledge space might affect evaluations, independent of the true quality of a proposal. First, the evaluation process might simply be understood as a matter of evaluators each discerning a noisy signal of true quality, following a classical statistical decision making under uncertainty perspective. In this case, greater intellectual distance (less expertise, greater ignorance) would lead to less precise evaluations but no differences in mean evaluations. By contrast, a bounded rationality perspective predicts that cognitive limits and closely associated behavioral and heuristic responses lead to systematic biases with intellectual distance. An agency perspective suggests the possibility that some evaluators might adjust their evaluations—one way or another—in response to private interests.

Our empirical task is to precisely observe variation in intellectual distance and relate this to evaluation outcomes, independent of conflating factors, including the true quality of research proposals. A deep and fundamental challenge for research of this kind is that the true quality and potential of a research proposal is not observed and difficult to unequivocally infer–even after, if and when, the research is finally executed (e.g., Merton 1968 ). Therefore, a key feature of our research is to devise an approach to deriving inferences that does not rely on observing true quality. 1

To implement a suitable experimental research design, we collaborated with the administrators of a research-intensive U.S. medical school. We devised ways of modifying details of a research grant process for endocrine-related disease to allow us to make experimental comparisons. We then worked closely with the grant organization to manage, administer, and execute details. Key aspects of the design included recruiting an especially large number of evaluators, 142 world-class researchers from within the institution that were drawn from fields both inside and outside the disease domain. We randomly assigned each evaluator to 15 proposals from a total of 150 research proposals, yielding 2,130 evaluator–proposal pairs. The process was “triple blinded,” with evaluators and authors blinded to one another, and evaluators, too, blinded to one another. Focusing our analysis on the first stage of the grant process, in which ideas and new hypotheses were solicited and evaluated, allowed us to standardize the format and content of proposals and to simplify submission requirements so that we could restrict the process to single-author submissions. Thus, we could associate each proposal with fine-grained metrics at the level of individual submitters and evaluators.

We found that evaluators gave systematically lower scores to research proposals that were closer to their own areas of expertise. The relationships are strikingly large and driven by behaviors across a wide mainstream of the population. The variation in intellectual distance across this group of medical researchers accounts for 1.1 points of variation on a 10-point evaluation scale. (The standard deviation of evaluation scores overall is 1.7 points, after removing proposal and evaluator fixed effects.) Given the research design, these can be interpreted as causal effects. Simulating an alternative ranking scheme, we find that intellectually “closest” expert evaluators would have generated scores that would lead proposals to change their rank order by over 30 positions, on average. The evidence suggests that experts’ rank ordering is more meaningful than is that of averages of larger groups of (less expert) evaluators, particularly among highest-quality proposals.

Our second main finding is that more novel proposals are associated with lower evaluations, with magnitudes of effects comparable to those associated with intellectual distance. It is proposals with particularly high levels of novelty—the “right tail” of novelty—that account for this result. (For low levels of proposal novelty, evaluation scores were increasing with incrementally greater levels of novelty.) It is, of course, not possible to experimentally vary the novelty of a proposal entirely independently of its other characteristics. Therefore, it is important to note that we instead implemented a best feasible approach to estimate the relationship between novelty and evaluation scores, all else being equal. We use a series of specifications and diagnostics to rule out omitted variable bias.

These and each of a number of other patterns studied herein are consistent with biases rooted in bounded rationality in a context of especially high uncertainty ( Kahneman et al. 1982 , Johnson et al. 1982 , Camerer and Johnson 1991 ). In relation to intellectual distance, the pattern of lower scores provided by most expert evaluators is consistent with experts more readily “seeing” and “sampling” more informational cues than do less expert evaluators—with experts observing a disproportionately greater number of demerits, problems, and limitations of research proposals. In relation to novelty, the pattern of lower scores associated with novel proposals, along with other patterns, is consistent with boundedly rational evaluators systematically misconstruing ideas outside the established paradigm.

The range of patterns in these data is inconsistent with characterizations of the evaluation process as simply one of inferring true quality from noisy signals, as in classical statistical decision making under uncertainty characterizations. Findings are also inconsistent with evaluators being biased by private interests. We are, however, unable to rule out the possibility that novelty is somehow inevitably and inextricably associated with truly lower mean expected outcomes.

These findings have profound implications for evaluation of frontier projects. First, these effects are insensitive to usual procedures such as blinding of the identity of researchers. Second, unlike, say, the evaluation of prices in markets or product ratings by online “crowds,” bounded rationality implies limits to what can be achieved by tallying and aggregating large numbers of opinions. Third, whereas problems with intellectual distance have the potential to “scramble” rank ordering, problems with novelty have the potential to systematically dissuade experimentation.

The remainder of the paper proceeds as follows. Section 2 reviews past literature and motivates possible links between intellectual distance and evaluations. Section 3 describes the research design. Section 4 presents main results. These are discussed and interpreted in §5, together with a series of supplementary discriminating tests. Section 6 concludes.

2. Advancing Scientific Knowledge and Evaluations

In this section, we first describe how recurrent patterns of knowledge accumulation in science inevitably lead to some degree of intellectual distance between new research proposals and the knowledge of evaluators. We distinguish intellectual distance between particular pairs of research proposals and evaluators from novelty in relation to the entire existing body of research. We then discuss three distinct theoretical perspectives suggesting intellectual distance might shape and evaluations, independent of the true quality of a proposal. (Note, each of the perspectives reflects vast literatures, and we only provide a brief overview of arguments as a means of summarizing key differences in their implications.)

2.1. Intellectual Distance and Novelty in the Advance of Scientific Knowledge

Advances in scientific knowledge tend not to be a scattershot of isolated experiments in all directions but rather a series of regular accumulative patterns ( Gibbons et al. 1994 ). Initial progress on the resolution of a scientific problem gives rise to a scientific paradigm ( Kuhn 1962 ), defined as common knowledge and consensus on what is to be observed, which questions are legitimate and interesting to ask, what constitutes appropriate and useful approaches to addressing these questions, what methods might be fruitfully employed, and even what legitimate answers might look like. Thus, except in the rare instances in which one paradigm is abandoned for another, the stock of knowledge tends to grow by regular accretion within the prevailing paradigm.

Disclosure and diffusion of scientific knowledge through publication, conferences, seminars, textbooks, graduate training, and other means creates something of a common stock of open knowledge ( Boudreau and Lakhani 2015 ), as well as a commonly perceived knowledge frontier or an envelope that demarcates what is currently known from what remains to be investigated. New research, which by definition aims to extend the current state of knowledge, creates intellectual distance between evaluators and proposals if only by requiring evaluators to look beyond the existing knowledge frontier. Incrementally novel advances can be made by continuing within existing pathways and paradigms. More novel departures from the existing paradigm might also be pursued, in hopes of finding new viable research pathways and “breakthroughs” ( Uzzi et al. 2013 ). Thus, novelty should be considered a matter of degree. Just as incremental advances largely proceed in a cumulative process that draws on existing templates, knowledge, and ideas, novel departures themselves do not come from utterly unprecedented work. Rather, as documented in a range of empirical and theoretical considerations ( Weitzman 1998 , Fleming 2001 , Uzzi et al. 2013 ), novel approaches themselves draw on existing knowledge but tend to then recombine and reconfigure this knowledge in unprecedented ways.

Intellectual distance between a particular evaluator and particular research proposal also arises as a result of growing specialization as scientific research advances. Despite the open and shared knowledge commons, scientific knowledge remains too vast, nuanced, and complex to be understood in its entirety by any one scientist ( Cowan et al. 2000 , Jones 2009 ). Figure 1 illustrates the growth of scientific knowledge in the life sciences over 60 years (1950–2010) and the tendency toward specialization into subfields through the increase in the cumulative numbers of journals, articles, and research keywords. Even scientists that prima facie appear to be working in the same domain will differ in the particulars of their research program and differ in precise experience, training, and exposure to phenomena and methods. As a result, evaluation of new research proposals also requires evaluators to look across the knowledge frontier to other domains not precisely overlapping with their own expertise, training, and experience. Hence, the very nature of scientific inquiry and our society’s reliance on experts to evaluate and allocate resources generates intellectual distance between evaluators and new proposals and creates evaluation challenges.

Time Trend of Cumulative Numbers of Publications, Unique Journals, and Unique Pairs of Keyword Topics and Article Counts

Notes. Based on data from the PubMed database. Keywords are based on standardized lexicon (MeSH terms).

2.2. Three Perspectives on Intellectual Distance and the Evaluation of New Projects

Here, we review three broad theoretical perspectives, each motivating possible links between intellectual distance and research evaluations, apart from any differences in true research quality. Although these perspectives are not mutually exclusive or entirely independent, it is useful to consider their arguments in turn. Predictions of these perspectives are summarized in Table 1 .

Alternative Theoretical Mechanisms Possibly Relating Intellectual Distance to Evaluations

2.2.1. Agency Problems and the Private Interests of Evaluators

Much of the existing research on research evaluations hypothesizes some form of evaluator bias shaping evaluations. Most existing evidence is correlational and associative and not yet directly related to the question of intellectual distance. 2 Nonetheless, we take the more general point emphasized by this work that evaluators’ private interests might lead to systematic deviations between expected quality and reported evaluations. Even just the content of a research proposal may relate to private interests of evaluators. For example, a negative relationship between evaluations and intellectual distance could exist, if evaluators are inclined to be less critical of or to favor “close” research. This is plausible given the nature of institutions and rewards in science ( Stephan 1996 ). Increased attention can attract additional resources and renown for one’s area of research, boosting the prospects of all involved—including the evaluator. Equally, a negative relationship could exist if evaluators’ have preferences for given “schools of thought” or have a propensity for “cognitive cronyism” ( Travis and Collins 1991 , p. 323). Alternatively, a positive relationship could exist where, for example, research in the same domain and in close proximity is perceived to exert a negative externality on the evaluator, creating incentives to discount evaluations. For example, in certain instances, a close and competitive proposal might be expected to draw resources and attention away from an evaluator’s own work ( Campanario and Acedo 2007 ). Similarly, a wish to “protect” orthodox theories might dispose evaluators to look negatively at research that is both proximate and proposes a conflicting perspective ( Travis and Collins 1991 ). These biases, in whichever direction, might also occur more subtly than simply evaluation in bad faith, as when personal interests affect how much effort an evaluator is willing to devote to an evaluation ( Johnson and Payne 1985 ).

Past empirical research with some relevance to these arguments is not conclusive on these points. For example, several papers have failed to find upward bias in evaluations of research that cites evaluators ( Sandström and Hallsten 2008 , Sugimoto and Cronin 2013 ). Li (2015) finds clearer evidence of a positive causal bias toward close researchers in the context of NIH committee evaluation; however, committee dynamics and nonblinded evaluations make it difficult to interpret results in relation to intellectual distance per se.

2.2.2. Uncertainty, Risk, and Decision Theory Perspectives

Another theoretical perspective views proposal evaluation as akin to the problem of classical (statistical) decision making under uncertainty (e.g., Berger 1985 , Anand 1993 ). This might be understood in terms of reported evaluation scores ( V reported ) being understood as reflecting both some true, unobserved quality ( V true ) and some “error” term (e.g., Blackburn and Hakal 2006 , p. 378); i.e., V evaluation = V true + error . This perspective is implicit in the many references to “luck” and “noise” in the literature (e.g., Cole et al. 1981 , Marsh et al. 2008 , Graves et al. 2011 ). This view also relates to the common practice of averaging multiple evaluation scores in hopes of canceling noise and errors ( Lee et al. 2013 ).

Following this view, greater intellectual distance can be interpreted as an evaluator being less well informed—and therefore making an evaluation under greater uncertainty. Greater intellectual distance and uncertainty might perhaps manifest as a larger “error” term, potentially producing greater dispersion and variance among more distant evaluators without necessarily having any effect on mean evaluations. Alternatively, greater intellectual distance and uncertainty might reduce confidence in assessments—even where it has no effect on one’s evaluation—possibly encouraging discounting for perceived risk of more distant evaluations.

Novel research proposals may face an added hurdle. Apart from uncertainty in the form of risk or errors, novelty introduces a form of fundamental uncertainty that can not entirely be resolved without experimentation. It is thus difficult to assign probabilities to outcomes ex ante. In cases of such unresolvable uncertainty or “ambiguity,” researchers in the behavioral decision making under uncertainty literature have found that individuals tend to discount outcomes on the basis of “ambiguity aversion” ( Fox and Tversky 1995 ). This reasoning also predicts a negative relationship between novelty and evaluations.

2.2.3. Bounded Rationality and Expert Cognition Perspectives

Research on bounded rationality and expert cognition also suggests links between intellectual distance and evaluations. The literature in this tradition finds that, across a wide range of human endeavor, expert judgment is associated with qualitatively distinct cognitive processes from those of non-experts. Experts, those closest to a particular subject matter, are able to observe and exploit a far broader array of informational cues. They perceive and appreciate more detail, complexity, patterns, and meaning when making the very same observations as nonexperts (see Kahneman et al. 1982 , Johnson et al. 1982 , Camerer and Johnson 1991 ). These advantages in information processing are rooted in the development of a richer, more textured library of domain-specific knowledge accumulated through extended periods of training, experience, and practice. As a result, experts require the same or less time and effort to generate more discerning judgments ( Johnson and Russo 1984 , Johnson 1988 , Bedard 1989 ). Expert cognitive processes are even often seemingly automatic, and even instantaneous, as a result of knowledge stored and comprehended in “chunks” and mental maps of hierarchies, relationships, contingencies, and “configural rules” ( Fitts and Posner 1967 , Newell and Simon 1972 , Chase and Simon 1973 , Ericsson and Smith 1991 ).

Therefore, rather than a matter of intellectual distance resulting in more or less “error” in perceiving the same object, these points raise the possibility of information processing and “seeing more” creating differential sampling of information. Following this interpretation, the effect of intellectual distance and expertise depends on whether experts disproportionately see (sample) merits or demerits in relation to those perceived (sampled) by less expert evaluators. It is only when experts differentially sample positive merits as they do negative demerits of a research proposal (and also weight them equally) where we would not expect some effect of expertise on mean evaluations. If merits and contributions are much plainer to see than are more subtle questions of feasibility, implementation, and correctness, greater expertise could result in more negative evaluations. This suggests the possibility of a positive relationship between intellectual distance and evaluations.

A distinct branch of the research on cognitive biases, studying effects of extrapolating on the basis of one’s existing knowledge into new domains, also suggests implications around questions of novelty. Extrapolation beyond the domain for which knowledge was developed has been documented to result in sharply degraded performance, even to the point that human judgment becomes inferior to naïve actuarial models (e.g., Johnson 1988 , Sternberg 1996 , Chi 2006 ). Expert mental maps have thus been described as “brittle” ( Camerer and Johnson 1991 ) and subject to breakdown when applied to new areas ( Brehmer 1980 , Holland et al. 1986 , Camerer and Johnson 1991 , Chi 2006 ). These findings suggest that novel approaches might be systematically “misconstrued” if uncertainty surrounding them leads them to be interpreted on the basis of existing knowledge and mental maps. If this leads to discounted evaluations, a negative relationship between evaluations and “novel” research proposals will manifest.

2.3. Summary and Research Questions

Intellectual distance is a regular feature of the evaluation process and deserves careful study as a variable that might influence evaluation and resource allocation in science. The theoretical perspectives reviewed above and the mechanisms they suggest are summarized in Table 1 , with predictions in relation to mean evaluations. Several points relate specifically to the case of novel departures from existing research approaches. Our main goal in this study is to test for systematic relationships between evaluation scores and intellectual distance. A secondary goal is to attempt to rule in and rule out alternative theories.

3. Research Design

In this section, we describe the setting and research design, providing details on proposal generation, evaluator recruitment, random assignment, and our key measures.

3.1. A Call for Research Proposals from the “First Phase” of a Grant Process

We carried out our research in the context of a scientific grant solicitation and evaluation process for research on endocrine-related disease, a major economic and health burden on society and a focus of considerable research effort at the host medical school. Working closely with grant administrators, we altered the usual grant procedures to allow us to make precise observations and to derive meaningful inferences. The grant process we studied involved seed grant awards, intended to enable investigators to initiate their research efforts to generate preliminary data (to support later NIH grant applications).

In terms of defining the scope, we deliberately defined the grant solicitation in terms of a disease area rather than making any mention of existing literature, the existing body of scientific knowledge, or established research pathways. The articulated aim for the grant was otherwise stated in general terms of directing research attention and financial resources to make progress in endocrine system–related disease research, treatment, and care. The content of proposals was otherwise unconstrained; we welcomed submissions related to diagnosis, treatment, and pro-phylaxis. To attempt to draw a variety of submissions, the university president communicated an open call to participate to all members of the medical school and broader university community via email.

A fundamental research design choice was to partition the grant proposal process into two phases. The first, involving solicitation of proposals for approaches and ideas, was essentially a call for research hypotheses. It is this first phase—of defining research goals, approaches, and hypotheses—that is most relevant to the questions raised earlier (in §2). Partitioning the proposal process in this manner also reduced “entry costs” to prospective submitters, making it possible to document submissions in shorter proposals. (The average proposal length in this exercise was roughly six pages.) This design decision also allowed us to require submissions be authored by individual scientists rather than teams. Thus, we could associate each proposal with the attributes of the individual submitter. The shorter and more standardized proposal format also allowed us to minimize the extent to which submission format shaped evaluations ( Langfeldt 2006 ).

Explicit incentives in this process included a $2,500 cash prize awarded to each of the top 12 winners. The process also generated additional incentives: the winning proposals would form the basis for a call for research proposals, the second phase, in which a total of $1 million in seed grants would be available. Being in the top of the first phase increased the odds of being able to create a successful second-stage proposal. (Indeed, four second-phase winners were also first-phase winners.) The first phase of the process also served as a platform for high-profile exposure among peers and university leaders, as awards were conferred by the dean of the medical school in a formal public ceremony attended by colleagues, White House staff, and members of the media. This process elicited 150 research proposals, with 72 coming from within the host university.

3.2. Recruiting Evaluators

Major funding agencies regularly invite researchers with relevant subject knowledge to participate in evaluating research proposals ( Langfeldt 2006 ). An ad hoc evaluation team might include a few, perhaps five to seven ( Langfeldt 2006 ), specialized researchers whose phenomenological interest, research methods, and/or topical focus relate to the research proposal(s) in question ( Jayasinghe et al. 2003 ). More extensive evaluation processes covering large numbers and steady flows of proposals, such as those employed by the NIH and NSF, often involve standing committees and subcommittees formed around topic areas to which proposals are directed, as appropriate. Such committees can be as large as 30 to 50 researchers ( Li 2015 ) and their identities publicly disclosed.

Given our interest in generating variation, as well as abundant replication and degrees of freedom, we recruited roughly equal numbers of evaluators from among three distinct groups of host university faculty: (i) those with at least one publication in the disease area, (ii) those without publications in the particular disease area but with at least one publication with someone with a publication in the disease domain, and (iii) those without any publications or links to the disease area. Within each of these groups, we recruited equal numbers of senior and junior faculty (30 of each). We populated these six groups by rank-ordering faculty at the medical school according to publication counts and inviting the top-ranked faculty from each of the three groups to participate. Drawing on faculty from the host university ensured high-caliber participants, independent of rank. Strong institutional support helped minimize dropout. Of the 180 invitations (i.e., 6 groups times 30 invitations per group), 142 individuals accepted and participated in the exercise. This produced roughly equal proportions, balanced across the groups in relation to both the literature and junior and senior scholars. Each group also reflects considerable diversity in gender, age, and training (in terms of M.D. or Ph.D.). The group is uniform in including just highly accomplished researchers, with an average publication count of 101. Submitters are themselves accomplished but clearly more junior, on average, with roughly 1/10 as many publications, on average.

3.3. Evaluator Assignment and the Evaluation Process

Our assignment of evaluators and proposals yielded 2,130 proposal–evaluation pair observations. Ten blocks of 15 research proposals, randomly drawn from 150 total, were randomly assigned to each of the 142 evaluators, giving an average of 14.2 randomly selected faculty per proposal. Block randomization in this fashion was implemented to ease back-office implementation of the procedure by administrators at the institution. 3 Following convention in medical research grant proposal evaluations, the task of evaluators was to score proposals by responding to the question, “On a scale of 1 to 10 (1 [being the] lowest to 10 [being the] highest) please assess the impact on disease care, patients, or research.”

Given our interest in having evaluators respond to the content of proposals rather than the identities of submitting researchers, we designed the process to minimize the probability of identities being revealed. Submitters’ names were blinded on proposals, and evaluators, whose identities were also blinded, performed their evaluations independently and had access only to the 15 assigned proposals. Evaluators neither knew the names of nor interacted with other evaluators. With evaluators thus effectively blinded from one another, the overall evaluation process was triple blinded.

3.4. Data Collection and Variables

Our central concerns are to measure the relationship between evaluation scores and intellectual distance and to novelty in relation to existing research. We therefore devised means of measuring these key objects and identified several control variables relevant to our analysis. The data set includes evaluators’ score sheets, submitted proposals, detailed backgrounds, and résumés (of those evaluators and submitters at the host university) from the host university’s database; third-party topical keyword coding of submissions; and the PubMed database (an extensive database of research publications in life sciences). An overview of definitions and summary statistics for the main variables are provided in Tables 2 and ​ and3, 3 , respectively.

Definitions of Main Variables

Means, Standard Deviations, and Correlations of Main Variables

3.4.1. Evaluation Scores

The main dependent variable, EVALUATION_SCORE , is the integer score from 1 to 10 (mean = 5.7; mode = 7; s.d. = 2.6) given by evaluators in response to the main scoring question. Figure 2 displays all scores assigned to each proposal. Proposals appear in descending order by average score, along the x axis. (The average score was the basis for conferring awards.) Figure 2 also presents the plus and minus of one standard deviationas a means of highlighting the consistently wide variation in evaluations across each proposal. The patterns are consistent with considerable noise in the evaluation process. For example, dummy variables for individual research proposals explain just 26% of variation in terms of the R 2 statistic; dummy variables for individual evaluators explain 19% of variation in terms of the R 2 statistic.

Evaluation Scores for Each Proposal, Ordered By Mean Scores (Mean and Plus/Minus One Standard Deviation Shown)

Note. Individual integer scores are vertically randomly “jittered” to avoid overlap.

3.4.2. Intellectual Distance Between Evaluators and Research Proposals

A first approach to measuring intellectual distance in our setup is simply to distinguish those evaluators who have previously published within the disease domain versus those who have not, as captured by the indicator variable OUTSIDE_DOMAIN . We also constructed a continuous measure of intellectual distance on the basis of keywords used to describe and categorize the content of research in the life sciences, collectively referred to as Medical Subject Heading (MeSH) terms. This is a controlled vocabulary used by the U.S. National Library of Medicine to index articles for PubMed. MeSH keywords are assigned not by authors but rather by professional science librarians trained specifically to perform this task. Use of this controlled vocabulary is intended to ensure global and consistent assignment of keywords across the life sciences ( Coletti and Bleich 2001 ). We hired a professional librarian trained in standardized procedures for evaluating the content of research according to NIH National Library of Medicine (NLM) guidelines to code the proposals. We used the 2012 edition of the MeSH set, which contains 26,579 terms. On average, proposals in our sample were assigned 12.42 MeSH terms (s.d. = 5.42). This enabled us to represent each proposal as a vector of ones and zeroes, depending on relevant MeSH terms. We constructed analogous vectors to reflect evaluators’ backgrounds, with counts of numbers of papers referring to MeSH terms. Our continuous measure of intellectual distance is then simply the angular separation or cosine between the vectors for the proposal and the evaluator, expressed as a percentile, EVALUATOR_DISTANCE . The value of 1% reflects the closest and 100% the greatest intellectual distance. We refer to “evaluator” distance in naming this variable to emphasize that distance varies in relation to evaluator–proposal pairs. Formulating the variable as a percentile lead the distribution to be uniform and also eased interpretation; coefficients can be directly read as the effect of moving from the min (1st) to max (100th) percentile. (Alternative formulations of the variable produce similar results, as noted in the analysis.)

3.4.3. Novel Departures of Proposals from Existing Research

Our measure of novelty is also based on the MeSH lexicon. MeSH keywords attributions are intended to capture key aspects of the research, including scientific approach, topic, methods, and other key issues. To develop a measure of novelty, we therefore simply looked for novelty in MeSH term combinations in relation to the existing literature. We compared the MeSH term combinations of a proposal with combinations that appear in the entire existing scientific literature, as reflected in the PubMed database. 4 We examined all possible pairs of MeSH terms (i.e., for N terms there would be N ( N − 1)/2 pairs) and determined what fraction of these pairs for a given proposal had not previously appeared in the accumulated literature. The variable is then expressed as the percentile, PROPOSAL_NOVELTY , with 1% being least and 100% most novel. We refer to “proposal” novelty in the naming of this variable to emphasize its relation to the proposal in relation to the broader stock of research rather than to any one evaluator. (Alternative formulations of the variable produce similar results, as noted in the analysis.)

3.4.4. Other Variables

The analysis relies most heavily on the research design’s randomization and exploitation of multiple observations per proposal and per evaluator, with a series of dummy variables for evaluators and proposals providing controls. We also use a series of proposal covariates as a control vector (number of words, number of references cited, number of figures, presence of an introductory section that provides context in the proposal) where we cannot use proposal dummy variables. We discuss the relevance of these covariates in the analysis to follow.

4. Main Results

Here, we present our main results, estimating the relationship between evaluation scores and intellectual distance, and with proposal novelty. We report results in separate subsections, given that estimates of relationships with distance and novelty require different econometric approaches.

4.1. Intellectual Distance and Evaluation Scores

The evaluation of proposal i by evaluator j can be shaped by proposal covariates ( X i ) (e.g., underlying quality, type), evaluator covariates ( X j ), and luck or noise, which we describe with a zero-mean error term ( ε ij ). Regarding pairwise proposal–evaluators variation, our main focus here is on intellectual distance between evaluators and proposals ( EVALUATOR_DISTANCE ). (The design of our experiment controls for other pairwise factors, such as relationships among evaluators and researchers.) These variables relate to evaluation scores through some function g (−), EVALUATION _ SCORE ij = g ( EVALUATOR _ DISTANCE ij , X i , X j ; ε ij ). Our empirical models estimate this expression in a series of linearly separable specifications. Coefficients and robust standard error estimates are reported in Table 4 . 5

Estimated Relationship Between Evaluations ( EVALUATION_SCORE ) and Intellectual Distance Between Evaluators and Research Proposals ( EVALUATOR_DISTANCE )

Note . Heteroskedasticity-autocorrelation robust standard errors are reported; number of observations = 2,130 research proposal–evaluator pairs.

We begin with a most straightforward comparison between evaluation scores of those evaluators who have conducted research within the disease domain versus those who have not. As in Model 1, evaluation scores of those outside the disease domain are 0.37 points higher (s.e. = 0.12), on average. Given randomized assignment, adding proposal dummy variables, as in Model 2, does not change the estimated coefficient but reduces standard errors. 6

Apart from discrete differences, we expect that continuous variation in intellectual distance will also shape evaluations. We therefore add our continuous measure, EVALUATOR_DISTANCE , to the model. 7 As reported in Model 3, we again find a positive relationship with distance, the estimated coefficient on EVAL-UATOR_DISTANCE being 1.10 (s.e. = 0.19).

Importantly, using the continuous measure allows us to introduce evaluator dummy variables as controls. Thus, our preferred and most stringent specification includes dummy variables for both research proposals ( η ) and evaluators ( δ ) (with OUTSIDE_ DOMAIN dropping out of the model) as follows:

where ε is a zero-mean error term. As reported in Model 4, this produces a slightly smaller, but statistically unchanged, coefficient on EVALUATOR_ DISTANCE (0.86, s.e. = 0.33).

Therefore, there is a large positive relationship between evaluation scores and intellectual distance. Given a random assignment of proposals to evaluators, the estimated relationship can be interpreted as a causal relationship. Therefore, not only do specialized experts provide more discerning evaluations but they also provide systematically lower—and more critical—evaluations. Having defined EVAL-UATOR_DISTANCE in terms of percentiles, we can interpret the coefficient as indicating a roughly one-point difference in score across the entire population, with varying intellectual distance, in addition to the earlier-reported 0.4 added points for those outside the research domain. This is a large effect in comparison with the standard deviation of evaluation scores, 2.6 (or a standard deviation of 1.7, if calculated after removing proposal and evaluator fixed effects).

4.2. Novel Departures from Existing Research and Evaluation Scores

We now examine the relationship between evaluation scores and novelty. Because this reintroduces a proposal covariate, PROPOSAL_NOVELTY , to the model, we can no longer exploit proposal dummy variables. Instead, we include a vector of precise proposal covariates, X j , as control variables, as follows:

where we continue to control for evaluator characteristics with dummy variables, δ i ; ζ is the vector of parameters to be estimated on control variables. The error term is redefined accordingly. We control for differences in scores related to different specific fields and topics with the series of dummy variables of individual MeSH terms. We control for differences in quality with numbers of author publications and citations. We also control for a series of descriptive features of proposals (number of words, number of references cited, number of figures, presence of an introductory section that provides context in the proposal). Exploiting this control vector requires that we study just the subsample of 689 proposal–evaluator pairs for which we have these control variables (i.e., submissions from within the host university) rather than our full sample of 2,130 evaluator–proposal pairs. This leaves ample degrees of freedom, and the mean and variance of EVALU-ATION_SCORE are statistically the same in the sub-sample. Results are reported in Table 5 . 8

Estimated Relationships Between Evaluations and Proposal Novelty

Note . Heteroskedasticity-autocorrelation robust standard errors are reported; number of observations = 689 proposal–evaluator pairs and pertain only to submitting researchers from within the host university.

Model 1 regresses evaluation scores on PRO-POSAL_NOVELTY , together with evaluator dummy variables and the control vector of proposal covariates. The estimated coefficient on PROPOSAL_ NOVELTY is large and negative, at −2.67 (s.e. = 0.64). Most of the coefficients on proposal covariates are statistically significant. The exception is the number of words per proposal, which becomes insignificant when included with other proposal variables (but is positive and significant as other control variables are dropped). The control vector is highly effective at accounting for proposal characteristics; variation explained (unadjusted R 2 = 0.428) is even almost the same as when proposal dummy variables were earlier included (unadjusted R 2 = 0.475). Therefore, introducing the long list of controls leaves little room for lingering omitted variable bias—if only because there is little omitted variation.

If the model is indeed well controlled and there is little scope for unobserved proposal characteristics that spuriously account for the negative relationship with novelty, then introducing more controls should have no effect on estimates. To assess this point, Model 2 reestimates the model, adding controls for the number of author citations in the past seven years (in case the recency of citations plays a role), counts of publications in which the researcher appears as first author, and the maximum number of citations to any one of a researcher’s publications. As in Model 2, adding these controls has no effect on coefficient estimated on PROPOSAL_NOVELTY . 9

Model 3 introduces EVALUATOR_DISTANCE into the model at the same time; the coefficient on PROPOSAL_NOVELTY is unchanged with this change. Further, the coefficient on EVALUATOR_DISTANCE is itself statistically unchanged from earlier estimates in Table 4 that used proposal fixed effects (rather than the control vector used here). 10 This again affirms the effectiveness of our specification in isolating the relationship of interest. (As discussed in §5.1, there is no interaction between distance and novelty.) Therefore, the all-else-being-equal relationship between evaluation score and novelty is negative.

Having established the meaningfulness and stability of our model specification, we move to investigating whether the relationships of interest are nonlinear. In Figure 3 , we present results in which we allow for nonlinear relationships between evaluation scores and our measures of intellectual distance and novelty in two different specifications. The first approach is to simply add quadratic terms for both distance and novelty to the model (while continuing to control for evaluator fixed effects and a full complement of proposal covariates as controls). The second approach is to include a series of dummies for different levels (five quintiles) of both intellectual distance and novelty variables, estimating them in the same model. (Estimating the relationships with distance and novelty simultaneously or in separate models leads to similar patterns.) We present the estimates related to distance and novelty across these two models in the two panels of Figure 3 . They each produce similar results. We find that the relationship with intellectual distance is in fact linear and increasing, with no evidence of nonlinearities. Both models also indicate that the negative relationship between evaluation scores and novelty is largely driven by the most novel proposals. These are proposals in the fifth quintile of novelty, which might be understood as the right tail of novelty. The overall relationship is nonmonotonic, because at low levels of novelty there is an increase of scores with increases in novelty.

Flexible, Nonlinear Specification (Second-Order Polynomial and Quintile Means)

Notes. Shown are 90% confidence intervals. See §5.2 for discussion of specifications.

5. Evaluation of Alternative Explanations

Here, we interpret results in light of theoretical perspectives described in §2.2 (summarized in Table 1 ). We find that it is only the third of the perspectives we consider here—a bounded rationality characterization of the evaluation process—that is wholly consistent with patterns observed here. Thus, the first two subsections primarily deal with ruling-out possible explanations, and it is the third that finally rules in an explanation.

5.1. Agency Problems and Private Interests

Here, we consider the possibility that agency problems lead some evaluators to bias their evaluations upward or downward, depending on how they perceive that “close” research proposals will influence their own careers and private interests (§2.2.1). A series of patterns in the data run counter to this explanation.

5.1.1. Mean Responses and Bias

One possible interpretation of agency problems is that both low intellectual distance and low novelty can be regarded as close research. Therefore, whatever the general directional response to close research, we would expect the same direction of response in relation to both low novelty and low distance. However, in earlier analysis we measured that distance and novelty relate to scores with opposite signs.

Another possible interpretation of agency problems is that research that is at low intellectual distance is close, but it is only close research that is at the same time highly novel that might present a competitive threat to evaluators. However, we find no significant interaction between novelty and distance in explaining evaluation scores (see Table 6 , Model 1).

Interactions Between Evaluator Distance and Factors Plausibly Influencing Incentives and Behaviors

In addition, if private interests were to play some sort of systematic role, we would expect to see a heightened response (of some sort) to research that is especially close, perhaps resulting in a step function response on scores or at least some sort of nonlinear effect or impact on variance of scores. However, the relationship between scores and intellectual distance is linear with no signs of outsized response or even greater variance in the case of close research.

5.1.2. Heterogeneous Responses Across the Distribution of Evaluators

To investigate the possibility that certain evaluators are perhaps more susceptible than others to agency problems and bias and that this might result in heterogeneous responses across evaluators, we reestimate the model described in expression (3), allowing the coefficient on EVALUATOR_DISTANCE to be heterogeneous across evaluators β i ~ N ( β ¯ , σ β 2 ) .

Estimating this random coefficient specification, we find that the mean coefficient on our intellectual distance variable, β ̄ , is 1.48 (s.e. = 0.41), and we estimate that the standard deviation of this coefficient across the population of evaluators, σ β , is 0.61 (s.e. = 0.32). The relative size of the positive mean to standard deviation indicates that the response to intellectual distance is overwhelmingly mostly positive across the population. To underline this point, Figure 4 plots fitted individual linear relationship estimates for each evaluator across the multiple proposals they each evaluated, demonstrating the consistency of responses across evaluators.

Fitted Linear Relationships for Individual Evaluators

Note. Quantile and mean fitted lines are also shown to provide additional perspective on the distribution of data; each is regressed as a second-order polynomial.

5.1.3. Interactions and Evaluator Types

As still another test for agency problems, we examine whether effects of EVALUATOR_DISTANCE somehow systematically vary with factors plausibly linked to strength of self-interest, strategic orientation, and susceptibility to agency problems. As reported in Table 6 , we test for possible interactions with novelty (Model 1), evaluator seniority (Model 2), years since graduating (Model 3), and gender (Model 4), as well as all interaction terms at once (Model 5). We find no significant interactions.

5.2. Uncertainty, Risk, and Decision Theory Perspectives

Here, we consider the possibility that the evaluation process is analogous to a statistical decision-making problem, whereby greater distance and uncertainty creates noisier “signals” of the unobserved true quality of proposals (see §2.2.2).

It is also difficult to reconcile this perspective with the data, beginning with the most basic mean associations. For example, on the one hand, this perspective predicts that expected true mean assessment of quality should be invariant to distance and uncertainty. But evaluator scores systematically varied with both distance and novelty. On the other hand, it is possible that evaluations were risk-discounted relative to the expected mean quality, as uncertainty and distance increase. However, instead of a negative relationship with distance, we see a positive one. Only the relationship with novelty is negative.

5.2.1. Dispersion and Variance

This perspective also suggests the possibility of greater uncertainty leading to wider variance and dispersion of evaluations with varying distance or novelty. Simple descriptive statistics provide no indication of differences in variance at low and high levels of either intellectual distance or novelty. For example, the standard deviations of evaluation scores for fifth quintiles of either distance or novelty are no different from the standard deviation for lower quintiles. To investigate this possibility, here we reestimate the earlier model but allow the model error term to vary with novelty and distance, redefining the error term as m ij · ε ij , where multiplier m is allowed to vary with key explanatory measures: m ij = 1 + β ε · EXPERT _ DISTANCE + γ ε · PROPOSAL _ NOVELTY j . We simultaneously estimate conditional mean and error model coefficients via maximum likelihood. Coefficients in the conditional mean model are statistically unchanged in this specification, and estimated coefficients in the error term multiplier expression are statistically indistinguishable from zero ( β ε = −0.21, s.e. = 0.17; γ ε = −0.19, s.e. = 0.16). (Reestimating the multiplier model with quadratic terms or any subset of the variables, one at a time, produces the same zero result.) Therefore, we find no evidence of changing variance and dispersion with either distance or novelty.

5.3. Bounded Rationality and Expert Cognition Perspectives

Here, we consider the possibility that uncertainty is sufficiently high in evaluations where heuristic and behavioral decision making play a prominent role—a bounded rationality perspective (see §2.2.3).

As regards intellectual distance, this perspective suggests that those with most relevant knowledge, closest experts, will better discern informational cues, sample from a wider array of information, and make better sense of these cues. Following this perspective, the finding of a positive relationship between evaluations and intellectual distance (more negative evaluations by closest experts) is consistent with experts being more critical—applying more extensive tests, uncovering more errors, problems, and limitations.

To seek additional evidence related to bounded rationality and intellectual distance, we compared rank ordering based on the 15 randomly assigned evaluators with rank ordering based on scores given by intellectually closest evaluators (“experts”) for each proposal from 150 proposals. 11 On average, rank order is different by a staggering 31.8 positions (s.d. = 26.0). 12 To look for evidence as to whether experts provide more discerning evaluations than do less expert groups, we examine whether reducing the idiosyncratic noise of expert evaluations (taking out individual fixed effects and correcting for varying distance) leads expert rank ordering and less expert group average rank ordering to become more or less similar. 13 We find that taking away noise from expert rankings leads them to become more different from rankings of less expert group averages—but only for high-quality proposals. These patterns are consistent with expert evaluations being more discerning in relation to the subtle differences separating high-quality proposals.

As regards novelty, the bounded rationality perspective suggests that established knowledge and mental models are “brittle,” and this leads to systematic errors in judging new ideas (see §2.2.3). This is consistent with our finding of a negative relationship between evaluation scores and novelty. This is also perhaps consistent with the negative relationship being largely driven by the most novel proposals (see Figure 3 ). For example, some minimal amount of novelty is necessary in making a research contribution. However, it is perhaps just largest novel departures that are most likely to be misconstrued and discounted.

Each of the other patterns documented in this and previous sections are themselves also reconcilable with the bounded rationality perspective. For example, the “smooth” and gradually changing relationships documented in Figure 4 are consistent with gradual changes in cognition and behaviors with incrementally varying uncertainty. The similar behavior across the wide cross section of evaluators (see Figure 4 ) and absence of distinct effects across different groups (see Table 6 ) is consistent with the universal effects of bounded rationality. The invariance of dispersion in evaluations with varying levels of uncertainty—whether measured in terms of distance or novelty (see §5.2)—is consistent with common behavioral and heuristic responses to uncertainty.

6. Summary and Conclusions

This paper reported the results of an experiment designed to evaluate how evaluation scores of scientific research proposals are related to intellectual distance (between evaluator knowledge and the knowledge content of research proposals) and novelty (of research proposals in relation to the body of accumulated research). We conducted our field experiment as part of a regular research grant proposal process involving a group of world-class medical researchers. We worked closely with grant administrators to alter and manipulate features of a grant proposal process to implement a controlled research design. We focused on effects of relative positions in “knowledge space” (intellectual distance and novelty), striving to isolate these effects from the many other factors plausibly influencing evaluations. Important in this regard, we implemented a triple-blinding procedure while having individual evaluators (working in isolation) evaluate single-authored research proposals that followed a standard format. Randomization of assignment allowed us to estimate causal effects of intellectual distance (between evaluator–proposal pairs) on evaluation scores, holding proposal and evaluator characteristics constant. It is, of course, not possible to experimentally vary the novelty of a proposal entirely independently of its other characteristics. Therefore, we implemented a best feasible approach to estimate the relationship between novelty and evaluation scores, all else being equal. Given limitations in observing true quality and potential of research initiatives—even ex post—a key feature of our research design is to derive inferences without relying on observing true quality.

6.1. Results

We found that evaluators gave systematically lower scores to research proposals that were closer to their own areas of expertise. Within the range of variation observed here, the effect of intellectual distance on evaluation scores was large—a 1-point or more difference on a 10-point scale. These effects are observed across a wide cross section of evaluators. By contrast, we found no evidence of changing variance (deviations from mean model predictions) with varying intellectual distance. Therefore, closer experts were systematically more critical in the sense of assigning lower scores.

Our second main finding is that more novel proposals are associated with lower evaluations. The size of the relationship is large and comparable in magnitude, in these data, to the earlier effect of intellectual distance on evaluation scores. It is proposals with particularly high levels of novelty that account for this result. (For low levels of proposal novelty, evaluation scores were increasing with incrementally greater levels of novelty.) We found no evidence of changing levels of variance in scores at different novelty levels. A series of alternative specifications and diagnostic tests all but rule out the possibility that unobserved proposal characteristics somehow account for the observed patterns.

6.2. Interpretation

We considered a range of possible explanations for the patterns. Only theories emphasizing the bounded rationality of evaluators ( Kahneman et al. 1982 , Johnson et al. 1982 , Camerer and Johnson 1991 ) provide explanations for all observed patterns, on their own.

In relation to intellectual distance, bounded rationality characterizations suggest that closer experts “see” or “sample” more informational cues than do nonexperts. (There is no reason to expect that added informational cues seen by experts should necessarily be sampled equally from both merits and demerits of a proposal.) The pattern of lower scores provided by most expert evaluators is consistent with experts more readily discerning added demerits, problems, and limitations of research proposals rather than hidden demerits, in relation to what is perceived by less expert evaluators. Also consistent with this interpretation, counterfactual simulations comparing experts with wider groups suggested expert judgment was especially more discerning when judging higher-quality proposals (where differences in quality are presumably more subtle).

In relation to novelty, a bounded rationality characterization suggests that experts extrapolating beyond the knowledge frontier to comprehend novel proposals are prone to systematic errors, misconstruing novel work. This implies that rather than receiving unbiased assessments (with zero-mean errors), novel proposals are discounted relative to their true merit, quality, and potential.

Other theories failed to reconcile with all or some of the patterns documented here. For example, the patterns are inconsistent with evaluators shading their scores in relation to private interests. The patterns are also inconsistent with a statistical decision theory perspective in which evaluations simply become more “noisy” with greater distance and uncertainty.

The negative relationship between evaluation scores and proposal novelty is, however, consistent with possible discounting on the basis of uncertainty and ambiguity ( Fox and Tversky 1995 ). But this leaves the question of why greater uncertainty from greater intellectual distance is not then also discounted. It is plausible that the ambiguity associated with novelty plays some sort of special role, whereas uncertainty associated with distance does not, but then ambiguity only exists in a context of bounded rationality, complementing the earlier bounded rationality interpretation of patterns.

Our analysis ruled out the possibility that unobserved proposal characteristics that are unrelated to novelty somehow accounted for novel proposals receiving lower evaluations. Nonetheless, on a much subtler point, it remains possible that novelty per se is unavoidably and inextricably linked to lower expected mean outcomes (cf. Fleming 2001 ). However, if evaluations were to, in fact, reflect true quality, it remains then a question why we do not also see greater variance of evaluations associated with novel proposals to reflect greater true variance of these proposals ( Fleming 2001 ).

As regards generalizability of findings, we should emphasize that evaluators in these data were each world-class medical researchers drawn from both inside and outside the disease area (endocrine-related disease). This implies a span of intellectual distance that is perhaps closer to that within distinct subfields of natural sciences, engineering, or social sciences—rather than to larger differences across distinct fields. Just as we did not study especially intellectually distant evaluators, we also did not study differences between experts and lay people here.

6.3. Contributions and Relationships to Literature

Our work relates to the evaluation of frontier innovative projects. However, it most specifically contributes to several decades of research on scientific evaluations. To date, the bulk of this research has been carried out in fields of life sciences, medical research, and science policy (see, e.g., Cole et al. 1981 , Chubin and Hackett 1990 , Lee et al. 2013 ) with yet limited attention from social scientists, economists, and management scholars. Within this literature, our paper adds to the few studies attempting to make causal empirical inference (e.g., McNutt et al. 1990 , van Rooyen et al. 1999 , Li 2015 ). We believe that pursuit of careful causal inference (along with careful distinctions between underlying mechanisms) is especially important both because of the potentially staggering consequences of resource (mis)allocation in innovation and science and because many of the theories and claims in this literature are both complex and controversial.

Certainly, questions of “distance” between researchers and evaluators have been considered in research on scientific evaluations, but the focus thus far has been on distance between evaluators and researchers in terms of factors such as race, gender, and social and professional networks. Our work departs by instead considering relative positions and distance in knowledge space between evaluators and the proposals (rather than the researchers) they evaluate. In this regard, our study is closest to Li’s (2015) study of committee grant evaluations at the NIH. She finds that proposals citing committee members are more likely to win a grant. Further, grant decisions on proposals citing committee members are more closely correlated with a proxy for research quality (a citation-based measure) than are proposals that do not cite committee members. Li interprets these patterns as committees generally not only favoring researchers who cite them but also being more familiar with those researchers. Especially relevant here, her finding that citing proposals having a greater probability of being granted is directionally opposite to our finding that close intellectual proximity causes lower evaluations. This might relate to a number of institutional details of the NIH context that differ from our artificially manipulated and controlled environment, including researcher identity not being blinded, the evaluation process being conducted via open committee discussions, and potentially differing levels of variation observed in intellectual distance. It is also possible that the sorts of researchers with a high hazard of citing NIH committee members differ from those with a low hazard of doing so. Li’s study thus complements our own in highlighting the importance of additional research in seeking to better comprehend how these added issues and mechanisms drive outcomes in evaluation.

Our research also relates to (and was inspired) the observation that research, innovation, and technical advance tend to advance in an incremental fashion within defined paradigms , knowledge trajectories , research pathways , or dominant designs —some basic approach to solving the problem—that is then incrementally refined through a continuous series of cumulative, incremental advances (e.g., Kuhn 1962 , Dosi 1982 , Sahal 1985 , Romer 1990 , Gibbons et al. 1994 , Murray and O’Mahony 2007 , Furman and Stern 2011 , Williams 2013 , Boudreau and Lakhani 2015 ). This overriding tendency toward within-paradigm, incremental advance rather than more novel and exploratory innovations might be explained by any number of mechanisms, such as the strategic incentives and organization of innovators (e.g., Kuhn 1962 , Utterback and Abernathy 1975 , March 1991 , Manso 2011 ). The current paper raises another possibility: even when novel projects are proposed, they are met with resistance from relevant gatekeepers and purse holders.

6.4. Implications for the Evaluation of Frontier Projects

Regarding intellectual distance, the earlier analysis suggests that closest expert evaluations can offer more discerning evaluations than can more distant evaluators. A challenge, however, in implementing expert reviews is that different experts might be needed to evaluate different proposals, introducing evaluator-specific idiosyncratic noise. One possible remedy is to algorithmically correct for individual fixed effects and variation in distance (as we did earlier). Where data are not available, perhaps senior evaluators can more informally make judgments to virtually correct for evaluator differences. The results highlight the limits of aggregating the views of more distant evaluators who individually offer more superficial evaluations. Lesser experts simply cannot “see” what experts can see. It is not clear whether this is a problem that can be solved by averaging and aggregation.

The challenges related to novelty are greater still. Any discounting of novel proposals relative to true quality implies underinvestment in novel proposals. No amount of aggregation and averaging, blinding, or other conventional policies can address this problem. Plausible avenues to address this problem include priming and coaching evaluators to create greater understanding and awareness of resource allocation goals and their own cognitive limits. As our analysis demonstrates, it is also possible to supplement evaluations with statistics providing objective measures of the degree of novelty of a given proposal. Programs geared to providing researchers with less stringent constraints in allocating resources might also play a role in fostering novel innovation (e.g., Manso 2011 ). However, this presumes that innovators themselves may be better able to judge the merits of novel projects than will independent evaluators—something not addressed in this study.

Our findings suggest still other reasons for under-investment in novelty that do not depend on whether evaluations discount novel proposals relative to true expected outcomes. Consider that innovators might, in principle, wish to trade off lower expected mean innovation outcomes for greater ex post variance of outcomes, and possible upside risk ( Fleming 2001 ). However, here we found no evidence whatsoever of greater variance in the ex ante evaluations of novel proposals. This raises the question of whether—in the context of bounded rationality, uncertainty, and ambiguity—evaluators can even perceive the high variance potential of novel proposals ex ante (let alone implement the trade-off).

More broadly, these points regarding bounded rationality in the evaluation of novel proposals raise the question of what evaluators (and innovators) can hope to know, plan for, and anticipate as they pursue novel exploration. How utterly uncertain is such experimentation ex ante? Recent studies of ex post patterns of outcomes with novel innovations suggest that it is productive for foresighted innovators and evaluators not only to promote novel proposals but to promote quite particular kinds of novel proposals (e.g., Uzzi et al. 2013 , Kaplan and Vakili 2015 ). And yet it is far from clear that evaluators (or innovators) can be all that foresighted to steer their innovation in these particular directions. These are questions for future research.

Acknowledgments

The authors are indebted to administrators at an anonymous granting organization and host university for their invaluable support. They also thank executives at Inno-Centive who assisted with platform and information technology design and support. They would also like to acknowledge Thomas Astebro, Michael Bikard, Amy Edmonson, Dan Elfenbein, Alfonso Gambardella, Yigal Gerchak, Shane Greenstein, Ben Hallen, Danielle Li, Lars Bo Jeppesen, Gary Pisano, Anisa Shyti, Eric von Hippel, Keyvan Valkili, and participants at presentations at Bocconi University, Cambridge University, INSEAD, London Business School, National Bureau of Economic Research (NBER), the Roundtable for Engineering Entrepreneurship Research (REER), University of Maryland, and Washington University at St. Louis for useful comments. The authors acknowledge support from the London Business School’s Deloitte Institute on Innovation and Entrepreneurship, London Business School RAMD, the NASA Tournament Laboratory, German Research Foundation [Grant RI 2185/1-1], Harvard Catalyst, and the National Institutes of Health National Center for Advancing Translational Sciences [Grants UL1TR000170 and UL1TR001102S] and National Center for Research Resources [UL1RR025758-02S4]. Eric Lonstein provided excellent research support. Griffin Weber assisted in selecting evaluators. Eric Lin provided access to the PubMed citation data. All errors are the authors’ own.

1 See Li (2015) for an approach using a form of proxy measure for true quality.

2 Sources of bias considered include social category, status and prestige, sex, nationality, language, and relationships between evaluators and researchers.

3 We tested for and found no evidence of statistical differences across the blocks.

4 PubMed includes approximately 185 million MeSH term combinations (from a body of 26,579 unique terms) assigned to some 21 million articles published between 1855 and 2010.

5 Alternative specifications allowing for truncation or for the nonnegative integer nature of the dependent variable do not alter the results.

6 Of those doing research outside of the disease domain, roughly half had and half did not have a coauthor publishing within the domain. We find no differences in evaluations between these subgroups.

7 Alternative measures, such as the simple cosine (not expressed as a percentile), Euclidean distance, and simple counts of overlapping areas, produce similar results.

8 The results do not depend on whether novelty is measured as a share or absolute number of new keyword pairs, triplets, or quadruplets; whether the variable measures departures from the last 10 years or the entire history of the literature on the PubMed database; or whether the model controls for the absolute numbers of keywords.

9 As additional assessments of the possibility that omitted variable bias is the reason for the negative relationship with novelty, we also considered effects of removing controls. When examining all possible combinations of control variables (including the main control vector and in the added controls introduced in Model 2), we find that progressively adding more controls to the model generally produces more negative estimates, not less . For example, dropping control variables altogether produces a far less negative coefficient (−0.25; s.e. = 0.20).

10 This is also the case despite these latter estimates being based on just a subset of authors for whom we observe control variables.

11 Mean EVALUATOR_DISTANCE is 0.13 (i.e., 13th percentile) for closest experts and 0.50 for groups.

12 Even among the top 25 ranks, the mean absolute change in rank positions when pursuing this alternative policy of evaluation by lone experts would have been 23.8 positions (s.d. = 27.0).

13 The rationale is that if reducing noise makes expert evaluations more similar to group averaged rank orderings, then we can presume that it is the noise rather than signal of expert evaluations that leads to the divergence. By contrast, if reducing noise only increases the divergence in ranking, then we can presume that it is the random noise that makes expert evaluations similar to the group averages (and that by reducing noise we are isolating a more meaningful signal).

Contributor Information

Kevin J. Boudreau, London Business School, London NW1 4SA, United Kingdom; and Harvard Business School, Boston, Massachusetts 02163.

Eva C. Guinan, Dana-Farber/Harvard Cancer Center, Boston, Massachusetts 02215.

Karim R. Lakhani, Harvard Business School, Boston, Massachusetts 02163.

Christoph Riedl, D’Amore-McKim School of Business, Northeastern University, Boston, Massachusetts 02115.

• Agrawal A, Catalini C, Goldfarb A. Working paper. Massachusetts Institute of Technology; Cambridge: 2014. Crowdfunding science. [ Google Scholar ]
• Anand P. Foundations of Rational Choice Under Risk. Oxford University Press; Oxford, UK: 1993. [ Google Scholar ]
• Astebro T, Elhedhli S. The effectiveness of simple decision heuristics: Forecasting commercial success for early-stage ventures. Management Sci. 2006; 52 (3):395–409. [ Google Scholar ]
• Bedard J. Expertise in auditing: Myth or reality? Accounting, Organ Soc. 1989; 14 (1):113–131. [ Google Scholar ]
• Berger J. Statistical Decision Theory and Bayesian Analysis. 2. Springer-Verlag; New York: 1985. [ Google Scholar ]
• Blackburn JL, Hakel MD. An examination of sources of peer-review bias. Psych Sci. 2006; 17 (5):378–382. [ PubMed ] [ Google Scholar ]
• Bornmann L, Daniel H-D. The effectiveness of the peer review process: Inter-referee agreement and predictive validity of manuscript refereeing at Angewandte Chemie . Angewandte Chemie Internat Ed. 2008; 47 (38):7173–7178. [ PubMed ] [ Google Scholar ]
• Boudreau K, Lakhani K. “Open” disclosure of innovations, incentives and follow-on reuse: Theory on processes of cumulative innovation and a field experiment in computational biology. Res Policy. 2015; 44 (1):4–19. [ Google Scholar ]
• Brehmer B. In one word: Not from experience. Acta Psych. 1980; 45 (1):223–241. [ Google Scholar ]
• Camerer CF, Johnson EJ. The process-performance paradox in expert judgment: How can experts know so much and predict so badly? In: Ericsson KA, Smith J, editors. Toward a General Theory of Expertise: Prospects and Limits. Cambridge University Press; Cambridge, UK: 1991. pp. 195–217. [ Google Scholar ]
• Campanario JM, Acedo E. Rejecting highly cited papers: The views of scientists who encounter resistance to their discoveries from other scientists. J Amer Soc Inform Sci Tech. 2007; 58 (5):734–743. [ Google Scholar ]
• Chase WG, Simon HA. Perception in chess. Cognitive Psych. 1973; 4 (1):55–81. [ Google Scholar ]
• Chi MTH. Two approaches to the study of experts characteristics. In: Ericsson KA, Charness N, Feltovich PJ, Hoffman RR, editors. The Cambridge Handbook of Expertise and Expert Performance. Cambridge University Press; New York: 2006. pp. 21–30. [ Google Scholar ]
• Chubin DE, Hackett EJ. Peerless Science: Peer Review and US Science Policy. State University of New York Press; Stony Brook: 1990. [ Google Scholar ]
• Cole S, Cole JR, Simon GA. Chance and consensus in peer review. Science. 1981; 214 (4523):881–886. [ PubMed ] [ Google Scholar ]
• Coletti MH, Bleich HL. Medical subject headings used to search the biomedical literature. J Amer Medical Informatics Assoc. 2001; 8 (4):317–323. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Cowan R, David PA, Foray D. The explicit economics of knowledge codification and tacitness. Indust Corporate Change. 2000; 9 (2):211–253. [ Google Scholar ]
• Dasgupta P, David PA. Toward a new economics of science. Res Policy. 1994; 23 (5):487–521. [ Google Scholar ]
• Dosi G. Technological paradigms and technological trajectories. Res Policy. 1982; 11 (3):147–162. [ Google Scholar ]
• Ericsson KA, Smith J, editors. Toward a General Theory of Expertise: Prospects and Limits. Cambridge University Press; Cambridge, UK: 1991. [ Google Scholar ]
• Fitts PM, Posner MI. Learning and Skilled Performance in Human Performance. Brock-Cole; Belmont, CA: 1967. [ Google Scholar ]
• Fleming L. Recombinant uncertainty in technological search. Management Sci. 2001; 47 (1):117–132. [ Google Scholar ]
• Fox CR, Tversky A. Ambiguity aversion and comparative ignorance. Quart J Econom. 1995; 110 (3):585–603. [ Google Scholar ]
• Furman J, Stern S. Climbing atop the shoulders of giants: The impact of institutions on cumulative research. Amer Econom Rev. 2011; 101 (5):1933–1963. [ Google Scholar ]
• Gibbons M, Limoges C, Nowotny H, Schwartzman S, Scott P, Trow M. The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies. Sage; London: 1994. [ Google Scholar ]
• Graves N, Barnett AG, Clarke P. Funding grant proposals for scientific research: Retrospective analysis of scores by members of grant review panel. British Medical J. 2011; 343 :d4797. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Hallen BL. The causes and consequences of the initial network positions of new organizations: From whom do entrepreneurs receive investments? Admin Sci Quart. 2008; 53 (4):685–718. [ Google Scholar ]
• Holland JH, Holyoak KJ, Nisbett RE, Thagard PR. Induction: Processes of Inference, Learning and Discovery. MIT Press; Cambridge, MA: 1986. [ Google Scholar ]
• Jackson JL, Srinivasan M, Rea J, Fletcher KE, Kravitz RL. The validity of peer review in a general medicine journal. PLoS ONE. 2011; 6 (7):e22475. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Jayasinghe UW, Marsh HW, Bond N. A multilevel cross-classified modeling approach to peer review of grant proposals: The effects of assessor and researcher attributes on assessor ratings. J Roy Statist Soc Ser A. 2003; 166 (3):279–300. [ Google Scholar ]
• Johnson EJ. Expertise and decision under uncertainty: Performance and process. In: Chi MTH, Glaser R, Farr MJ, editors. The Nature of Expertise. Lawrence Erlbaum Associates; Hillsdale, NJ: 1988. pp. 209–228. [ Google Scholar ]
• Johnson EJ, Payne JW. Effort and accuracy in choice. Management Sci. 1985; 31 (4):395–414. [ Google Scholar ]
• Johnson EJ, Russo JE. Product familiarity and learning new information. J Consumer Res. 1984; 11 (1):542–550. [ Google Scholar ]
• Johnson PE, Hassebrock F, Duran AS, Moller JH. Multi-method study of clinical judgment. Organ Behav Human Performance. 1982; 30 (2):201–230. [ PubMed ] [ Google Scholar ]
• Jones BF. The burden of knowledge and the “death of the Renaissance man”: Is innovation getting harder? Rev Econom Stud. 2009; 76 (1):283–317. [ Google Scholar ]
• Kahneman D, Slovic P, Tversky A, editors. Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press; Cambridge, UK: 1982. [ PubMed ] [ Google Scholar ]
• Kaplan S, Vakili K. The double-edged sword of recombination in breakthrough innovation. Strategic Management J. 2015; 36 (10):1437–1457. [ Google Scholar ]
• Kravitz RL, Franks P, Feldman MD, Gerrity M, Byrne C, Tierney WM. Editorial peer reviewers? Recommendations at a general medical journal: Are they reliable and do editors care? PLoS ONE. 2010; 5 (4):e10072. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kuhn TS. The Structure of Scientific Revolutions: 50th Anniversary Edition. University of Chicago Press; Chicago: 1962. [ Google Scholar ]
• Lamont M. How Professors Think: Inside the Curious World of Academic Judgment. Harvard University Press; Cambridge, MA: 2009. [ Google Scholar ]
• Langfeldt L. The policy challenges of peer review: Managing bias, conflict of interests and interdisciplinary assessments. Res Evaluation. 2006; 15 (1):31–41. [ Google Scholar ]
• Lee CJ. A Kuhnian critique of psychometric research on peer review. Philos Sci. 2012; 79 (5):859–870. [ Google Scholar ]
• Lee CJ, Sugimoto CR, Zhang G, Cronin B. Bias in peer review. J Amer Soc Inform Sci Tech. 2013; 64 (1):2–17. [ Google Scholar ]
• Li D. HBS Working Paper 16-053. Harvard Business School; Boston: 2015. Expertise vs. bias in evaluation: Evidence from the NIH. [ Google Scholar ]
• Manso G. Motivating innovation. J Finance. 2011; 66 (5):1823–1860. [ Google Scholar ]
• March JG. Exploration and exploitation in organizational learning. Organ Sci. 1991; 2 (1):71–87. [ Google Scholar ]
• Marsh HW, Jayasinghe UW, Bond NW. Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability. Amer Psychologist. 2008; 63 (3):160–168. [ PubMed ] [ Google Scholar ]
• McNutt R, Evans A, Fletcher R, Fletcher S. The effects of blinding on the quality of peer review: A randomized trial. J Amer Medical Assoc. 1990; 263 (10):1371–1376. [ PubMed ] [ Google Scholar ]
• Merton RK. The Matthew effect in science. Science. 1968; 159 (3810):56–63. [ PubMed ] [ Google Scholar ]
• Mollick E, Nanda R. Wisdom or madness? Comparing crowds with expert evaluation in funding the arts. Management Sci. 2015 ePub ahead of print September 11, http://dx.doi.org/10.1287/mnsc.2015.2207 .
• Murray F, O’Mahony S. Exploring the foundations of cumulative innovation: Implications for organization science. Organ Sci. 2007; 18 (6):1006–1021. [ Google Scholar ]
• Newell A, Simon HA. Human Problem Solving. Prentice-Hall; Englewood Cliffs, NJ: 1972. [ Google Scholar ]
• Piezunka H, Dahlander L. Distant search, narrow attention: How crowding alters organization’s filtering of suggestions in crowdsourcing. Acad Management J. 2014; 58 (3):856–880. [ Google Scholar ]
• Romer P. Endogenous technological change. J Political Econom. 1990; 98 (5 Part 2):S71–S102. [ Google Scholar ]
• Rothwell PM, Martyn CN. Reproducibility of peer review in clinical neuroscience—Is agreement between reviewers any greater than would be expected by chance alone? Brain. 2000; 123 (9):1964–1969. [ PubMed ] [ Google Scholar ]
• Sahal D. Technological guideposts and innovation avenues. Res Policy. 1985; 14 (2):61–82. [ Google Scholar ]
• Sandström U, Hallsten M. Persistent nepotism in peer-review. Scientometrics. 2008; 74 (2):175–189. [ Google Scholar ]
• Stephan PE. The economics of science. J Econom Literature. 1996; 34 (3):1199–1235. [ Google Scholar ]
• Stephan PE. How Economics Shapes Science. Harvard University Press; Cambridge, MA: 2012. [ Google Scholar ]
• Sternberg RJ. Costs of expertise. In: Ericsson KA, editor. The Road to Excellence: The Acquisition of Expert Performance in the Arts and Sciences, Sports, and Games. Lawrence Erlbaum Associates; Hillsdale, NJ: 1996. pp. 347–354. [ Google Scholar ]
• Stevens GA, Burley J. 3,000 raw ideas = 1 commercial success! Res Tech Management. 1997; 40 (3):16–27. [ Google Scholar ]
• Sugimoto CR, Cronin B. Citation gamesmanship: Testing for evidence of ego bias in peer review. Scientometrics. 2013; 95 (3):851–862. [ Google Scholar ]
• Travis GDL, Collins HM. New light on old boys: Cognitive and institutional particularism in the peer review system. Sci, Tech, Human Values. 1991; 16 (3):322–341. [ Google Scholar ]
• Utterback J, Abernathy W. A dynamic model of product and process innovation. Omega. 1975; 3 (6):639–656. [ Google Scholar ]
• Uzzi B, Mukherjee S, Stringer M, Jones B. Atypical combinations and scientific impact. Science. 2013; 342 (6157):468–472. [ PubMed ] [ Google Scholar ]
• van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers? Recommendations: A randomized trial. British Medical J. 1999; 318 (7175):23–27. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Weitzman ML. Recombinant growth. Quart J Econom. 1998; 113 (2):331–360. [ Google Scholar ]
• Williams H. Intellectual property rights and innovation: Evidence from the human genome. J Political Econom. 2013; 121 (1):1–27. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Xie Y, Killewald AA. Is American science in decline? Harvard University Press; Cambridge, MA: 2012. [ Google Scholar ]

The New Frontier of Microbiome Science: Computational Challenges and Solutions

The microbiome refers to the whole sum of microorganisms in a particular environment, such as the collective sum of gut bacteria in a human being. Microbiome research is a new frontier of scientific exploration. Studies that use big data technology to examine whole genomes of hundreds of organisms simultaneously represent a field called metagenomics. As this field matures, scientists are increasingly recognizing the need for sophisticated tools and technologies to decipher the complexities hidden within these microbial ecosystems.

To that end, on April 2, Mihai Pop , a professor in the Department of Computer Science and the director of the Institute for Advanced Computer Studies at the University of Maryland, gave a talk on the analytical challenges of microbiome science and how they can be combated by computational methods. The talk focused on the pivotal role of computational tools in unraveling the secrets of microbiomes and addressing the challenges associated with analyzing the vast datasets generated by these studies.

A key focus of metagenomics is the taxonomic classification of different microbes. The primary method for organizing and classifying microbes is comparing them to a database of known organism sequences. These similarity-based techniques are especially effective when the organisms in the sample are well represented in the database. Pop mentioned one of the most common similarity search methods used to classify microorganisms, the Basic Local Alignment Search Tool (BLAST). However, BLAST often misidentifies the closest neighbor to the microorganism of interest; the “most similar” organism according to BLAST may not actually be the most closely related.

“How can we find what’s the real [closest hit] if there is a hit? The E-value is misleading,” Pop explained during the talk, suggesting that BLAST may not always accurately identify the most similar organism to the microbiome of interest.

The E-values Pop mentioned refer to parameters in BLAST that describe the number of hits one can “expect” to see by chance when searching a database of a particular size. Pop also emphasized how many of these problems were only discovered years after BLAST integrated into common use.

“These are things we found out 20, 30, 40 years after [the computational tool] was written ... even though something has been used for many, many years, there [are] still things to learn about it,” Pop explained.

One of the other main challenges Pop highlighted is how the structure of biological databases affects scientists’ ability to reliably reveal insights on the microbiome. Reference databases are not all-encompassing. Many microorganisms cannot be cultured in labs, and a large proportion of those that can have not been sequenced or added to these reference databases. Consequently, not all environmental organisms are included in the sequence database, which limits the accuracy of similarity-based methods.

These problems are further compounded by the lack of contiguous information available in most sequencing datasets. Many sequencing analyses have to begin by joining together many sequence fragments and stitching together a whole related sequence. Assembling the sequencing data is also an unstandardized process, as new technologies used for assembling genomes are constantly being developed. These limitations can impede researchers' ability to derive meaningful insights and connections from microbiome datasets, because it substantially limits precision and decreases the accuracy of reference databases.

Pop then transitioned to discussing algorithms and software approaches to sequence similarity. Many current software used in classification employ the most recent common ancestor (MRCA) method. MRCA provides an annotation (marking of a specific feature of the DNA sequence) at the broadest taxonomic class that encompasses all of the possible markings in a sequence. However, this means that different types of software that use MRCA only make a few classifications at the genus or species level, meaning that stronger relationships between two microbes cannot be determined at the family, class or phylum level.

To address this challenge, Pop shared efforts from his own lab to develop advanced computational tools tailored specifically for microbiome analyses. He specifically focused on the Ambiguous Taxonomy eLucidation by Apportionment of Sequences (ATLAS). ATLAS is a data-driven database partitioning method, which aims to divide a large dataset into smaller, more easily analyzable datasets. ATLAS groups sequences into biologically meaningful partitions by querying the sequence against a reference database and then identifying and clustering hits that are considered significant. ATLAS also represents a shift away from the MRCA method.

As the talk concluded, Pop emphasized the critical need for interdisciplinary collaboration to advance microbiome research. Integrating expertise from fields such as biology, computer science and statistics is essential for developing innovative solutions to microbiome-related challenges. This interdisciplinary approach allows researchers to harness the power of computational tools to extract meaningful patterns and associations from microbiome datasets.

—This article by Shreya Tiwari was originally published in The News-Letter, a student publication at John Hopkins University.

The Department welcomes comments, suggestions and corrections.  Send email to editor [-at-] cs [dot] umd [dot] edu .

• AI Research: The New Frontier
• Georgia State Research Magazine , Science & Technology

AI RESEARCH: THE NEW FRONTIER

Written by Noelle Toumey Reetz

Researchers at Georgia State are working to address critical issues like equity and fairness and accelerate advances in medical diagnoses in the world of AI—all while preparing for a seismic shift in how we work and live.

From education to commerce and medicine, research is playing an increasingly important role in addressing the new societal and technological changes underway.

Experts from across Georgia State are studying the potential impacts as well as expected leaps forward in medical diagnoses and treatment. They are also preparing the next generation of experts for the technical and sociological changes that will drive the workforce of the future.

TRANSFORMING THE WORKFORCE

“These technologies are providing individuals, teams and organizations the potential to reconceive what we do, how we do it, how we collaborate, how we create products and even how we live our lives,” says Arun Rai , Regents’ Professor and Howard S. Starks Distinguished Chair and Director of the Center for Digital Innovation at Georgia State’s Robinson College of Business .

Rai has been working at the forefront of digital innovation for decades and — in addition to teaching and conducting research — his work is focused on helping businesses make sense of rapid changes in digital technologies. His research aims to determine how AI can bring benefits and minimize risks across industry contexts, from education to health care to logistics and supply chains, to high tech and consumer goods.

“What we are seeing is a much more pronounced shift towards AI affecting virtually every sector of our economy. It’s affecting all of our major industries and people’s living and working. But I think where one gets a more granular understanding, is to shift the discussion from what it’s doing at the level of jobs to what it is doing at the level of skills,” explains Rai.

He says that the discussion about how AI will affect the workforce is not static: certain skills will be augmented by AI, certain skills will be displaced, while new skills will be needed for existing and new jobs. Rai says partnerships among industry, academia and government will be crucial to upskilling and reskilling the workforce alongside the rapid development of the technology.

MEDICAL ADVANCES

At Georgia State, researchers are using AI and machine learning to study the deepest recesses of the human brain.

Newly published research is finding new ways to produce earlier diagnoses of mental illness and other diseases, including schizophrenia and epilepsy. The developments offer solutions in real-world settings that can aid both patients and medical professionals.

“AI is poised to make a major impact in expanding our understanding of the brain and also in the way we make decisions about how to treat or prevent illness,” says Vince Calhoun , Distinguished University Professor and director of the Tri-Institutional University Center for Translational Research in Neuroimaging and Data Science ( TReNDS Center ). “The main strength of AI is in synthesizing large amounts of data to help us maximize the information we have available, too much for a person to put together. While it is still early, AI has already helped us to improve our ability to make reliable predictions and to suggest the most informative aspects of the data.”

Calhoun says researchers are harnessing these advances in technology to both better understand and visualize the impact of mental illness on the brain and to make better diagnoses that will lead to more effective treatment options and could eventually allow doctors to stave off some diseases altogether.

A recent study published in JAMA Neurology finds that artificial intelligence models can be trained to interpret routine clinical electroencephalograms (EEGs) with the accuracy equivalent to that of human experts. It is known as Automated Electroencephalogram Interpretation.

Sergey Plis , an associate professor of Computer Science at Georgia State and director of the machine learning core at TReNDS, who worked on the study along with Dr. Calhoun and an international collaborative team of researchers and industry partners, says their findings represent a leap forward toward harnessing AI for clinical use. But these principles and strategies can be expanded to countless applications, including climate change models or space exploration. The same approach could also be used for wide-ranging medical applications including detecting tumors or other conditions.

“What the model can do is greatly reduce the amount of time highly trained clinicians are spending on clear cases, and bring up the possibly controversial cases where radiologists can spend their time focused on red flags instead of sifting through data,” explained Plis.

Recently, another international team of researchers at TReNDS was able to identify brain pattern changes connected to schizophrenia risk in children with subthreshold symptoms using a new hybrid, data-driven method in a study published in the proceedings of the National Academy of Sciences.

In yet another recent study published in Nature Scientific Reports, scientists at the TReNDS Center built a sophisticated computer program that was able to comb through massive amounts of brain imaging data and discover novel patterns differentially linked to autism spectrum, Alzheimer’s disease and schizophrenia. The brain imaging data came from scans using functional magnetic resonance imaging (fMRI), which measures dynamic brain activity by detecting tiny changes in blood flow.

Plis says, like automated mechanical labor during the industrial revolution, artificial intelligence will help to automate cognitive labor.

“There are so many applications where AI can be used, but it’s very hard to predict where it will go,” says Plis. “I think we will automate a lot of tasks that require cognitive load, but automating some tasks will be much harder than we thought initially.”

WATCHING THE WATCHERS:

Recent research by Georgia State Criminologists find that AI-powered facial recognition can lead to increased racial profiling.

Facial Recognition Technology (FRT) is an artificial intelligence–powered technology that tries to confirm the identity of a person from an image.

The study by Georgia State researchers finds that law enforcement agencies that use automated facial recognition disproportionately arrest Black people.

“We believe this results from factors that include the lack of Black faces in the algorithms’ training data sets, a belief that these programs are infallible and a tendency of officers’ own biases to magnify these issues,” says one of the study’s authors Thad Johnson .

Johnson is a former police officer and teaches criminology at Georgia State. He is one of many safety advocates that while acknowledging technology’s potential to improve public safety, is calling for enforceable safeguards to prevent unconstitutional overreaches from racial profiling and false arrests.

As machine learning and artificial intelligence evolve, both curriculums and tools for student success at GSU are developing too.

One example is Georgia State’s AI-enhanced text messaging tool, “Pounce.” The chatbot is nationally recognized for its success at improving student progress and retention rates. Research finds that student performance jumps when classes employ the chatbot to keep them connected. Students get direct text messages about class assignments, academic support and course content and the tool has proved transformative for student success by reducing the average time it takes to earn a degree by almost a full semester.

“We are partnering with MIT (The Massachusetts Institute of Technology), supported by grants by the Axim Collaborative , to design and evaluate an AI tutor for equitable student success in programming courses,” says Rai. “Leveraging generative AI, we are developing a solution for personalized anytime-anywhere tutoring for students which we will evaluate with respect to equitable student learning, internship opportunities and career aspiration.”

Established in spring 2020, Georgia State’s Inspire Center is one of just a handful in the U.S. designated by the National Security Agency and the Department of Homeland Security as a National Center of Academic Excellence in Cyber Defense Research (CAE-R) and a National Center of Academic Excellence in Cyber Defense Education (CAE-CDE). Georgia State is the only university in Georgia to have received both designations.

The Cyber CORPS scholarship service offers nearly $4 million in funding annually that essentially provides a scholarship to students who are studying cybersecurity programs. The scholarships are not limited to computer science – they are also available to students studying information systems. Then after graduation, in exchange for the scholarship, the students work for a federal agency.

AI AND ROBOTICS

Research underway at Georgia State is leveraging the power of artificial intelligence and robotics. Dr. Jonathan Shihao Ji is an associate professor of Computer Science and Director of the Intelligent Systems Lab at Georgia State.  A recent grant from the Department of Defense brought ‘Spot’ – a four-legged, dog-like robot – to Georgia State from Boston Dynamics.

“The main applications of our AI and Robotics research are for search and rescue, facilities maintenance, and emergence response, where it’s unsafe to deploy human investigators for the tasks due to unfriendly, hazardous or even hostile situations,” says Dr. Ji. “In such cases, we could deploy a robot (e.g., Spot) for tasking.”

Dr. Ji says Spot has exceptional mobility, allowing it to traverse a wide range of terrains, including rocky and uneven surfaces, stairs or snow. Spot is also equipped with a variety of sensors, such as RGB-D cameras, Infrared, and Lidar, to perceive the surrounding environment. The research taking place at Georgia State involves developing AI models, specifically computer vision and natural language processing algorithms, to enable Spot for things like navigation, object detection and manipulation, with a natural language interface for human-robot interaction. That means the user can direct Spot with natural language instructions in real-time. This is one of the projects that will be enhanced with the addition of a 5-year, $10 million grant from the Department of Defense.

SAFETY AND EQUITABLE AI

Researchers from units across campus are working at the intersection of cybersecurity and privacy. Raj Sunderraman is the Associate Chair for the Computer Science department who has been working at the leading edge of computer science for more than 20 years.

“We think of ‘Trustworthy AI’ as systems that are inherently accountable, fair, ethical, transparent, reliable, safe, unbiased, secure and privacy protecting,” explains Sunderraman.

He says there are numerous problems that can still arise with AI that experts are working to address.

“AI systems that are deemed untrustworthy include those which exhibit bias towards certain populations, reveal personal data to the public, don’t provide explanations on why certain decisions were made by the system, do psychological or physical harm to the end user, or are not accountable for their actions,” he says.

Most people have been relating with AI for years without even realizing it, such as when you apply for a mortgage or credit card online or get a quote for insurance.

When data is privacy sensitive­ – for example, financial or healthcare-related information – regulations like HIPPA prevent companies from sharing data or overseeing how the information is handled. Researchers are working to develop techniques and tools to safeguard encrypted data and enable the use of machine learning as a service platform without compromising privacy.

FINDING A WAY FORWARD

As AI becomes more widely integrated, there are growing calls for both regulation and fairness. Safety advocates question who has access to all the data, where the data comes from and where these developments might lead.

Experts say the advancement of AI is reminiscent of the onset of the internet, there were few guardrails for security or privacy. But, because we are still in the early days, there is a window of opportunity to address some of these issues in a more fundamental way.

A new 5-year, $10 million grant will fund a Department of Defense Center of Excellence in Advanced Computing and Software (COE-ACS) at Georgia State. The principal investigator, Associate Professor of Computer Science Jonathan Ji , says the center is an interdisciplinary research alliance led by Georgia State in collaboration with Duke University and partners from the U.S. Army Research Laboratory.

The research agenda is driven by a collaborative cohort of researchers drawn from multiple disciplines to solve the most critical problems in AI and robotics, particularly, human-robot interaction, VR/AR (Virtual Reality/Augmented Reality), edge computing and trustworthy AI. Education and outreach are also critical components of the center, which will train and employ 12 Ph.D. students and 100 undergraduate students over the life of the initial grant.

Recently the U.S. Senate Judiciary Committee held the first ever hearing on regulating AI technology, and the United Nations also held the first-ever summit on AI regulation where Security Council members urged regulation of the technology to stave off possible misuses and stay ahead of what comes from these rapid advancements.

“We are still working to learn how a lot of AI systems work, and that’s the scary part that if you don’t address these issues, it’s likely to become much more difficult down the road,” says Sunderraman.

Those who are working to adapt to these rapid changes say they see positive benefits ahead along with the risks, including the democratization of information models.

“We are examining whether open data initiatives can actually end up democratizing these AI innovation processes. Right now, the platform companies have a tremendous advantage with data, like Google for example,” explains Rai. “So, how can we leverage these technologies in smart and creative ways so we can achieve inclusive prosperity and democratize innovation, so we don’t end up with winners and losers.”

As it turns out, AI isn’t the only one learning. We are too.

Illustrations by Tara Jacoby

RELATED CONTENT

Leave a reply.

Your email address will not be published. Required fields are marked *

Related Posts

On the Front Lines

A Path to Resiliency

Teach Your Teachers Well

The Power of Trees

The Antioxidant Myth

Flower Power

Crime and Punishment

The Air at the Daycare

When Bacteria Fight Back

Charting a New Course for Childhood Cancer Survivors

School of Medicine Department of Neurology

• Enter keyword Search

Department of Neurology

At the frontier & beyond: ms research from the hill.

Join the OHSU MS Center in-person or online for this year’s half-day conference to explore research in multiple sclerosis and future directions.

This year’s conference topics include:

• Lipoic acid, gait and balance
• Visual outcomes in MS research
• Pathways in remyelination research
• Strategies to promote myelin repair

The conference concludes with the popular Q&A panel with speakers answering audience questions. 

Who should attend : People living with MS and those who support them are invited to attend.

Registration

Free to participate in-person or online. Registration is required to attend.

Registration coming soon.

8:30 - 9a.m.    Registration and Continental Breakfast

9 - 9:10a.m.  Welcome Vijayshree Yadav, M.D., M.C.R., FANA, FAAN OHSU Department of Neurology and Portland VA MS Center of Excellence West Lindsey Wooliscroft, M.D., M.Sc., M.C.R. OHSU Department of Neurology and Portland VA MS Center of Excellence West

9:10 - 9:40a.m.    Lipoic Acid: Does it Treat MS? Rebecca Spain, M.D., M.S.P.H., FAAN OHSU Department of Neurology and Portland VA MS Center of Excellence West

9:40 - 10:10a.m.  Gait and Balance in MS Michelle Cameron, M.D., PT, M.C.R. OHSU Department of Neurology and Portland VA MS Center of Excellence West

10:10 - 10:25a.m.  Break and Exhibits

10:25 - 10:55a.m.  Visual Outcomes in MS Research Elizabeth Silbermann, M.D. OHSU Department of Neurology and Portland VA MS Center of Excellence West

10:55 - 11:25a.m.  Pathways in Remyelination Research Ben Emery, Ph.D. The Jungers Center, OHSU

11:25 - 11:55a.m.  Strategies to Promote Myelin Repair Lindsey Wooliscroft, M.D., M.Sc., M.C.R. OHSU Department of Neurology and Portland VA MS Center of Excellence West

11:55a.m. - 12:10p.m.  Break and Exhibits

12:10 - 1:10p.m.  Q & A Panel and Lunch

In collaboration with

VA MS Center of Excellence West, Portland VA Health Care System

Virtual exhibits

To learn more about exhibiting at this conference, please contact Dawn at [email protected]

Space Station

Station Orbits into Eclipse, Crew Works Research and Spacesuits

The International Space Station soared into the Moon’s shadow during the solar eclipse on Monday afternoon. The Expedition 71 crew members had an opportunity to view the shadow at the end of their workday filled with cargo transfers, spacesuit maintenance, and microgravity research.

The windows on the cupola, the orbital outpost’s “window to the world,” were open and NASA Flight Engineers Matthew Dominick and Jeanette Epps were inside photographing and videotaping the Moon’s shadow on Earth, or umbra, beneath them. They were orbiting 260 miles above southeastern Canada as the Moon’s umbra was moving from New York state into Newfoundland.

The space station experienced a totality of about 90% during its flyover period. Views of the solar eclipse itself, the Moon orbiting directly between the sun and the Earth, were only accessible through a pair of windows in the space station’s Roscosmos segment which may not have been accessible due to cargo constraints.

Before the eclipse activities began on Monday, Dominick worked on orbital plumbing, serviced a pair of science freezers and swapped cargo in and out of the SpaceX Dragon spacecraft. Dominick then joined NASA astronaut Mike Barratt inspecting spacesuit tethers and organizing spacewalking tools.

Epps installed a small satellite orbital deployer inside the Kibo laboratory module ’s airlock and also participated in the Dragon cargo work. NASA Flight Engineer Tracy C. Dyson assisted Epps with the small satellite installations and cargo transfers. Dyson also reviewed operations with the BioFabrication Facility and prepared research hardware for an upcoming session to print cardiac tissue cell samples.

Station Commander Oleg Kononenko spent Monday on inspection tasks in the aft end of the Zvezda service module and Progress 87 resupply ship. Flight Engineer Nikolai Chub focused his attention on electronics and ventilation maintenance. Chub also spent a few moments assisting Flight Engineer Alexander Grebenkin as he attached sensors to himself measuring his heart activity for a long-running Roscosmos space cardiac investigation. He later turned on an ultrasound device and scanned surfaces inside Zvezda.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.