research papers on mobile robotics

A review of the challenges in mobile manipulation: systems design and RoboCup challenges

Recent developments with a special focus on the RoboCup

Ein Überblick über aktuelle Herausforderungen im Bereich der mobilen Manipulation: Systementwurf, Regelung und Sensorik. Aktuelle Entwicklungen mit besonderem Fokus auf die Herausforderungen des RoboCup

• Originalarbeit
• Open access
• Published: 03 September 2020
• Volume 137 , pages 297–308, ( 2020 )

Cite this article

You have full access to this open access article

• Martin Sereinig 1 ,
• Wolfgang Werth 2 &
• Lisa-Marie Faller   ORCID: orcid.org/0000-0001-8434-1440 2  

8233 Accesses

12 Citations

Explore all metrics

Mobile robotics is already well established in today’s production lines. Navigation, control and perception for mobile robots are vivid fields of research fostering advances in Industry 4.0. In order to increase the flexibility of such mobile platforms, it is also common practice to add serial manipulator arms to their yielding systems with nine degrees of freedom and more. These platforms are not limited to industry but are supportive in various field such as service, assistance, teleoperation and also rehabilitation. Due to the operation of such increasingly complex systems in less structured and dynamic environments - often in close contact with humans - more demanding challenges evolve in terms of systems design, control and sensors. These challenges are also reflected in the various RoboCup leagues. In this paper, we discuss state-of-the-art developments in mobile manipulation using developments and work done in the context of the RoboCup competition as design examples. Additionally, we elaborate on the recent challenges of the RoboCup Rescue League as well as on the RoboCup@Work League.

Zusammenfassung

Die mobile Robotik ist im Bereich unserer heutigen Produktionslinien bereits eine etablierte Technologie. Wichtige Teilbereiche wie Navigation, Regelung und Wahrnehmung für mobile Roboter stellen inzwischen stark beforschte Themen unserer Forschungslandschaft dar. In aktuellen Systemkonzepten der mobilen Robotik im Kontext von Industrie 4.0., also zur Erweiterung der Automatisierung als auch der Flexibilität solcher mobilen Plattformen, werden heutzutage vermehrt auch serielle Roboterarme zur Manipulation auf mobilen Plattformen eingesetzt. Die resultierenden Systeme besitzen in der Folge neun Freiheitsgrade und mehr. Im Allgemeinen beschränkt sich der Einsatz solcher Systeme nicht nur auf die Industrie, sodass sie in den Bereichen Dienstleistung, Assistenz, Teleoperation und auch Rehabilitation allgegenwärtig sind. Aufgrund des Einsatzes solcher zunehmend komplexen Systeme in weniger strukturierten und dynamischen Umgebungen - oft in engem Kontakt mit Menschen - entwickeln sich anspruchsvollere Herausforderungen in Bezug auf das Systemdesign, die Regelung und auch die Sensorik. Diese Herausforderungen werden weiterhin in den unterschiedlichen Ligen des RoboCup abgebildet. In dieser Arbeit vermitteln wir einen Einblick in aktuelle Forschungsarbeiten im Bereich mobiler Manipulatoren und stellen auch die entsprechenden Problemstellungen im Kontext des RoboCup-Wettbewerbs anhand der RoboCup-Rescue- sowie der RoboCup@Work-Ligen näher dar.

Similar content being viewed by others

Robotics in Healthcare: A Survey

David Silvera-Tawil

Systematic Review on Wearable Lower Extremity Robotic Exoskeletons for Assisted Locomotion

Shuang Qiu, Zhongcai Pei, … Zhiyong Tang

A Review on Lower Limb Rehabilitation Exoskeleton Robots

Di Shi, Wuxiang Zhang, … Xilun Ding

Avoid common mistakes on your manuscript.

1 Introduction

Historically, humans often tried to construct machines which would resemble living beings, either human-like or closer to certain animals. While these approaches have been rather artistic, later the upsurge of industrial robotics began with low Degree-Of-Freedom (DOF) manipulator arms such as the PUMA 560 by Unimation. The idea behind the development of these early machines was to alleviate certain fabrication tasks for human operators. Traditionally, these are dull, dirty and dangerous tasks. While the substitution of human workers in such production environments with the aforementioned context has already been achieved to a great extend, novel fields of robot usage arise. These fields are no longer easily covered by pre-programming routines into robot arms. These novel robot tasks include dynamic operation and movement in flexible production lines, human assistance in service and medical areas, in rehabilitation as well as elderly care and rescue (compare Fig.  1 ). These tasks require mobility as well as manipulation capabilities which former robotic systems would not provide. New approaches in path-planning and control of these high DOF systems are subject of current research in the same way as are human-robot-interaction concepts and new physical sensors which provide robots with sensing capabilties often more sophisticated than those of human beings. It is no longer possible to draw a line clearly distinguishing between the research areas involved, e.g., path-planning, motion control, manipulator and grasp control.

Mobile manipulator for rescue purposes, developed and built at Carinthia University of Applied Sciences (CUAS) by the RoboCup Rescue Team CUAS-RRR

In the following, we give an insight into recent developments with respect to the hardware construction as well as motion, control and sensing concepts. First, we show construction concepts for mobile manipulator platforms. Then we outline the necessary physical capabilties (moving, manipulation and grasping, sensing, interfacing and interacting) of the same. Next, we discuss the systems’ intelligence enabling them to understand humans and the environment, move smartly and do the right thing.

Then we focus on the RoboCup, which is established to publicly promote robotics research by posing forefront challenges in the field, which, at the same time, should make significant social impact as well. As such, it is the optimal setting for active groups to present new ideas and recent developments. Consequently, we include this here with a special emphasis on rescue (RoboCup Rescue) and industrial employment (RoboCup@Work) of mobile manipulators.

2 Physical capabilities (hardware)

The combination of a mobile robot platform with a manipulator (in combination called mobile manipulator) quickly results in having a DOF equal or greater than nine and must therefore be described as robot system with kinematic redundancy. Modern mobile manipulators are often equipped with collaborative manipulation arms. This type of manipulation arms is specially designed to work with human workers in a collaborative manner. With some additional restrictions in terms of speed, force and torque of the mobile manipulator these robots can be gathered under the term collaborative mobile manipulators (see ISO 15066 and ISO 10218). Figure  5 shows an example of a collaborative mobile manipulator with a degree of freedom of nine which can be used in an industrial environment. In Fig.  1 , a mobile rescue robot with a DOF of 12 is shown which will be further described in the following as a design example for the RoboCup Rescue League. In [ 10 ] an overview of mobile manipulators used in industrial environments is given. Different search and rescue robots are described in [ 21 , 95 ] as well as in [ 11 ] (Part F, Sect. 50).

The method to move a mobile robot platform depends on the main purpose of use, mostly these mobile robots are classified into wheel type, track type and leg type. Here, two different driving methods (wheel type and tracked type) including their advantages and disadvantages are described.

2.1.1 Tracked vehicle with additional linkages

When it comes to tasks in difficult unstructured environments (e.g. damaged buildings or collapsed factories) where different obstacles prevent safe rapid movement, tracked vehicles have advantages in terms of stable movement and system robustness. Additional linkages with ground adaption (often called flipper or flipper tracks) are used to overcome obstacles and to handle uneven ground. The mobile rescue robot T.R.U.D.I. (Third Robot for Urban Disaster Intervention) developed and built at CUAS [ 38 ] (Fig.  1 ) uses four active flipper tracks to extend its movement capabilities. Also, a special gearbox was developed which includes all mechanical parts to move the main tracks, flipper tracks and to rotate the flipper tracks with three electric brushless DC (BLDC) motors for each flipper/track combination. The drive train is realized with four of these gearboxes, one on each corner of the chassis.

Figure  2 shows a schematic drawing of the mobile base including main tracks and flippers ( \(f_{1} \dots f_{4}\) ). This mobile platform is considered as non-holonomic system where the velocity in the plain is constrained while the position is not. The system can only move in body x-direction with velocity \(v_{R}\) and rotate around the body z-axis with angular velocity \(\dot{\phi }_{R}\) .

Schematic drawing of the mobile rescue robot T.R.U.D.I

Movement in the plane is achieved by the two main tracks and follows the rules of a differential drive mechanism where the forward velocity kinematics for a rotation with \(\dot{\phi _{R}}\) around the body fixed z-axis can be denoted as

where \(v_{i}\) are the wheel circumferential velocities calculated as \(v_{1}=\omega _{1} R\) and \(v_{2}=\omega _{2} R\) , \(R\) is the wheel radius and \(\omega _{i}\) is the corresponding rotational velocity of the sprocket, \(b\) means the wheelbase. The linear speed along the body fixed x-axis \(v_{R}\) matches to the circumferential velocities of both main tracks if they are equal. The inverse kinematics to control the tracks’ movement regarding the desired body velocities \(v_{R}\) and \(\phi _{R}\) , whereby \(v_{R}\) is the desired velocity in body x direction, can be described by

The speed of the flipper tracks for each side (flipper \(f_{1}\) , \(f_{3}\) on the left and \(f_{2}\) , \(f_{4}\) on the right hand side) is due to the mechanical construction directly linked to the speed of the corresponding main track. Due to the fact that the sprocket of the flipper tracks and the sprocket of the main tracks have the same diameter and both tracks are used with the same velocity it can be combined as one big track (pointed out in Fig.  2 , market as red rectangle). Figure  3 shows a view of the backside of the mobile manipulator T.R.U.D.I. where this combined track can be seen in detail. The for simplicity the slip problem of tracked vehicles as described in [ 23 ] and [ 57 ] was neglected.

Mobile rescue robot T.R.U.D.I. in a test lane at the RoboCup WorldCup 2017

The angle of the flippers \(\phi _{fi}\) with \(i=1\dots 4\) of each flipper can be controlled individually. Due to those additional linkage structures, the mobile platform extends its DOF in position to five. With a certain combination of flipper rotations, the platform can be moved as shown in the kinematic chain of Fig.  1 a.

The joints \(P_{1}\) and \(P_{2}\) can be assigned to the differential drive system, the joints \(P_{3}\) and \(P_{4}\) can swivel the whole platform around \(x_{R}\) and \(y_{R}\) axis. A movement of all four flippers downwards by the same angle can be used to fulfill an elevation of the whole system and this translational movement in \(z_{R}\) direction can be seen as joint \(P_{5}\) as also shown in Fig.  2 .

2.1.2 Omnidirectional mecanum wheels

The requirements for the movement capabilities of an mobile robot used in industrial environment are totally different compared to the scenario described before. The floor in factories is commonly rather smooth, obstacle-free and free of steps. Consequently, these environments can be considered structured quasi well-known. This brings levitations to the construction of the drive train of mobile platforms and simplifications in autonomous navigation and path planing of mobile manipulators. Taking the advantage of the well-known environment into account, often holonomous mobile platforms which can move in all directions (omnidirectional) in the plane, are used.

The usage of four actuated Mecanum wheels ensures a linear velocity in \(x_{R}\) and \(y_{R}\) direction as well as an angular velocity around the \(z_{R}\) axis as shown in Fig.  4 . That is why such mobile platforms with omnidirectional drive mechanism (Fig.  5 ) can be represented using three DOFs. In the kinematic chain of the mobile manipulator KAIROS (Fig.  5 a), the joints \(P_{1}\) , \(P_{2}\) and \(P_{3}\) represent these DOFs.

Schematic drawing of the mobile platform SUMMIT XL-Steel used as base for the mobile manipulator KAIROS

Mobile manipulator KAIROS for industrial usage, build by the company Robotnik and developed by the working group MEKT at the Campus Technik Lienz in Austria (Leopold Franz University Innsbruck)

Detailed explanation of Mecanum wheels, as well as modeling and motion control algorithms of Mecanum wheel driven mobile manipulators, are given by Röhring et al. in [ 72 ], by Lin et al. in [ 51 ] and by Taheri et al. in [ 81 ]. The simplified inverse kinematics on velocity level is given by

with \(\omega _{i}\) the rotational wheel velocity of the i-th wheel, \(l_{x}\) and \(l_{y}\) are the half robot lengths and widths and \(R\) is the wheel radius. For an autonomous exploration, or path following, the velocities in the robot frame can be transformed into the world frame:

with \(\dot{\mathbf{x}}_{W}=[\dot{x}_{W},\;\dot{y}_{W},\;\dot{\theta }_{R}]^{T}\) , \(\dot{\mathbf{x}}_{R}=[\dot{x}_{R},\;\dot{y}_{R},\;\dot{\theta }_{R}]^{T}\) and

2.2 Manipulating and grasping

Manipulator arms to be mounted on moving robotic platforms need to exhibit a high ratio of payload compared to their own weight, this has been achieved in some of the recent manipulators such as KUKA LWR IIWA [ 8 ], the TM-series by Techman [ 82 ], or the Universal Robots Cobot-series [ 71 ]. Research approaches in this context show unified design for lightweight robot arms [ 96 ] and also more specific work in the fields of medicine, e.g. the da Vinci system (compare [ 44 ]), for the employment in space, e.g. the NASA ROBONAUT [ 6 ] or JPL arms (compare, e.g., [ 12 ]), or anti-terror missions such as the devices developed by the Forrest Miller and iRobot company (see also, e.g. [ 58 ]).

With respect to industry, manipulation and grasping employed in robotic devices, exploits a rather low degree of dexterity, still most of the systems present in production are used for monotonous tasks which do not require much grasping and manipulation intelligence (compare also [ 30 ]). The more our industrial settings change, i.e. demanding more flexible production lines and highly agile processes, the higher the requirements regarding robot manipulators become. Such problems are addressed by commercially available robotic hands such as the shadow-hand [ 16 ], the DLR-hand [ 9 ] and in research where a recent focus is also on soft devices for human-robot-interaction [ 24 ]. Currently, specialized grippers are developed specifically for each application, e.g. [ 28 ], approaches based on biomemimetic, soft devices (e.g., [ 37 ]) which naturally conform to a large number of different objects could lead to a more efficient use of grippers. Modular, highly redundant manipulators, which can change their configuration with respect to the assigned task are presented by Pieber et al. [ 67 ]. Manipulation in a mobile setting (see Fig.  6 ) is even more complex and different methods have to be developed to address flexibility [ 88 ].

Husky platform equipped with two UR5 manipulator arms in a grasping scenario where the moving platform starts, approaches a soup can on a support, grasps it and picks it up using reinforcement learning. Figure adapted from [ 88 ]

2.3 Sensing

Perceiving the environment is a crucial capability of manipulators in general and mobile manipulators especially, it is important in a variety of fields where robots physically interact with the environment as well as with human beings: in human care and assistance applications [ 30 , 62 ], in industry settings as Cobots (e.g., which have names such as Walt [ 22 ]), in fields of service [ 59 ] and also novel public mobility applications [ 36 ].

In order to perceive their environment properly, mobile robots and especially also mobile manipulators need to be equipped with the corresponding sensors. These sensors may be placed on different parts of the platform depending on the purpose they fulfill:

2.3.1 Elevated sensors

Such a setup resembles a concept known from biology: eyes. In the case of humans who walk upright, the eyes are elevated sensors - at the top of the body – in order to get the best possible impression of the surroundings. In mobile robotics, such a concept can enable, e.g., strategic ahead-of-path-planning and obstacle avoidance [ 50 ]. Important aspects in this context are the choice of proper sensor hardware, placement of the sensors and intelligent task planning: e.g. while a manipulation tasks is ongoing, these sensors can be used for planning next actions after completing the current task. These elevated sensors are often optical systems such as, e.g. light detection and ranging (LIDAR) [ 20 ], stereo cameras and RGB-D cameras [ 61 ].

2.3.2 Touch sensors

Due to the limited capabilities of cameras (e.g., with respect to the field of view: occlusion, bad lighting conditions, mounting and integration issues, e.g., with respect to the object properties: temperature, texture, material) size in terms of highly dynamic processes, often physical sensors are preferred over vision-based systems. Additionally they are preferable in such tasks, where properties of the manipulated object (surface texture, temperature, fragility, stiffness, etc.) are of major importance. Research in terms of visuo-tactile object perception has been shown recently, however, purely physical approaches show promising results in terms of resolution and bandwidth. Based on electrical sensors (see also [ 55 ]), e.g., employing capacitive sensing electrodes [ 74 ], such approaches can be easily fabricated using rapid-prototyping technology [ 26 ] and can enable exteroceptive as well as proprioceptive sensing of robotic devices and been shown previously for walking of humanoid robotics [ 27 ].

2.3.3 Base sensors

For holonomic platforms, such sensors often have to cover the whole proximity region of the base platform. This can be achieved using, e.g., laser rangefinders, LIDAR, Time-of-Flight and Radar [ 5 ], as well as ultra-sonic-based principles are in use. These often times need to be mounted on strategic positions on the mobile platform in order to optimally cover a field of view of \(f_{V} = 360^{\circ}\) . These sensors are often used for map building in an unknown environment (such as is necessary in a dynamic industrial context or in the RoboCup Rescue League). This is commonly known as Simultaneous Localization and Mapping (SLAM). The quality (accuracy of the trajectory followed) of the SLAM depends strongly on the hardware used [ 29 ]. Depth estimation (such as enbaled by, e.g., LIDAR) can significantly improve the quality of the SLAM [ 70 ], this, however, also comes with an increased computational complexity.

2.4 Interfacing and interacting

Another aspect is the interfacing of robots with their environment, this is especially important when teaching and controlling the robot system in a collaboration task. Such interfaces can either be established using touchpad technology [ 25 , 86 ] or in contactless gesture-based approaches [ 78 ]. In the latter it is often mandatory to find a common language which is, on the one hand, intuitive for the human operator and on the other hand easily interpretable and manageable in terms of signal processing [ 85 ].

The terms human-robot interaction and human-robot collaboration describe all those application scenarios and the various methods used to implement these. In most cases, especially in industry, it is important to provide a safe environment for humans. In this respect, lately, strong research efforts have been put into qualifying and quantifying the resulting injury and level of pain caused by an impact with certain energy and location on the body [ 33 , 34 ]. Apart from the application of ISO/TS 15066 [ 56 ], more effort is necessary in the definition of standard testing methods and standardized test setup and hardware. Currently available developments which enable safe human robot co-working are airbags systems [ 90 ], robotic skins [ 15 , 53 ], but also control strategies for intuitive hand guidance [ 92 ] based on force/torque-sensing.

3 Intelligence (software)

The software implementation of mobile manipulators is the systems’ intelligence. It is connecting the hardware and corresponding sensor readings in a sense-plan-act-loop. It consequently is the brain of the whole system. Important tasks here are:

Interfacing: communicating with the sensors and actuators hardware.

Perception: human and object detection algorithms, mapping.

Knowledge-base: gather information about the robot, the environment, objects.

Planning: path and task planning components.

Control: actuate the system in order to follow the planned movement as accurately as possible.

3.1 Understanding humans and the environment

The authors of [ 77 ] claim already that common metrics are important in Human-Robot-Interaction and safety standards have been implemented since then. Vision-based distance monitoring has recently been presented in [ 80 ] and in [ 73 ], a strategy for human-robot collaboration is presented where the signals from 378 sensors of an artificial skin are used in a constraint-based optimization task to control the robot and avoid collisions. Delicate grasp force control, for manipulation, based on physical sensors for a high DOF robot hand (Fig.  7 ) has been shown in [ 18 ]. While in the previous approaches, human-robot physical contact is monitored, in [ 17 ], the authors establish an interaction scheme based on gaze recognition of the human co-operator. Even the role of robot-telepathy is evaluated in [ 93 ]. In this field of robot intelligence, common metrics and a standardized approach, however, are still missing. It is very likely that different strategies will have to be pursued depending on the field of robot application (elderly or child care, robot tele-operation, flexible industry with a lot of interaction, pure production lines, etc.).

A 20 DOF robotic hand including multimodal touch sensors and the grasp force control strategy as suggested by Deng et al. Figure adapted from [ 18 ]

3.2 Moving smartly (path and motion planning)

Path planning is generally defined as follows: A collision free and continuous path in the joint angle space is to be found to bring a robot from a start configuration (position and orientation) to a target configuration without violating the limitations of the system [ 89 ]. Redundant robots, as described before, are able to move while the position and orientation of the end effector are fixed. An advantage of such kinematically redundant systems is the capability of collision avoidance, while the end effector follows a specific path [ 60 ]. Therefore, such systems can be used for technically difficult dexterity tasks, for example grasping in a tube or between two walls. The workspace of a stationary industrial robot is also limited by the arm length and allowed joint torques. The initial placement of the base of a static industrial serial manipulator when installing it is often done by trial and error or by help of human experience. For mobile manipulators controlled by an operator (as often in rescue missions) this is also valid. For autonomous driven mobile manipulators the positioning task of the mobile platform which is equal to the base of the mounted serial manipulator is part of the path planning algorithm and there are many possibilities how to fulfill this planing task.

Mobile manipulator movement can be considered in three ways, movement of the mobile platform, movement of the manipulator or a combination of both. Depending on the system the separation of mobile platform and manipulator reduces the difficulty to solve the inverse kinematics problem of such kinematically redundant systems [ 35 ], e.g., Husty et al. [ 41 ] show an analytic solution of the inverse kinematic for a general serial link manipulator. The problem of path planning and collision avoidance in general of redundant configurations can be solved using different methods. In [ 100 ], the authors describe the solution, based on numerical optimization algorithms using the pseudo-inverse of the Jacobi matrix. In [ 14 ] neural networks are used to find a solution for a planar manipulator with three joints. Quendler [ 69 ] uses a combination of different concepts to solve the inverse kinematics problem for a 6-DOF manipulator used on the mobile rescue robot R.U.D.I. [ 42 ] which was later extended with an additional joint [ 68 ] and mounted on the rescue robot platform T.R.U.D.I. [ 38 ] (Fig.  1 a). Here, the Cyclic Coordinate Descent and Broyden-Fletcher-Goldfarb-Shanno Quasi-Newton  [ 87 ] methods are used. The manipulator is controlled by a human operator with the help of a joystick input in either of the following modes: Control of each joint individual (forward kinematics), control position and orientation of the end effector relative to the actual end effector coordinate system or to the robots body fixed coordinate system (inverse kinematics).

In [ 7 ] and [ 79 ] an approach is shown where not only the Cartesian working space of the robot is used for the solution but also the joint angle space. Both [ 49 ] and [ 31 ] give an overview of many possible solutions for path planning and collision avoidance in the Cartesian workspace or low dimensional joint angle space. A summary of methods for higher dimensional spaces is given in [ 66 ]. In particular, the three methods Net-based, Sampling-based and Trajectory-Optimizing Path Planning are discussed. Additional to collision avoidance while following a desired path with redundant robotic systems also the configuration regarding manipulability from different points of view can be considered. Thereby often manipulator Jacobian based methods as a measure of robot manipulability introduced by Yoshikawa in [ 98 ] are used. It is based on the idea to determine the capability of the end effector to move in an certain direction as well as how easy the end effector can generate forces. The detailed calculation of those measurements is based on the idea of manipulability and force ellipsoids [ 52 ]. Based on this idea also the dynamic manipulability measure was introduced [ 97 ] and reformulated for redundant systems by Chiacchio in [ 13 ]. The authors in [ 75 ] use a combination of different concepts such as manipulability, reachability [ 54 ] and stiffness [ 3 ] to define a so-called comfort zone for kinematically redundant mobile manipulators.

Additionally, non-traditional methods such as robot learning based on infant development theory has been presented in [ 94 ]. In this setting, the mobile manipulator is made to play games with varying difficulty as shown in Fig.  8 . As soon as there is no more progress on one level of difficulty, it will automatically step forward in complexity.

Game setup with varying complexity for robot learning. Figure adapted from [ 94 ]

3.3 Do the right thing (control)

Mobile Robots are a set of complex mechanical systems, sensors and computational processes. A control system architecture and its implementation solve the challenging task to transfer these components to a well performing working robot. A traditional control methodology decomposes the robot’s tasks into functional steps: sense – plan (control) – act. This is needed and often suitable for solving the problem of driving a mobile robot, i.e. to guarantee a smooth navigation of a mobile robot platform. Classical PD or PID controller are still used in terms of servo control of the robot’s individual drives. Many effective methods for the navigation and trajectory tracking have been proposed [ 46 ].

Robot control of mobile manipulator platforms in general deals with the central problem of finding appropriate forces or torques that are generated by the actuators, e.g. DC-motors, in order to move the robot arm or hand to a desired position or to track a desired reference-trajectory.

Cascade control structures of multiple linear and nonlinear controllers are widely used. The mobile rescue robot T.R.U.D.I. uses four brushless DC (BLDC) motors with a power of \(P_{M} = 200~\mbox{W}\) to drive the main tracks. Each main drive (left and right side of the robot) is powered by two motors to achieve a higher driving torque. The two motors on each side are permanently coupled via a chain connection (Fig.  9 marked in red). Here, a combination of closed loop an open loop controller together with the BLDC motor controller was implemented. Figure  9 shows the overall control structure, one can see the user input (robot velocities \(\dot{x}_{R}\) and \(\dot{\phi }_{R}\) , the inverse kinematics block is used to calculate the desired angular velocities of the chain sprocket for each track ( \(\omega _{1}\) and \(\omega _{2}\) ). The green blocks represent linear (LEAD-compensator [ 39 ]) speed controllers which control the first motor (of each side) in an closed loop control structure. The control signal (controller output) is also sent to the second BLDC motor controller ( \(M_{2L}\) / \(M_{2R}\) ) to achieve the same speed.

Schematic of the drive control structure

In known or adaptable environments (e.g. in a factory), it is suitable to employ model-based dynamic control strategies including inverse kinematics approaches. They are based on mathematical descriptions of the manipulators in terms of nonlinear differential equations and their parameters. Computed torque control together with PD or PID control have successfully been employed for control in joint space [ 43 , 45 ]. Such a combined control strategy was also used to control the 7-DOF manipulation arm on the mobile rescue robot T.R.U.D.I. (Fig.  1 a) [ 68 ].

Depending on the used drive concept (e.g. wheeled robots), present (nonholonomic) constraints and the complexity (highly nonlinear systems) nonlinear control techniques like feedback linearization or Lyapunov based control turn out to be mathematically intense, but suitable methods [ 19 , 83 ].

In uncertain (highly dynamic) and uncontrolled environments, the control algorithms must be more sophisticated involving some kind of (artificial) intelligence. One of the methodologies for treating such uncertain systems is the adaptive control methodology [ 40 , 91 ]. Furthermore, the design of a robust control has been attracting great attention. The sliding mode control methodology is a robust control procedure often used in mobile robots’ control [ 2 ].

Recently, ANN-based control methods have been attracted much more attention. The authors in [ 99 ], for example, address the control problem for a class of mobile robot systems, where multi-layer feed forward ANNs and deep learning are both employed for the mobile robot system to achieve the trajectory tracing and obstacles avoidance.

Especially, when mobile rescue robots are used, the mobile manipulator is often teleoperated by a human operator. This can be supported by additional algorithms which may use additional sensor readings, be based on inverse kinematics (for example as described in Sect.  4.1 ) or semi-autonomous driving methods. Semi-autonomous here means assistant algorithms which help an operator to control a robot with higher DOFs. For Example the mobile manipulator T.R.U.D.I. described in Sect.  2.1 needs additional operator inputs to move the four flippers to support the mobile manipulator while overcoming difficult obstacles like stairs or steps. Additionally, in [ 32 , 63 , 64 ] different possibilities to control additional flipper tracks by sensor information about the environment or soil condition are described.

A comprehensive review of control strategies and their advantages and disadvantages can be found in [ 84 ].

4 RoboCup challenges

The first RoboCup competition was held at IJCAI-97, Nagoya. The idea was to push the developments of soccer playing robots up to a level to participate in a soccer game against human players in year 2050. At this time only soccer robots where in the view of the researchers by comparing real robots as well as robot simulation algorithms in the competition [ 47 ]. Step by step, different leagues where established each pursuing a different goal. With competitions in the Small-Size-, Soccer Simulation-, the Middle-Size-, the Standard Platform- as well as the Humanoid-League, soccer is still one main focus in the competition, with different aspects of soccer robots taken into account. Furthermore, more practice related tasks were included in the RoboCup competitions over the years. This results in a diversity of different disciplines of which the main ones are shown in Table  1 (including date of first event and main drive method).

In the following, we discuss mobile manipulators in view of the RoboCup Rescue and RoboCup@Work competitions. Mobile manipulators in general, with different suitable drive methods as shown in Table  1 , are used in RoboCup Rescue, RoboCup@Work, RoboCup Logistics, RoboCup@Home. Mobile robot platforms with omnidirectional drive system but without classical serial manipulators are used in Soccer Small-Size and Soccer Middle-Size leagues. Steinbauer et al. [ 76 ] give an overview of the development of the different RoboCup leagues from year 2000 to year 2016.

4.1 RoboCup rescue

After the Great Hanshin Earthquake (1995) in Kobe, the Japanese government decided to promote research related to urban disaster and rescue robots [ 4 ]. The RoboCup Rescue competition simulates the search for victims in a disaster scenario such as after an earthquake or flooding in an urban environment. Until 2015 the main focus of the competition was to find simulated victims in a big area with various obstacles like stairs, pipes, ramps and stones to overcome as shown in Fig.  10 .

RoboCup Rescue arena used until 2015

Finding those victims, scores points for the competition ranking, extra points are gained for mapping the area, marking various objects in the map or various audio and visual detections. Those simulated victims can be placed all over the competition area and consists of visual markers as QR codes, E or C charts, a heat pad to simulate body temperature and an audio player.

They can be located at a height of up to \(h_{o} = 1.6~\mbox{m}\) over ground, be hidden in holes or can be found in pipes and behind closed doors. The search for victims could be done by a swarm of robots, where only one of these migh be teleoperated and the others needed to operate in autonomous mode. Each year, the rules for the RoboCup Rescue competitions are extended to support the development of new features for the participating robots and to enhance their existing capabilities. The competition is manly based on the mobile robot test strategies developed by the American National Institute of Standards and Technology (NIST).

In the year 2016, a new rule set was introduced during the RoboCup World Championship in Leipzig (Germany) with the focus to test repeatability, functionality and operator proficiency, and all capabilities should be combined onto a single robot platform.

So-called individual test lanes (example shown in Fig.  12 ) where introduced. Each lane has another testing focus [ 65 ]. For each repetition of one lane, the team earns one point and by the help of basic sensor and dexterity tests at the begin of each test round this number can be multiplied by the number of successful fulfilled sensor/dexterity tasks. The best teams in this individual tests are qualified to participate in the final runs where the individual test lanes are used to build a large arena with the goal to find as many simulated victims as possible.

The basic sensor and dexterity tests at the beginning of each test lane ensures the reliability of the system. Especially, test lanes out of the so called mobility tests suits, include driving over terrain with high difficulty. This are for example driving over sand and gravel (with \(15^{\circ}\) slope) as well as stepfields (a diagonal hill terrain consisting of 20 cm square steps made from posts with flat tops and stairs ( \(35^{\circ}\) and \(45^{\circ}\) ) and obstacles partly blocked with debris, e.g. angled bars in defined locations). These tests require a high degree of stability and reliability in the mechanical as well as in the electronic and sensor system. The dexterity and sensor tests have to be done every time the mobile manipulator starts on a new test lane which is approximately 5 times a day. The schedule during one competition day is very tight therefore a problem in the sensor system or with the manipulation arm can lead to a loss of points.

Figure  11 shows the used test board which consists of two parts where the lower part is used to test the sensors of the mobile manipulator and the upper part is used to test the dexterity of the mobile manipulator. For the sensor test the robot system has to fulfill 6 identification tasks:

Video Image Resolution: Visual identification the third biggest concentric C gap by the competition judge.

Motion Detection: Highlight the moving target (1-4) automatically by any usage of any camera and integrated video processing. Additionally track the identified motions on the operator station textually or audibly.

Thermal Image Resolution: Visual identification of the concentric Landolt C with a 2 cm gap to evaluate thermal resolution by the competition judge.

Audio Acuity: Two way communication from mobile manipulator to operator station. Tested by judge and team members.

Color/Pattern Recognition: Automatically identification of two hazards labels out of 12 possibilities. Highlight and track the identified labels on the operator station textually or audibly.

Gas: Operator demonstrates active display of increase in \(\mathrm{CO}_{2}\) concentration when a team-mate breaths into the robot’s sensor or a \(\mathrm{CO}_{2}\) cartridge is opened near the sensor.

All identification tasks has to be done with a minimum distance of 40 cm to the sensor board. Furthermore 4 dexterity tasks has to be performed:

Touch: Touch a 1 cm diameter circular target on the end of a pipe with a pencil attached to the manipulator end effector.

Rotate: Grab a 5 cm octagonal pipe cap and rotate \(180^{\circ}\) with the end effector.

Extract: Grab a 5 cm octagonal pipe cap and pull out of the pipe after holding it 1 s it can be dropped to the floor.

Inspect: Visual identification the number of bars placed on the internal walls of a 5 cm pipe by the competition judge.

Simulated victim and sensor board to evaluate the sensor capabilities

Test Line (MAN2- Align) with two parallel bars (10 cm in width and in height) adjusted to the dimension of the robot

For the sensor and dexterity tests a precise movement of the manipulator without oscillations or vibrations as well as a stable positioning of the mobile platform (without tilting) is crucial. To unveil the mobile manipulator in rescue purpose often additional flipper tracks as described in Sect.  2 are used. Moreover it has been shown that a remote controlled movement of the end effector by using an inverse kinematics solution as described in 3.2 is a big advantage in terms of usability and time consumption to solve the task compared to a simple forward kinematic movement. Especially for the sensor and dexterity tests the operator uses the inverse kinematics mode which allows a movement of the end effector related to the end effector-camera coordinate system. In this right handed coordinate system the z-y plane is equal to the camera image plane and the x-axis is equal to the camera optical axis. Therefore a movement parallel to the sensor/dexterity board is done very intuitive in comparison to a movement with classical forward kinematics where every joint of the manipulator has to be moved individually.

4.2 RoboCup@Work

In 2012, the RoboCup Federation included the RoboCup @Work league to increase the development progress of mobile manipulators in a more industry-related setting. The mobile robotic systems used in RoboCup @Work are aimed to be used as flexible multipurpose systems also for smaller companies with changing product range. In the competition, different tasks like acquiring, pick and place as well as transportation of different machine parts and objects are to be performed. The contest procedure is separated into two parts. After some basic tests which must be completed, an extended, more difficult test scenario has to be fulfilled.

The basic tests include testing navigation, manipulation and transportation capabilities of the participating mobile manipulators with three different test tasks:

Basic navigation: The mobile manipulator will be send a string contains a series of triples, each of which specifies a location, an orientation and a pause duration. The challenge is to navigate autonomous robust and safe in the scenario environment. Additional to specific service areas, arena objects, wall markers and floor markers random obstacles my be placed in the environment.

Basic manipulation: The mobile manipulator will be send some task specifications consist of the objects source location, the objects destination location, a list of objects to manipulated from source to the destination area and the final place for the robot. As objects different obstacles like aluminum profiles of different size and color, screws, nuts and plastic tubes can be used. An object does not count as placed when it dropped from a height above 5 cm. The pose of the object at the destination location can be chosen freely by the robot.

Basic transportation: Here the ability of the robots to combine navigation and manipulation tasks as well as its task planning capabilities are tested. The task is to pic objects from the source location to the destination location. Two lists will be send to the robot. The first list contains a list of manipulation object descriptions, the second list contains for each destination area a configuration of manipulation objects the robot is supposed to achieve. A robot may carry up to three objects at the same time and it is not allowed to place objects anywhere except on the robot itself and the correct destination areas.

Figure  13 a shows a KUKA youBot robot platform during the basic manipulation test [ 1 ]. Figure  13 b shows the mobile manipulator of evocortex used by the RoboCup@Work team AutonOHM at the Nuremberg Institute of Technology during the precision placement test where the mobile manipulator has to pick up different obstacles like screws, nuts or aluminum profiles and has to place them into prepared cavities on a target desk after a short movement of the robot platform. Figure  14 shows the rotating table which is used in the competition for the so called rotating table test. The robot has to navigate autonomous to the table and has to grasp all objects from the moving table. Objects can pass multiple times by the robot end-effector until the maximum time is over. Target location of the objects is on the robot itself. The rotating table test assesses the ability to pick and place moving objects which are placed on a rotating turntable with unknown speed. In this context, the vision system as well as the physical movement of the end-effector (this is the robots’ hand-eye-coordination ) have to meet very high requirements in terms of speed and precision [ 48 ].

Two RoboCup@Work examples

Rotating table used in the competition

The score calculation for each test is defined individually, it includes points for achieving certain sub tasks, for winning a run and also penalty points can be assigned each time an incident occurs (collisions, losing an object during manipulation or transportation). Additional to this tests, there are further technical challenges (e.g. cluttered pick test, human-robot collaboration, open challenge) to evaluate specific capabilities of the robotic system. These challenges are separately awarded and could be included into the major competition in future.

5 Conclusion

Mobile manipulation is a vivid research field with a variety of applications such as industry, rescue, service and health. Each subfield covered in the development of mobile manipulators, such as construction of the hardware (platform, manipulator, gripper), choice of suitable algorithms for path-planning, obstacle avoidance, control and perception is a huge field of research on its own. In this paper, we have tried to give a brief overview of recent developments in all those fields covered together with an insight into the development on the example of a tracked rescue robot developed at CUAS. Additionally, we have discussed mobile manipulation challenges in the context of the RoboCup and especially the RoboCup Rescue and RoboCup@Work leagues. We consider the RoboCup as an attractive platform to motivate young researchers and students to have a deeper look into the research challenges ahead in robotics in general and in this context especially in mobile manipulation.

Ahmed, S., Jandt, T., Kulkarni, P., Lima Carrion, O., Mallick, A., Moriarty, A., Nair, D., Thoduka, S., Awaad, I., Dwiputra, R., Hegger, F., Hochgeschwender, N., Sanchez, J., Schneider, S., Kraetzschmar, G. (2016): b-it-bots RoboCup@Work team description paper.

Aienizi, T. O., Aienizi, N. S., Rouili, Z. A., Dhahri, S. (2019): Trajectory tracking of wheeled mobile robot using adaptive second order sliding mode controller. In 2019 6th international conference on control, decision and information technologies (CoDIT) (pp. 2039–2045).

Google Scholar  

Ajoudani, A., Tsagarakis, N. G., Bicchi, A. (2017): Choosing poses for force and stiffness control. IEEE Trans. Robot., 33(6), 1483–1490.

Akin, H. L., Ito, N., Kleiner, A., Pellenz, J., Visser, A. (2013): Robocup rescue robot and simulation leagues. AI Mag., 34, 78–86. https://doi.org/10.1609/aimag.v34i1.2458 .

Article   Google Scholar  

Ali, F., Bauer, G., Vossiek, M. (2014): A rotating synthetic aperture radar imaging concept for robot navigation. IEEE Trans. Microw. Theory Tech., 62(7), 1545–1553.

Ambrose, R. O., Aldridge, H., Askew, R. S., Burridge, R. R., Bluethmann, W., Diftler, M., Lovchik, C., Magruder, D., Rehnmark, F. (2000): Robonaut: NASA’s space humanoid. IEEE Intell. Syst. Appl., 15(4), 57–63.

MATH   Google Scholar  

Bhattacharya, S., Pivtoraiko, M. (2015): A classification of configuration spaces of planar robot arms for a continuous inverse kinematics problem. Acta Appl. Math., 139(1), 133–166.

MathSciNet   MATH   Google Scholar  

Bischoff, R., Kurth, J., Schreiber, G., Koeppe, R., Albu-Schaeffer, A., Beyer, A., Eiberger, O., Haddadin, S., Stemmer, A., Grunwald, G., Hirzinger, G. (2010): The KUKA-DLR lightweight robot arm - a new reference platform for robotics research and manufacturing. In ISR 2010, 41st international symposium on robotics, and ROBOTIK 2010, 6th German conference on robotics (pp. 1–8).

Borst, C., Fischer, M., Haidacher, S., Liu, H., Hirzinger, G. (2003): DLR hand II: experiments and experience with an anthropomorphic hand. In 2003 IEEE international conference on robotics and automation (Cat. No. 03CH37422) (Vol. 1, pp. 702–707).

Brandstötter, M., Mirkovic, D., Hofbaur, M. (2017): Mobile Manipulation – eine altbekannte Technologie findet durch sensitive Robotertechnologie Einzug in die Industrie. In C-AR2017 – conference on automation and robotics (pp. 1–6).

Bruno Siciliano, O. K. (2008): Springer handbook of robotics (p. 2008). Berlin: Springer.

Burkhardt, M., Karumanchi, S., Edelberg, K., Burdick, J. W., Backes, P. (2018): Proprioceptive inference for dual-arm grasping of bulky objects using robosimian. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 4049–4056).

Chiacchio, P., Concilio, M. (1998): The dynamic manipulability ellipsoid for redundant manipulators. In Proceedings. 1998 IEEE international conference on robotics and automation (Cat. No. 98CH36146) (Vol. 1, pp. 95–100).

Choong, P. M., Aziz, M. A. S., Yahya, S., Almurib, H. A. F., Moghavvemi, M. (2016): Simulation of a three-links planar redundant manipulator for obstacle avoidance using neural networks. In 2016 IEEE industrial electronics and applications conference (IEACon) (pp. 297–303). https://doi.org/10.1109/IEACON.2016.8067395 .

Chapter   Google Scholar  

Cirillo, A., Ficuciello, F., Natale, C., Pirozzi, S., Villani, L. (2016): A conformable force/tactile skin for physical human-robot interaction. IEEE Robot. Autom. Lett., 1(1), 41–48.

Company, S. R. (2020). https://www.shadowrobot.com/wp-content/uploads/shadow_dexterous_hand_technical_specification_E_20190221.pdf .

Das, D., Rashed, M. G., Kobayashi, Y., Kuno, Y. (2014): Recognizing gaze pattern for human robot interaction. In 2014 9th ACM/IEEE international conference on human-robot interaction (HRI) (pp. 142–143).

Deng, Z., Jonetzko, Y., Zhang, L., Zhang, J. (2020): Grasping force control of multi-fingered robotic hands through tactile sensing for object stabilization. Sensors, 20(4), 1–21. https://doi.org/10.3390/s20041050 .

Devon, D., Bretl, T. (2007): Kinematic and dynamic control of a wheeled mobile robot (pp. 4065–4070). https://doi.org/10.1109/IROS.2007.4399599 .

Dietrich, R., Dörr, S. (2019): Deep learning-based mutual detection and collaborative localization for mobile robot fleets using solely 2D lidar sensors. In 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 6706–6713).

Dissanayake, G., Paxman, J., Miro, J. V., Thane, O., Thi, H. (2006): Robotics for urban search and rescue. In First international conference on industrial and information systems (pp. 294–298).

El Makrini, I., Elprama, S. A., Van den Bergh, J., Vanderborght, B., Knevels, A., Jewell, C. I. C., Stals, F., De Coppel, G., Ravyse, I., Potargent, J., Berte, J., Diericx, B., Waegeman, T., Jacobs, A. (2018): Working with walt: how a cobot was developed and inserted on an auto assembly line. IEEE Robot. Autom. Mag., 25(2), 51–58.

Endo, D., Okada, Y., Nagatani, K., Yoshida, K. (2007): Path following control for tracked vehicles based on slip-compensating odometry. In 2007 IEEE/RSJ international conference on intelligent robots and systems (pp. 2871–2876).

Escaida Navarro, S., Nagels, S., Alagi, H., Faller, L. M., Goury, O., Morales-Bieze, T., Zangl, H., Hein, B., Ramakers, R., Deferme, W., Zheng, G., Duriez, C. (2020): A model-based sensor fusion approach for force and shape estimation in soft robotics. IEEE Robot. Autom. Lett., 5, 5621–5628.

Faller, L. M., Mühlbacher-Karrer, S., Zangl, H. (2016): Inkjet-printing rapid prototyping of a robust and flexible capacitive touch panel. In IEEE sensors (Vol. 2016).

Faller, L. M., Granig, W., Krivec, M., Abram, A., Zangl, H. (2018): Rapid prototyping of force/pressure sensors using 3D- and inkjet-printing. J. Micromech. Mircoeng., 28(10). https://doi.org/10.1088/1361-6439/aaadf4 .

Faller, L. M., Stetco, C., Mitterer, T., Zangl, H. (2019): An all-flexible sensing sole for legged robots. In 2019 IEEE international conference on flexible and printable sensors and systems (FLEPS).

Faller, L. M., Stetco, C., Zangl, H. (2019): Design of a novel gripper system with 3D- and inkjet-printed multimodal sensors for automated grasping of a forestry robot. In 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS).

Filipenko, M., Afanasyev, I. (2018): Comparison of various slam systems for mobile robot in an indoor environment. In 9th IEEE int. conf. on intelligent systems.

Garcia, I., Gonçalves, F., Ribeiro, T., Fernandes, P., Rocha, C., Boucinha, R., Lopes, G., Ribeiro, A. F. (2019): Autonomous 4DOF robotic manipulator prototype for industrial environment and human cooperation. In 2019 IEEE international conference on autonomous robot systems and competitions (ICARSC) (pp. 1–6).

Gasparetto, A., Boscariol, P., Lanzutti, A., Vidoni, R. (2015): Path planning and trajectory planning algorithms: a general overview. In Motion and operation planning of robotic systems (pp. 3–27).

Guo, Y., Bao, J., Song, A. (2009): Designed and implementation of a semi-autonomous search robot (pp. 4621–4626). https://doi.org/10.1109/ICMA.2009.5244797 .

Haddadin, S., Albu-Schaffer, A., Haddadin, F., Rosmann, J., Hirzinger, G. (2011): Study on soft-tissue injury in robotics. IEEE Robot. Autom. Mag., 18(4), 20–34.

Haddadin, S., Haddadin, S., Khoury, A., Rokahr, T., Parusel, S., Burgkart, R., Bicchi, A., Albu-Schäffer, A. (2012): A truly safely moving robot has to know what injury it may cause. In 2012 IEEE/RSJ international conference on intelligent robots and systems (pp. 5406–5413).

Hanafusa, H.T., Nakamura, Y. (1981): Analysis and control of articulated robot arms with redundancy. IFAC Proc. Vol., 14(2), 1927–1932.

Hashimoto, N., Tomita, K., Kamimura, A., Matsumoto, O. (2014): Technology evaluations of personal mobility vehicles in Tsukuba-city mobility robot designated zone — an experimental approach for personal mobility for sharing. In 2014 international conference on connected vehicles and expo (ICCVE) (pp. 773–774).

Hashizume, J., Huh, T. M., Suresh, S. A., Cutkosky, M. R. (2019): Capacitive sensing for a gripper with gecko-inspired adhesive film. IEEE Robot. Autom. Lett., 4(2), 677–683.

Hofer, P., Sereinig, M., Quendler, S., Werth, W. (2016): RoboCup rescue 2017 team description paper CUAS RRR. Tech. rep., RoboCup Rescue 2016 TDP Collection.

Horn, M., Dourdoumas, N. (2004): Regelungstechnik: Rechnerunterstützter Entwurf zeitkontinuierlicher und zeitdiskreter Regelkreise. Elektrotechnik: Regelungstechnik. Pearson Studium. https://books.google.at/books?id=qvFSDQEACAAJ .

Huang, H. C., Tsai, C. C. (2008): Adaptive trajectory tracking and stabilization for omnidirectional mobile robot with dynamic effect and uncertainties. IFAC Proc. Vol., 41(2), 5383–5388. https://doi.org/10.3182/20080706-5-KR-1001.00907 .

Husty, M. L., Pfurner, M., Schröcker, H. P. (2007): A new and efficient algorithm for the inverse kinematics of a general serial 6R manipulator. Mech. Mach. Theory, 42(1), 66–81.

Isop, W., Sereinig, M., Quendler, S., Werth, W. (2013): Robot league team CUAS RRR-Team (Austria). Tech. rep., RoboCup Rescue 2016 TDP Collection.

Kawamura, S., Svinin, M. (2006): Advances in robot control. From everyday physics to human-like movements. On the occasion of the 70th birthday of Suguru Arimoto. https://doi.org/10.1007/978-3-540-37347-6 .

Kazanzides, P., Chen, Z., Deguet, A., Fischer, G. S., Taylor, R. H., DiMaio, S. P. (2014): An open-source research kit for the da Vinci® Surgical System. In 2014 IEEE international conference on robotics and automation (ICRA) (pp. 6434–6439).

Kelly, R., Santibáñez Davila, V., Loría Perez, J.A. (2005): Control of robot manipulators in joint space. Springer-Verlag London Limited.

Kim, H., Kim, B. (2017): Minimum-energy cornering trajectory planning with self-rotation for three-wheeled omni-directional mobile robots. Int. J. Control. Autom. Syst., 15, 1857–1866. https://doi.org/10.1007/s12555-016-0111-x .

Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawa, E. (1998): Robocup: the robot world cup initiative. In Proceedings of the international conference on autonomous agents. https://doi.org/10.1145/267658.267738 .

Kraetzschmar, G., Hochgeschwender, N., Nowak, W., Hegger, F., Schneider, S., Dwiputra, R., Berghofer, J., Bischoff, R. (2015): RoboCup@Work: competing for the factory of the future. In RoboCup 2014: Robot World Cup XVIII. Lecture notes in computer science (Vol. 8992, pp. 171–182). https://doi.org/10.1007/978-3-319-18615-3-14 .

Latombe, J. C. (1991): Robot motion planning (Vol. 2). New York: Springer.

Lee, D., Lu, Y., Kang, T., Choi, I., Lim, M. (2012): 3D vision based local obstacle avoidance method for humanoid robot. In 2012 12th international conference on control, automation and systems (pp. 473–475).

Lin, L., Shih, H. (2013): Modeling and adaptive control of an omni-mecanum-wheeled robot. In Intelligent control and automation (Vol. 4, pp. 166–179). https://doi.org/10.4236/ica.2013.42021 .

Lynch, K., Park, F. (2017): Modern robotics: mechanics, planning, and control. Cambridge: Cambridge University Press.

Maiolino, P., Galantini, F., Mastrogiovanni, F., Gallone, G., Cannata, G., Carpi, F. (2015): Soft dieelectrics for capacitive sensing in robot skins: performance of different elastomer types. Sens. Actuators A, Phys., 226, 37–47.

Makhal, A., Goins, A. K. (2017): Reuleaux: Robot Base Placement by Reachability Analysis. ArXiv e-prints.

Mastrogiovanni, F., Natale, L., Cannata, G., Metta, G. (2015): Editorial for the special issue on advances in tactile sensing and tactile-based human-robot interaction. Robot. Auton. Syst., 63, 227–308.

Matthias, B., Reisinger, T. (2016): Example application of iso/ts 15066 to a collaborative assembly scenario. In Proceedings of ISR 2016: 47st international symposium on robotics (pp. 1–5).

Merhof, W. (2015): Fahrmechanik der Kettenfahrzeuge. Alsbach/Bergstr.: Leuchtturm-Verl.

Michael, K., Michael, M. G. (2014): The packbots are coming: boosting security at the 2014 FIFA World Cup. IEEE Consum. Electron. Mag., 3(3), 59–61.

Moradi, H., Kawamura, K., Prassler, E., Muscato, G., Fiorini, P., Sato, T., Rusu, R. (2013): Service robotics (the rise and bloom of service robots) [tc spotlight]. IEEE Robot. Autom. Mag., 20(3), 22–24.

Mühlbacher-Karrer, S., Brandstötter, M., Schett, D., Zangl, H. (2017): Contactless control of a kinematically redundant serial manipulator using tomographic sensors. IEEE Robot. Autom. Lett., 2(2), 562–569. https://doi.org/10.1109/LRA.2016.2632973 .

Mur-Artal, R., Tardós, J. D. (2017): ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Trans. Robot., 33(5), 1255–1262.

Nguyen, P. H., Sparks, C., Nuthi, S. G., Vale, N. M., Polygerinos, P. (2019): Soft Poly-Limbs: toward a new paradigm of mobile manipulation for daily living tasks. Soft Robot., 6(1), 38–53. https://doi.org/10.1089/soro.2018.0065 .

Ohno, K., Morimura, S., Tadokoro, S., Koyanagi, E., Yoshida, T. (2007): Semi-autonomous control of 6-dof crawler robot having flippers for getting over unknown-steps. In 2007 IEEE/RSJ international conference on intelligent robots and systems (pp. 2559–2560).

Okada, Y., Nagatani, K., Yoshida, K. (2009): Semi-autonomous operation of tracked vehicles on rough terrain using autonomous control of active flippers. In 2009 IEEE/RSJ international conference on intelligent robots and systems (pp. 2815–2820).

Pellenz, J., Dillenberger, D., Steinbauer, G. (2016): Novel rule set for the robocup rescue robot league. In 2016 IEEE international symposium on safety, security, and rescue robotics (SSRR) (pp. 282–283).

Petrovic, L. (2018): Motion planning in high-dimensional spaces. arXiv:1806.07457 .

Pieber, M., Neurauter, R., Gerstmayr, J. (2018): An adaptive robot for building in-plane programmable structures. In IEEE/RSJ international conference on intelligent robots and systems (pp. 5321–5327). https://doi.org/10.1109/IROS.2018.8593381 .

Quender, S., Sereinig, M., Werth, W. (2015): Entwicklung eines Greifarms für eine mobile Roboterplattform. Tech. rep., Konferenz der Mechatronik-Plattform.

Quendler, S. (2015): Methods for calculating the inverse kinematics of a 6-dof robot manipulation arm. Master thesis, Carinthia University of Applied Sciences.

Quigley, M., Batra, S., Gould, S., Klingbeil, E., Le, Q., Wellman, A., Ng, A. Y. (2009): High-accuracy 3d sensing for mobile manipulation: improving object detection and door opening. In 2009 IEEE international conference on robotics and automation (pp. 2816–2822).

Robots, U. (2020). https://www.universal-robots.com/ .

Röhrig, C., Heß, D., Künemund, F. (2017): Motion controller design for a mecanum wheeled mobile manipulator. In 2017 IEEE conference on control technology and applications (CCTA) (pp. 444–449). https://doi.org/10.1109/CCTA.2017.8062502 .

Sadrfaridpour, B., Wang, Y. (2018): Collaborative assembly in hybrid manufacturing cells: an integrated framework for human–robot interaction. IEEE Trans. Autom. Sci. Eng., 15(3), 1178–1192. https://doi.org/10.1109/TASE.2017.2748386 .

Schmitz, A., Maiolino, P., Maggiali, M., Natale, L., Cannata, G., Metta, G. (2011): Methods and technologies for the implementation of large-scale robot tactile sensors. IEEE Trans. Robot., 27(3), 389–400. https://doi.org/10.1109/TRO.2011.2132930 .

Sereinig, M., Manzl, P., Gerstmayr, J. (2020): Komfortzone mobiler Manipulatoren. In Sechste IFToMM D-A-CH Konferenz 2020: 27./28. Februar 2020, Campus Technik Lienz 2020. https://doi.org/10.17185/duepublico/71180 .

Ferrein, A., Steinbauer, G. (2016): 20 years of RoboCup. Künstl. Intell.. https://doi.org/10.1007/s13218-016-0449-5 .

Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., Goodrich, M. (2006): Common metrics for human-robot interaction. In Proceeding of the 1st ACM SIGCHI/SIGART conference on human-robot interaction – HRI ’06 (p. 33). Salt Lake City: ACM Press. https://doi.org/10.1145/1121241.1121249 . http://portal.acm.org/citation.cfm?doid=1121241.1121249 .

Stetco, C., Mühlbacher-Karrer, S., Lucchi, M., Weyrer, M., Faller, L. M., Zangl, H. (2020): Gesture-based contactless control of mobile manipulators using capacitive sensing. In 2020 IEEE international conference on instrumentation and measurement (I2MTC).

Stilman, M. (2010): Global manipulation planning in robot joint space with task constraints. IEEE Trans. Robot., 26(3), 576–584. https://doi.org/10.1109/TRO.2010.2044949 .

Svarny, P., Straka, Z., Hoffmann, M. (2018): Toward safe separation distance monitoring from RGB-D sensors in human-robot interaction. arXiv:1810.04953 [cs].

Taheri, H., Qiao, B., Ghaeminezhad, N. (2015): Kinematic model of a four mecanum wheeled mobile robot. Int. J. Comput. Appl., 113, 6–9. https://doi.org/10.5120/19804-1586 .

TM (2020). https://www.tm-robot.com/en/ .

Tzafestas, S. (2014): Introduction to mobile robot control. Amsterdam: Elsevier.

Tzafestas, S. (2018): Mobile robot control and navigation: a global overview. J. Intell. Robot. Syst., 91, 35–58.

Vergara, C., Borghesan, G., Aertbelien, E., De Schutter, J. (2018): Incorporating artificial skin signals in the constraint-based reactive control of human-robot collaborative manipulation tasks. In 2018 3rd international conference on advanced robotics and mechatronics (ICARM) (pp. 280–287). https://doi.org/10.1109/ICARM.2018.8610697 .

Wall, S. A., Brewster, S. (2006): Sensory substitution using tactile pin arrays: human factors, technology and applications. Signal Process., 86(12), 3674–3695. https://doi.org/10.1016/j.sigpro.2006.02.048 . http://www.sciencedirect.com/science/article/pii/S0165168406001411 .

Article   MATH   Google Scholar  

Wang, L. T., Chen, C. C. (1991): A combined optimization method for solving the inverse kinematics problems of mechanical manipulators. IEEE Trans. Robot. Autom., 7(4), 489–499.

Wang, C., Zhang, Q., Tian, Q., Li, S., Wang, X., Lane, D., Petillot, Y., Wang, S. (2020): Learning mobile manipulation through deep reinforcement learning. Sensors, 20(3), 1–18. https://doi.org/10.3390/s20030939 .

Wei, K., Ren, B. (2018): A method on dynamic path planning for robotic manipulator autonomous obstacle avoidance based on an improved RRT algorithm. Sensors, 18(2), 571. https://doi.org/10.3390/s18020571 . http://www.mdpi.com/1424-8220/18/2/571 .

Weitschat, R., Vogel, J., Lantermann, S., Höppner, H. (2017): End-effector airbags to accelerate human-robot collaboration. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 2279–2284).

Wen-Hong Zhu, Z. B., Schutter, J. D. (1998): Adaptive motion/force control of multiple manipulators with joint flexibility based on virtual decomposition. IEEE Trans. Autom. Control, 43(1), 46–60.

Weyrer, M., Brandstötter, M., Husty, M. (2019): Singularity avoidance control of a non-holonomic mobile manipulator for intuitive hand guidance. Robotics, 8(1), 14. https://doi.org/10.3390/robotics8010014 . http://www.mdpi.com/2218-6581/8/1/14 .

Williams, T., Briggs, P., Pelz, N., Scheutz, M. (2014): Is robot telepathy acceptable? Investigating effects of nonverbal robot-robot communication on human-robot interaction. In The 23rd IEEE international symposium on robot and human interactive communication (pp. 886–891).

Wu, R., Zhou, C., Chao, F., Zhu, Z., Lin, C. M., Yang, L. (2017): A developmental learning approach of mobile manipulator via playing. Front Neurorobot., 11, 53.

Xuexi, Z., Yuming, A., Genping, F., Guokun, L., Shiliu, L. (2019): Survey on key technology of robocup rescue robot. In 2019 Chinese control conference (CCC) (pp. 4746–4750).

Yin, H., Huang, S., He, M., Li, J. (2017): A unified design for lightweight robotic arms based on unified description of structure and drive trains. Int. J. Adv. Robot. Syst., 14(4), 1–14. https://doi.org/10.1177/1729881417716383 .

Yoshikawa, T. (1985): Dynamic manipulability of robot manipulators. Trans. Soc. Instrum. Control Eng., 21(9), 970–975. https://doi.org/10.9746/sicetr1965.21.970 .

Yoshikawa, T. (1985): Manipulability of robotic mechanisms. Int. J. Robot. Res., 4(2), 3–9.

MathSciNet   Google Scholar  

Zheng, W., Wang, H. B., Zhang, Z. M., Li, N., Yin, P. H. (2019): Multi-layer feed-forward neural network deep learning control with hybrid position and virtual-force algorithm for mobile robot obstacle avoidance. Int. J. Control. Autom. Syst., 17, 1007–1018. https://doi.org/10.1007/s12555-018-0140-8 .

Zlajpah, L., Petric, T. (2012): Obstacle avoidance for redundant manipulators as control problem. In Serial and parallel robot manipulators – kinematics, dynamics, control and optimization (pp. 203–230).

Download references

Open access funding provided by Carinthia University of Applied Sciences (CUAS).

Author information

Authors and affiliations.

Department of Mechatronics, University of Innsbruck, Technikerstraße 13, 6020, Innsbruck, Austria

Martin Sereinig

Engineering and IT, Carinthia University of Applied Sciences, Europastrasse 4, 9524, Villach, Austria

Wolfgang Werth & Lisa-Marie Faller

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Lisa-Marie Faller .

Ethics declarations

Conflict of interest.

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access Dieser Artikel wird unter der Creative Commons Namensnennung 4.0 International Lizenz veröffentlicht, welche die Nutzung, Vervielfältigung, Bearbeitung, Verbreitung und Wiedergabe in jeglichem Medium und Format erlaubt, sofern Sie den/die ursprünglichen Autor(en) und die Quelle ordnungsgemäß nennen, einen Link zur Creative Commons Lizenz beifügen und angeben, ob Änderungen vorgenommen wurden. Die in diesem Artikel enthaltenen Bilder und sonstiges Drittmaterial unterliegen ebenfalls der genannten Creative Commons Lizenz, sofern sich aus der Abbildungslegende nichts anderes ergibt. Sofern das betreffende Material nicht unter der genannten Creative Commons Lizenz steht und die betreffende Handlung nicht nach gesetzlichen Vorschriften erlaubt ist, ist für die oben aufgeführten Weiterverwendungen des Materials die Einwilligung des jeweiligen Rechteinhabers einzuholen. Weitere Details zur Lizenz entnehmen Sie bitte der Lizenzinformation auf http://creativecommons.org/licenses/by/4.0/deed.de .

Reprints and permissions

About this article

Sereinig, M., Werth, W. & Faller, LM. A review of the challenges in mobile manipulation: systems design and RoboCup challenges. Elektrotech. Inftech. 137 , 297–308 (2020). https://doi.org/10.1007/s00502-020-00823-8

Download citation

Received : 21 May 2020

Accepted : 04 August 2020

Published : 03 September 2020

Issue Date : October 2020

DOI : https://doi.org/10.1007/s00502-020-00823-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• mobile robotics
• grasping and manipulation
• mobile robot control and learning
• robotic sensing

Schlüsselwörter

• mobile Robotik
• greifen und Manipulation
• Lernkonzepte und Regelung für mobile Roboter
• Sensoren in der Robotik
• Find a journal
• Publish with us
• Track your research

Help | Advanced Search

Computer Science > Robotics

Title: a comprehensive review on autonomous navigation.

Abstract: The field of autonomous mobile robots has undergone dramatic advancements over the past decades. Despite achieving important milestones, several challenges are yet to be addressed. Aggregating the achievements of the robotic community as survey papers is vital to keep the track of current state-of-the-art and the challenges that must be tackled in the future. This paper tries to provide a comprehensive review of autonomous mobile robots covering topics such as sensor types, mobile robot platforms, simulation tools, path planning and following, sensor fusion methods, obstacle avoidance, and SLAM. The urge to present a survey paper is twofold. First, autonomous navigation field evolves fast so writing survey papers regularly is crucial to keep the research community well-aware of the current status of this field. Second, deep learning methods have revolutionized many fields including autonomous navigation. Therefore, it is necessary to give an appropriate treatment of the role of deep learning in autonomous navigation as well which is covered in this paper. Future works and research gaps will also be discussed.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Artif Intell
• PMC10500442

Mobile robotics in smart farming: current trends and applications

Darío fernando yépez-ponce.

1 Instituto Universitario de Automática e Informática Industrial, Universitat Politècnica de València, Valencia, Spain

2 Facultad de Ingeniería en Ciencias Aplicadas, Universidad Técnica del Norte, Ibarra, Ecuador

José Vicente Salcedo

Paúl d. rosero-montalvo.

3 Computer Science Department, IT University of Copenhagen, Copenhagen, Denmark

Javier Sanchis

In recent years, precision agriculture and smart farming have been deployed by leaps and bounds as arable land has become increasingly scarce. According to the Food and Agriculture Organization (FAO), by the year 2050, farming in the world should grow by about one-third above current levels. Therefore, farmers have intensively used fertilizers to promote crop growth and yields, which has adversely affected the nutritional improvement of foodstuffs. To address challenges related to productivity, environmental impact, food safety, crop losses, and sustainability, mobile robots in agriculture have proliferated, integrating mainly path planning and crop information gathering processes. Current agricultural robotic systems are large in size and cost because they use a computer as a server and mobile robots as clients. This article reviews the use of mobile robotics in farming to reduce costs, reduce environmental impact, and optimize harvests. The current status of mobile robotics, the technologies employed, the algorithms applied, and the relevant results obtained in smart farming are established. Finally, challenges to be faced in new smart farming techniques are also presented: environmental conditions, implementation costs, technical requirements, process automation, connectivity, and processing potential. As part of the contributions of this article, it was possible to conclude that the leading technologies for the implementation of smart farming are as follows: the Internet of Things (IoT), mobile robotics, artificial intelligence, artificial vision, multi-objective control, and big data. One technological solution that could be implemented is developing a fully autonomous, low-cost agricultural mobile robotic system that does not depend on a server.

1. Introduction

In recent years, the global population has increased unprecedentedly, leading to significant changes in food demand (Dhumale and Bhaskar, 2021 ). As we move into the future, it is expected that the demand for food will continue to rise, driven by factors such as population growth, urbanization, and changing dietary preferences. In addition, the effects of climate change have also impacted food demand and supply, creating new challenges for the food industry (Dutta et al., 2021 ). In Springmann et al. ( 2018 ), it is mentioned that by 2050, the food chain might increase production by 50%. Besides, the FAO shows that the world population will reach approximately 10 billion by that year (Ahmed et al., 2018 ). This population increase affects the environmental conditions, which changes the harvesting process forcing farmers to use fertilizers and pesticides (Shafi et al., 2019 ). The residuals of those chemical pollutants contaminate water (Rajeshwari et al., 2021 ). Another concern is the nutritional outcome that offers food since the previous statement that the environmental condition worsens, creating floods and droughts. Therefore, humans are not receiving enough nutrients to be healthy by eating processed food, requiring pills and supplements (Mostari et al., 2021 ). The Intergovernmental Panel on Climate Change (IPCC) warns that global warming reduces the nutritional value of crops due to the intensive use of fertilizers to boost crop yields; they also predict that in the incoming years, people might suffer from zinc deficiency, causing even their psychological and cognitive disorders (Ryan et al., 2021 ).

Technology in the food production industry is a significant challenge that impedes progress and innovation in this critical sector. With the rapidly growing global population and increasing demand for food, it has become imperative to adopt technological advancements to improve food production and distribution (Ferrag et al., 2021 ). However, in many parts of the world, particularly in developing countries, technology in food production still needs to be improved, resulting in low productivity, high food losses, and reduced efficiency. Given that a big part of food production is from developing countries, exists a lack of advanced agricultural technologies (Khan et al., 2021 ). They face significant financial constraints and limited access to modern technologies, which can impede their ability to improve their food production processes. This concern also extends to the education and training of the workforce, who may not have the knowledge and skills to operate and maintain technological tools and equipment effectively (Xuan, 2021 ).

To mitigate the concerns mentioned above about food supply, FAO proposes four bullet points to guarantee food quality in the incoming years, which they closely related to the use of technology since information plays a fundamental role in ensuring the economic and sustainability impacts of new cutting-edge techniques in the food production process (Mooney, 2020 ).

Implementing emerging technologies in agriculture is often called smart farming, which aims to improve productivity, efficiency, and sustainability (Raj et al., 2021 ). In Belhadi et al. ( 2021 ), mention that smart farming might use trend technologies such as robotics, artificial intelligence, and the IoT. Therefore, these devices can gather data from crops to extract intrinsic knowledge from plants to improve agricultural decision-making and reduce environmental impact (Megeto et al., 2021 ). However, the full exploitation of the potential of smart farming presents several challenges and technical, socio-economic, and administrative constraints (Mengoli et al., 2021 ). Works such as Ahmed et al. ( 2016 ), Jawad et al. ( 2017 ), Bermeo-Almeida et al. ( 2018 ), Kamilaris and Prenafeta-Boldu ( 2018 ), and Rahmadian and Widyartono ( 2020 ) present broad approaches to smart farming and trend technologies without focusing only on robots. These studies do not include a detailed discussion of the tools and techniques used to develop the different mobile systems and their level of maturity. It is relevant to discuss the use of mobile robotics in smart farming from different perspectives and describe their corresponding nuances.

This article stands out from others of a similar nature because it offers a broad overview of the challenges and opportunities presented by precision agriculture and robotic farming. The article focuses on the use of robotics and precision agriculture in agriculture 4.0 and provides a detailed description of the many types of agricultural robots used, as well as the techniques and hardware used for their operation and monitoring. Additionally, the article highlights the areas where literature is least developed and suggests potential solutions to address these challenges. Future trends in precision agriculture and robotics are also discussed, including the use of multi-objective control algorithms and artificial intelligence in low-cost mobile robots for planning the best path while accounting for energy efficiency, soil type, and obstacles, as well as for evaluating and managing pests and diseases that affect crops.

This work aims to present an overview of mobile robotics implemented for agricultural production related to smart farming techniques. The main contribution of this work is showing the existing frameworks, tools, and applications where robots are currently used. Also, it presents shortcomings in smart farming applications, which might provide future trends in robots. The rest of the manuscript is structured as follows: Section 2 gives the smart farming background and provides a detailed overview of the leading mobile robots with existing technologies. Section 3 presents the discussion highlighting the technical and socio-economic obstacles to successfully integrating mobile robotics in agriculture. Section 4 presents the future trends related to mobile robotics in agriculture. Finally, Section 5 presents the conclusions.

2. Research methodology

A systematic literature review (SLR) was performed to manage the diverse knowledge and identify research related to the raised topic (Ahmed et al., 2016 ), especially to investigate the status of mobile robotics in precision agriculture. In particular, we searched for papers on “mobile robotics” with the term “agriculture 4.0” in the title, abstract or keywords. Prior to the SLR, a review protocol was defined to ensure a transparent, high quality and comprehensive research process (Page et al., 2021 ) including three steps: formulating the research questions, defining the search strategy, and specifying the inclusion and exclusion criteria. The preferred reporting approach for systematic reviews and meta-analyses (PRISMA) was used to conduct the SLR.

2.1. Review protocol

Before starting the bibliographic analysis, a review protocol was defined to identify, evaluate and interpret the relevant results of the research topic (see Table 1 ). The first step was to formulate research questions to identify the studies published on the subject of interest from different approaches. The appropriate keywords were then identified order to formulate search strings to obtain relevant information using four databases: IEEE Xplore, Web of Science, Scopus, and ScienceDirect. To refine the search results, inclusion and exclusion criteria were defined to evaluate the content of the publications and used as a preliminary filter of the metadata sources and limit the scope of the research.

Review protocol for SLR.

After performing the SLR, 69 research articles were obtained on the proposed topic. After the PRISMA selection and eligibility steps with the help of the Mendeley bibliographic reference manager, similar files were identified and eliminated, leaving a total 65 research papers, as can be seen in Figure 1 .

Three-steps evaluation of literature search process (PRISMA).

2.2. Trends in agriculture

The distribution of the 65 articles by year, about 38% of the most recent scientific papers were published in 2021, reflecting the considerable progress of agriculture in the context of mobile robotics, although the pace can still be considered slow compared to other domains such as healthcare, the manufacturing, the mining, the automation, the energy, among others (Araújo et al., 2021 ).

Figure 2 gives the breakdown of publications on the five most common activities carried out by agriculture 4.0 and the type of mobile robot employed. The multiple tasks in the field category include activities such as row recognition and tracking, obstacle detection and avoidance, and information gathering and reporting in both outdoor and greenhouse agriculture.

Mobile robotics activities in agriculture.

According to the International Federation of Robotics (IFR), the top five service robot applications for professional use sold during 2019 and 2020 are: transportation and logistics, professional cleaning, medical robotics, hospitality, and agriculture (International Federation of Robotics, 2021 ). Figure 3 gives the percentage of robots employed in each of these areas.

Percentage of top five applications of service robots in 2020.

3. Background and related works

Smart farming is a technique that uses advanced technology to optimize yield and efficiency in agricultural production. In Lohchab et al. ( 2018 ), explored the application of IoT technologies in smart agriculture. Subsequently, in a 2020 review article, Sharma et al. ( 2020 ) focused on the use of artificial intelligence and machine learning in smart agriculture. Furthermore, in a 2021 review article, Ratnaparkhi et al. ( 2020 ) discussed the implementation of sensor technologies and Geographic Information Systems (GIS) for smart agriculture. Finally, in a recent 2022 review article, Botta et al. ( 2022 ) examined the integration of robotics and automation in smart agriculture. Some topics that very little has been addressed in smart agriculture are: the integration of smart agriculture with the circular economy and environmental sustainability, the development and application of artificial intelligence and machine learning technologies in pest and disease identification and management, increased focus on optimizing water use and irrigation management in response to climate change and limited water availability, improved connectivity and interoperability of systems to facilitate large-scale adoption and implementation, and the development of specific low-cost solutions for small farms and rural communities in developing countries to improve food security and reduce rural poverty.

3.1. Smart farming

Smart farming is based on the information provided by sensors placed on an agricultural field (Ahmed et al., 2016 ); Machine Learning (ML) models could learn patterns to support the farmers' decision-making (Mammarella et al., 2020 ; Shorewala et al., 2021 ). These sensors joined with a microcontroller sending data constantly, are considered part of the IoT. Besides, data might be processed in big servers allocated in the cloud (cloud computing). However, IoT devices are often a rigid solution since they are placed in a single location. Therefore, Autonomous Robotic Systems (ARS) can walk around crops taking data from the whole farm and providing accurate information (Ozdogan et al., 2017 ; Kamilaris and Prenafeta-Boldu, 2018 ). This combination of sensors, data analysis, and robots provides farmers with a smart farming application with diverse tools to address challenges related to productivity, environmental impact, food safety, crop losses, and sustainability. The objectives of smart farming are to increase crop yields, minimize costs, and improve product quality through using a modern system (Araújo et al., 2021 ). In the last years, with technological evolution, different types of sensors have been developed that make it possible to collect data in almost any location, allowing real-time monitoring of agricultural fields without wiring. Therefore, the three leading technologies that contribute significantly to this field are as follows:

• Drones: These are small flying robots commonly used for crop monitoring, food infrastructure inspection, supply chain monitoring, and food safety surveillance (Costa et al., 2021 ).
• Autonomous tractors: These are generally Unmanned Ground Vehicles (UGV) incorporating sensors and actuators that enable crop monitoring, irrigation, harvesting, and disease control (Lisbinski et al., 2020 ).
• Software for decision making: These are platforms where data acquired by drones and/or UGV sensors are visualized and analyzed. They generally provide information on weather, soil, crop yields, and other factors relevant to agricultural production to improve decision-making (Ojeda-Beltran, 2022 ).

3.2. Mobile robotics in agriculture

The emerging field of agricultural mobile robotics is UGV and UAV (Prakash et al., 2020 ). The main applications of mobile robotics in farming are:

• Identify the state of the crop and corresponding application of chemical products, fumigation, or harvesting, as required by the fruit or plant.
• Mobile handling through collaborative arms (harvesting, fruit handling).
• Collection and conversion of helpful information for the farmer.
• Selective application of pesticides and avoidance of food waste.

UGV and UAV have limited available power. Therefore, their design and control optimization is paramount for their application in smart farming. Therefore, research on the cooperation between UGV and UAV is being carried out to cover large agricultural areas. These autonomous robots are intelligent machines capable of performing tasks, making decisions, and acting in real-time with a high degree of autonomy (Rahmadian and Widyartono, 2020 ). Interest in mobile robotics in agriculture has grown considerably in the last few years due to its ability to automate tasks such as planting, irrigation, fertilization, spraying, environmental monitoring, disease detection, harvesting, and weed and pest control (Araújo et al., 2021 ). Furthermore, mobile robotics in smart farming uses a combination of emerging technologies to improve the productivity and quality of agricultural products (Bechar and Vigneault, 2016 ).

UGV are robots that control can be remote (controlled by a human operator through an interface) or fully autonomous (operated without the need for a human controller based on AI technologies) (Araújo et al., 2021 ). The main components of UGV are locomotive, manipulator and supervisory control systems, sensors for navigation, and communication links for information exchange between devices. The main locomotion systems used are wheels, tracks, or legs. To properly operate UGV in the field, they must meet size, maneuverability, efficiency, human-friendly interface, and safety requirements. Table 2 summarizes the diverse range of UGVs designed for agricultural operations.

Different types of UGV in agriculture 4.0.

The main issue of mobile robotics in agricultural fields is to perform multiple tasks (obstacle avoidance, tracking, path planning, crop data collection, disease detection, among others) autonomously with reduced hardware for low-cost robots that can be acquired and implemented by farmers. Most UGV presented above have a wheeled locomotion system, offering easy construction and control. Some UGV incorporate low-cost computer vision systems, i.e., using conventional cameras. UGV might employ heuristic algorithms still in the conceptual or prototyping phase. Due to the limitations of UGV and to cover larger areas and less time, in the last years, the UGV-UAV collaboration has been developed (Khanna et al., 2015 ). The UGV operates in the areas selected by the UAV, which also cooperates in the generation of 3D maps of the environment with centimeter accuracy; however, merging the maps generated by UAV and UGV in an agricultural climate is a complex task since the generated maps present inaccuracies and scale errors due to local inconsistencies, missing data, occlusions, and global deformations (Gawel et al., 2017 ; Potena et al., 2019 ). Table 3 reviews some collaborations between UGV and UAV in smart farming.

Collaborations between UGV and UAV in smart farming.

Most collaborative systems between UAV and UGV are in the conceptual (simulation) phase.

3.3. Multi-objective control in smart farming

Agricultural systems use and produce energy in the form of bioenergy and play a vital role in the global economy and food security. Modern agricultural systems might therefore consider economic, energy, and environmental factors simultaneously (Banasik et al., 2017 ). Multi-objective control is an important tool in smart farming to simultaneously run and optimize multiple objectives, such as productivity, water use efficiency, product quality, and economic profitability. Some cases of multi-objective control in smart farming are presented in Table 4 , which shows their primary function, control techniques, and the hardware deployed. However, there are few studies since this topic is new in smart farming applications with mobile robots. Furthermore, path planning is an essential application of smart agriculture that focuses on optimizing routes and movements of agricultural machinery to improve efficiency and reduce production costs (Nazarahari et al., 2019 ).

Multi-objective control in agriculture 4.0.

Another application of multi-objective control in path planning is the optimization of fertilization and pesticide application in crops. According to a study by Zhao et al. ( 2023 ), multi-objective control can optimize the routing of pesticide and fertilizer application machinery to reduce the number of inputs used and improve application efficiency. In addition, multi-target control can also improve product quality and reduce environmental pollution by accurately applying crop inputs.

Finally, a study proposes a Residual-like Soft Actor-Critic (R-SAC) algorithm for agricultural scenarios to realize safe obstacle avoidance and intelligent path planning of robots. The study also proposes an offline expert experience pre-training method to improve the training efficiency of reinforcement learning. Experiments verify that this method has stable performance in static and dynamic obstacle environments and is superior to other reinforcement learning algorithms (Yang et al., 2022 ).

4. Discussion

With the information mentioned above about mobile robots in smart farming, this section aims to show the future steps in this research field related to its challenges. Given the new UGV and UAV trends in Table 2 , the multi-objective control has yet to be widely explored in smart farming applications. It might be due to its complex setup and the expensive computational resources needed. However, multi-objective applications might be doable in incoming robots with the increasing microcontrollers and microprocessor development. Conversely, IoT devices that collect data from farms are extensively deployed in several applications. However, there are new concerns about their confidentiality and the risk that data is exposed when traveling by communication channels (Pylianidis et al., 2021 ).

Smart farming needs final devices with robust systems working in harsh conditions in outdoor scenarios. However, several works have shown prototypes with their tentative functionalities. Building robots may need several debugging rounds to solve issues with the hardware and software. Consequently, since the robot links people and plants, farmers, considered experts in smart farming, must work closely with the robot's developer. However, the variety of plant and crop species makes it challenging to develop a multi-task robot (Selmani et al., 2019 ).

The main challenges and future research for deploying smart farming are presented. The present study sought to articulate mobile robotics with smart farming. Looking at Table 4 , it can be seen that multi-objective control has not been significantly explored in smart farming. One of the reasons could be that applying advanced technologies with complex operations can be costly. Hence, the development of these technologies in smart farming should increase in the coming years. Also, the IoT is widely deployed in agriculture for crop monitoring and tracking. Therefore, it can be said that IoT is a research trend within smart farming. However, only a few studies have considered data security and reliability, scalability, and interoperability when developing a smart farming system (Pylianidis et al., 2021 ).

The results presented also show that most of the use cases are in the prototype phase. One possible reason could be that smart farming links people, animals, and plants making it more difficult than creating systems for non-living things. Another reason could be that the technology is due to the transdisciplinarity of this field, and therefore for the development of intelligent systems, farmers should be familiar with these technologies. Finally, the variety of plant and crop species makes implementing technology in agricultural fields complex (Selmani et al., 2019 ). The results also show that most systems developed are for free-range farms. In addition, it is also evident that research is limited to soil management, fruit detection, and crop quality management. With this, it is corroborated that work must be done on research and development of systems that guarantee the deployment of smart farming at affordable costs. The natural complexity of agricultural fields presents a number of obstacles that prevent the full integration of mobile robotics in smart farming. Therefore, from the analysis, blockages at the technical and socio-economic levels have been identified and classified.

4.1. Technical roadblocks

• Interoperability . To establish effective communication between heterogeneous devices, they need to be interconnected, and interoperable (Aydin and Aydin, 2020 ).
• Dataquality . Lack of decentralized systems impedes the deployment of smart farming (Liu et al., 2022 ).
• Hardware . A suitable casing must be constructed that is robust and durable enough to withstand actual field conditions (Villa-Henriksen et al., 2020 ).
• Power sources . A proper energy-saving scheme is necessary as instant battery replacement is complicated. A possible solution to optimize power consumption is using low-power hardware and proper communications management (Jawad et al., 2017 ).
• Wireless architectures . Wireless communication networks and technologies offer several advantages in terms of low cost, wide area coverage, network flexibility, and high scalability (Brinis and Saidane, 2016 ).
• Security . The nature of agricultural fields leads to risks to data privacy, integrity, and availability (Chen et al., 2017 ).
• User interface . Most graphical user interfaces are designed so that only experts can use them (Del Cerro et al., 2021 ).

4.2. Socio-economic roadblocks

• Costs . Costs associated with adopting robotic technologies and systems are the biggest drawback to deploying smart farming (Sinha and Dhanalakshmi, 2022 ).
• Return on investment . When implementing new technologies, farmers are concerned about the payback time and the difficulties in assessing the benefits (Miranda et al., 2019 ).
• Gap between farmers and researchers . Farmer involvement is paramount to the success of smart farming. Farmers face many problems during the production process that technology could solve (Bacco et al., 2019 ).

Finally, in Charatsari et al. ( 2022 ) discusses the importance of responsibility in the process of technological innovation in the agrifood industry. It highlights the need to consider not only technical aspects but also social implications and societal values when introducing innovative technologies. The authors argue that the perception of responsible innovation is limited in various industrial sectors, making it challenging to implement responsible innovation approaches. The complexity of responsible innovation in the agrifood industry requires addressing the multiple scales and levels of interaction between actors and the constant evolution of agrifood systems. Therefore, the article emphasizes the need to adopt responsible innovation practices that consider the social, ethical, and environmental implications of technological innovations in the industry.

5. Future trends

The upcoming initiatives related to using robots represent significant improvements in smart farming. Government initiatives, public-private sectors, and research work in this field might contribute to establishing the right conditions to add new hardware to crops. However, there are some challenges to consider when developing mobile robots in agriculture such as: navigation on uneven terrain (loose soil and unpredictable obstacles) without damaging plants or compromising their own safety, energy efficiency so that they can operate for long periods of time avoiding constant human intervention, crop manipulation, integration with farm management systems and adaptability to different crops and conditions.

For instance, a robotic system can be developed for smart farming, starting from a basic architecture with few components and simple functionality that allows the gradual addition of features and functionality to create a complex system. Future trends in smart farming involve using multi-objective control algorithms and artificial intelligence in low-cost mobile robots to plan the best trajectory considering energy efficiency, soil type, and obstacles while monitoring crop growth and assessing and controlling crop pests and diseases. To ensure good connectivity and live transmission of crop data, 5G technology needs to be widely explored. 5G technology minimizes internet costs and increases information management by remotely performing accurate inspections of agricultural fields (Abbasi et al., 2021 ). Finally, blockchain, combined with IoT and other technologies, should be applied to address the challenge of information privacy and security (Bermeo-Almeida et al., 2018 ).

As seen in the tables in the previous sections, most of the UGV have a computer, which increases the cost of this type of robots. Table 5 shows several state-of-the-art boards that could deploy smart farming at affordable prices for farmers.

Boards for agriculture 4.0.

Finally, in Figure 4 , we can see the future of agriculture, for which a correct 5G network deployment and path planning/tracking is essential. Artificial intelligence, machine learning, machine vision, IoT, and cloud computing are needed in each of the activities carried out in agricultural fields.

Future of agriculture.

6. Conclusions

Growing concerns about global food security have accelerated the need to incorporate mobile robots in agriculture. The scientific community and researchers are integrating disruptive technologies into conventional agricultural systems to increase crop quality and yields, minimize costs, and reduce waste generation. This article analyzes the current state and challenges of smart farming. Considering the impact of farming on climate change and healthy food production, it is vital to provide the agricultural sector with low-cost, functional mobile robots. Research questions were posed and answered regarding the use of mobile robotics in agriculture, the technologies, methods, and tools used in agricultural fields, and the main challenges of multi-target control in this area. Several conclusions were drawn, such as the integration of scalable mobile robots incorporating efficient systems. It should be noted that most cases address a specific problem and are in the prototype phase.

From the SLR conducted, it was identified that research on the following topics is limited:

• The implementation of digital twins for robot-based production lines
• Ingenious software project management while narrowing the impact aspect.
• Blockchain in agriculture.
• Context-aware wireless sensor network suitable for precision agriculture.
• Internet of Things (IoT) for smart precision agriculture and farming in rural areas.
• Semantic and syntactic interoperability for agricultural open-data platforms in the context of IoT using crop-specific trait.
• Multi-objective path planner for an agricultural mobile robot in a virtual and real greenhouse environment.
• Closing loops in agricultural supply chains using multi-objective optimization.
• New control approaches for trajectory tracking and motion planning of unmanned tracked robots.

These areas require further research to improve the efficiency and effectiveness of precision agriculture. Likewise, the information gathered in this article makes it clear that the emerging fields of research are:

• Autonomous navigation . Planning, tracking of trajectories, and task planning should be considered in this area.
• Energy efficiency . Good navigation autonomy is not the only thing that must be taken into account, but also the design and all components that make up the mobile robot since its size and cost directly influence the deployment of smart farming.
• Communication . Due to the number of devices involved in smart farming, middleware that improves communication between field devices and the station is important to ensure the reliability and security of information.

The interdependence of these challenges means that a practical solution must be sought with a suitable compromise between the theoretically optimal path that facilitates information exchange and overall system energy optimization. Moreover, the following questions must be considered: the kinematic and dynamic design of the mobile robot, the terrain traversability, the computational complexity of the various algorithms to ensure real-time performance, the use of sensors and low-energy control boards, and the sending and receiving of information. It also identifies the leading technical and socioeconomic obstacles that must be overcome to deploy smart farming successfully. We can see leaps and bounds being made in this area, but there is still a long way to go to mitigate the impact of farming on the environment in the coming years. Finally, one of the areas to be investigated is multi-objective heuristic optimization for autonomous navigation, communication, and energy efficiency of mobile robots.

Finally, numerous international political organizations play a crucial role in spreading awareness of the technologies involved in precision agriculture and advocating for their successful implementation. These organizations are:

• The FAO promotes the use of advanced agricultural technologies through programs and projects, providing technical assistance, training, and resource access for farmers.
• The European Union (EU) supports agricultural modernization and the adoption of innovative technologies in the industry through its Agricultural Common Policy (ACP). Additionally, the UE funds research and development projects in precision agriculture, agricultural robotics, and digital solutions to increase efficiency and sustainability.
• The Department of Agriculture (USDA) of the United States places emphasis on the adoption of cutting-edge agricultural technologies. The USDA supports the implementation of precise agriculture systems, the integration of sensors and IoT devices into agricultural operations, and the promotion of digitalization in the industry through its funding and grant program.
• The focus of AGRA is to encourage the use of contemporary agricultural technologies across the African continent. AGRA works in close partnership with governments, regional organizations, and the private sector to increase access to and availability of improved seeds, fertilizers, and digital farming technologies that boost agricultural productivity and sustainability.
• The World Economic Forum (WEF) has established initiatives and projects to advance precision agriculture. The WEF brings together many actors-including political leaders, business executives, and members of civil society-through its platform “Shaping the Future of Food Security and Agriculture” to develop innovative and collaborative solutions that foster the digital transformation of agriculture.

These political organizations play a crucial role in the spread of advanced agricultural technologies, and they are actively working to promote the adoption of “agriculture 4.0” on a global scale with the aim of enhancing the efficiency, productivity, and sustainability of the agricultural sector.

Author contributions

JVS supervised this project. PR-M and JS contributed in this project. DY-P made the first version of the article under the guidelines of the other authors; likewise, he made the corrections to the observations made by the reviewers and shared them with the other authors for their respective review and subsequent approval. All authors contributed to the article and approved the submitted version.

Funding Statement

This work was supported by Generalitat Valenciana regional government through project CIAICO/2021/064.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

• Abbasi R., Yanes A. R., Villanuera E. M., Ahmad R. (2021). “Real-time implementation of digital twin for robot based production line,” in Proceedings of the Conference on Learning Factories (CLF) (Elsevier: ), 55–60. [ Google Scholar ]
• Ahmed M. A., Ahsan I., Abbas M. (2016). “Systematic literature review: ingenious software project management while narrowing the impact aspect,” in Proceedings of the International Conference on Research in Adaptive and Convergent Systems, RACS '16 (New York, NY: Association for Computing Machinery; ), 165–168. [ Google Scholar ]
• Ahmed N., De D., Hussain I. (2018). Internet of things (IoT) for smart precision agriculture and farming in rural areas . IEEE Internet Things J. 5 , 4890–4899. 10.1109/JIOT.2018.2879579 [ CrossRef ] [ Google Scholar ]
• Araújo S. O., Peres R. S., Barata J., Lidon F., Ramalho J. C. (2021). Characterising the agriculture 4.0 landscape—emerging trends, challenges and opportunities . Agronomy 11 , 1–37. 10.3390/agronomy11040667 [ CrossRef ] [ Google Scholar ]
• Arindam S., Anjan K. R., Arun-Baran S. (2018). “Grid-based UGV navigation in a dynamic environment using neural network,” in 2018 International Conference on Inventive Research in Computing Applications (ICIRCA) , 509–514. 10.1109/ICIRCA.2018.8597389 [ CrossRef ] [ Google Scholar ]
• Aydin S., Aydin M. N. (2020). Semantic and syntactic interoperability for agricultural open-data platforms in the context of IoT using crop-specific trait ontologies . Appl. Sci. 10 , 1–27. 10.3390/app10134460 [ CrossRef ] [ Google Scholar ]
• Azimi-Mahmud M. S., Zainal-Abidin M. S., Mohamed Z., Iida M. (2019). Multi-objective path planner for an agricultural mobile robot in a virtual greenhouse environment . Comput. Electron. Agric. 157 , 488–499. 10.1016/j.compag.2019.01.016 [ CrossRef ] [ Google Scholar ]
• Bacco M., Barsocchi P., Ferro E., Gotta A., Ruggeri M. (2019). The digitisation of agriculture: a survey of research activities on smart farming . Array 3 , 100009. 10.1016/j.array.2019.100009 [ CrossRef ] [ Google Scholar ]
• Bacheti V. P., Brandao A. S., Sarcinelli-Filho M. (2021). “Path-following by a UGV-UAV formation based on null space,” in 14th IEEE International Conference on Industry Applications (INDUSCON) , 1266–1273. 10.1109/INDUSCON51756.2021.9529472 [ CrossRef ] [ Google Scholar ]
• Banasik A., Kanellopoulos A., Claassen G., Bloemhof-Ruwaard J. (2017). Closing loops in agricultural supply chains using multi-objective optimization: a case study of an industrial mushroom supply chain . Int. J. Product. Econ. 183 , 409–420. 10.1016/j.ijpe.2016.08.012 [ CrossRef ] [ Google Scholar ]
• Banihani S., Hayajneh M., Al-Jarrah A., Mutawe S. (2021). New control approaches for trajectory tracking and motion planning of unmanned tracked robot . Adv. Electric. Electron. Eng. 19 , 42–56. 10.15598/aeee.v19i1.4006 [ CrossRef ] [ Google Scholar ]
• Bawden O., Kulk J., Russell R., McCool C., English A., Dayoub F., et al.. (2017). Robot for weed species plant-specific management . J. Field Robot. 34 , 1179–1199. 10.1002/rob.21727 [ CrossRef ] [ Google Scholar ]
• Bechar A., Vigneault C. (2016). Agricultural robots for field operations: concepts and components . Biosyst. Eng. 149 , 94–111. 10.1016/j.biosystemseng.2016.06.014 [ CrossRef ] [ Google Scholar ]
• Belhadi A., Kamble S. S., Mani V., Benkhati I., Touriki F. E. (2021). An ensemble machine learning approach for forecasting credit risk of agricultural SMEs' investments in agriculture 4.0 through supply chain finance . Ann. Operat. Res. 1–29. 10.1007/s10479-021-04366-9 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Berenstein R., Edan Y. (2018). Automatic adjustable spraying device for site-specific agricultural application . IEEE Trans. Automat. Sci. Eng. 15 , 641–650. 10.1109/TASE.2017.2656143 [ CrossRef ] [ Google Scholar ]
• Bermeo-Almeida O., Cardenas-Rodriguez M., Samaniego-Cobo T., Ferruzola-Gómez E., Cabezas-Cabezas R., Bazán-Vera W. (2018). “Blockchain in agriculture: a systematic literature review,” in International Conference on Technologies and Innovation (Cham: Springer International Publishing; ), 44–56. [ Google Scholar ]
• Birrell S., Hughes J., Cai J. Y., Lida F. (2020). A field-tested robotic harvesting system for iceberg lettuce . J. Field Robot. 37 , 225–245. 10.1002/rob.21888 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Botta A., Cavallone P., Baglieri L., Colucci G., Tagliavini L., Quaglia G. (2022). A review of robots, perception, and tasks in precision agriculture . Appl. Mech. 3 , 830–854. 10.3390/applmech3030049 [ CrossRef ] [ Google Scholar ]
• Brinis N., Saidane L. A. (2016). Context aware wireless sensor network suitable for precision agriculture . Wireless Sensor Netw. 8 , 1–12. 10.4236/wsn.2016.81001 [ CrossRef ] [ Google Scholar ]
• Changho Y., Hak-Jin K., Chan-Woo J., Minseok G., Won S. L., Jong G. H. (2021). Stereovision-based ridge-furrow detection and tracking for auto-guided cultivator . Comput. Electron. Agric. 191 , 106490. 10.1016/j.compag.2021.106490 [ CrossRef ] [ Google Scholar ]
• Charatsari C., Lioutas E. D., De Rosa M., Vecchio Y. (2022). Technological innovation and agrifood systems resilience: the potential and perils of three different strategies . Front. Sustain. Food Syst. 6 , 872706. 10.3389/fsufs.2022.872706 [ CrossRef ] [ Google Scholar ]
• Chen L., Thombre S., Jarvinen K., Lohan E. S., Alén-Savikko A., Leppakoski H., Bhuiyan M. Z. H., et al.. (2017). Robustness, security and privacy in location-based services for future IoT: a survey . IEEE Access 5 , 8956–8977. 10.1109/ACCESS.2017.2695525 [ CrossRef ] [ Google Scholar ]
• Chirala V. S., Venkatachalam S., Smereka J. M., Kassoumeh S. (2021). A multi-objective optimization approach for multi-vehicle path planning problems considering human–robot interactions . J. Auton. Vehicles Syst. 1 , 041002. 10.1115/1.4053426 [ CrossRef ] [ Google Scholar ]
• Costa E., Martins M. B., Vendruscolo E. P., Silva A. G., Zoz T., Binotti F. F. S., et al.. (2021). Greenhouses within the agricultura 4.0 interface . Revista Ciência Agronômica , (Universidade Federal do Ceará) 51, e20207703. 10.5935/1806-6690.20200089 [ CrossRef ] [ Google Scholar ]
• Del Cerro J., Cruz-Ulloa C., Barrientos A., De León-Rivas J. (2021). Unmanned aerial vehicles in agriculture: a survey . Agronomy 11 , 1–19. 10.3390/agronomy11020203 [ CrossRef ] [ Google Scholar ]
• Dhumale N. R., Bhaskar P. C. (2021). “Smart agricultural robot for spraying pesticide with image processing based disease classification technique,” in 2021 International Conference on Emerging Smart Computing and Informatics, ESCI 2021 , 604–609. 10.1109/ESCI50559.2021.9396959 [ CrossRef ] [ Google Scholar ]
• Dutta A., Roy S., Kreidl O. P., Bölöni L. (2021). Multi-robot information gathering for precision agriculture: current state, scope, and challenges . IEEE Access 9 , 161416–161430. 10.1109/ACCESS.2021.3130900 [ CrossRef ] [ Google Scholar ]
• Edmonds M., Yigit T., Yi J. (2021). “Resolution-optimal, energy-constrained mission planning for unmanned aerial/ground crop inspections,” in IEEE 17th International Conference on Automation Science and Engineering (CASE) , 2235–2240. [ Google Scholar ]
• Ferrag M. A., Shu L., Djallel H., Choo K.-K. R. (2021). Deep learning-based intrusion detection for distributed denial of service attack in agriculture 4.0 . Electronics 10 , 1257. 10.3390/electronics10111257 [ CrossRef ] [ Google Scholar ]
• Gai J., Tang L., Steward B. L. (2020). Automated crop plant detection based on the fusion of color and depth images for robotic weed control . J. Field Robot. 37 , 35–52. 10.1002/rob.21897 [ CrossRef ] [ Google Scholar ]
• Galán-Martín A., Vaskan P., Antón A., Jiménez-Esteller L., Guillén-Gosálbez G. (2017). Multi-objective optimization of rainfed and irrigated agricultural areas considering production and environmental criteria: a case study of wheat production in Spain . J. Clean. Product. 140 , 816–830. 10.1016/j.jclepro.2016.06.099 [ CrossRef ] [ Google Scholar ]
• Gawel A., Dubé R., Surmann H., Nieto J. I., Siegwart R. Y., Cadena C. (2017). “3D registration of aerial and ground robots for disaster response: an evaluation of features, descriptors, and transformation estimation,” in IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR) , 27–34. 10.1109/SSRR.2017.8088136 [ CrossRef ] [ Google Scholar ]
• Gentilini L., Rossi S., Mengoli D., Eusebi A., Marconi L. (2021). “Trajectory planning ROS service for an autonomous agricultural robot,” in 2021 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor) , 384–389. 10.1109/MetroAgriFor52389.2021.9628620 [ CrossRef ] [ Google Scholar ]
• International Federation of Robotics (2021). World Robotics 2021 . [ Google Scholar ]
• Jain S., Ramesh D., Bhattacharya D. (2021). A multi-objective algorithm for crop pattern optimization in agriculture . Appl. Soft Comput. 112 , 107772. 10.1016/j.asoc.2021.107772 [ CrossRef ] [ Google Scholar ]
• Jawad H. M., Nordin R., Gharghan S. K., Jawad A. M., Ismail M. (2017). Energy-efficient wireless sensor networks for precision agriculture: a review . Sensors 17 , 1–45. 10.3390/s17081781 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kamilaris A., Prenafeta-Boldu F. X. (2018). A review of the use of convolutional neural networks in agriculture . J. Agric. Sci. 156 , 312–322. 10.1017/S0021859618000436 [ CrossRef ] [ Google Scholar ]
• Khan N., Ray R. L., Sargani G. R., Ihtisham M., Khayyam M., Ismail S. (2021). Current progress and future prospects of agriculture technology: Gateway to sustainable agriculture . Sustainability 13 , 1–31. 10.3390/su13094883 [ CrossRef ] [ Google Scholar ]
• Khan S., Guivant J., Li X. (2022). Design and experimental validation of a robust model predictive control for the optimal trajectory tracking of a small-scale autonomous bulldozer . Robot. Auton. Syst. 147 , 103903. 10.1016/j.robot.2021.103903 [ CrossRef ] [ Google Scholar ]
• Khanna R., Möller M., Pfeifer J., Liebisch F., Walter A., Siegwart R. (2015). “Beyond point clouds - 3D mapping and field parameter measurements using UAVs,” in IEEE 20th Conference on Emerging Technologies Factory Automation (ETFA) , 1–4. [ Google Scholar ]
• Kim J., Son H. I. (2020). A voronoi diagram-based workspace partition for weak cooperation of multi-robot system in orchard . IEEE Access 8 , 20676–20686. 10.1109/ACCESS.2020.2969449 [ CrossRef ] [ Google Scholar ]
• Li M., Fu Q., Singh V. P., Liu D., Li T. (2019). Stochastic multi-objective modeling for optimization of water-food-energy nexus of irrigated agriculture . Adv. Water Resour. 127 , 209–224. 10.1016/j.advwatres.2019.03.015 [ CrossRef ] [ Google Scholar ]
• Li Y., Yu J., Guo X., Sun J. (2020). “Path tracking method of unmanned agricultural vehicle based on compound fuzzy control,” in 2020 IEEE 9th Joint International Information Technology and Artificial Intelligence Conference (ITAIC) , 1301–1305. 10.1109/ITAIC49862.2020.9338981 [ CrossRef ] [ Google Scholar ]
• Liang X., Zhao S., Chen G., Meng G., Wang Y. (2021). Design and development of ground station for UAV/UGV heterogeneous collaborative system . Ain Shams Eng. J. 12 , 3879–3889. 10.1016/j.asej.2021.04.025 [ CrossRef ] [ Google Scholar ]
• Lisbinski F. C., Mühl D. D., Oliveira L. d., Coronel D. A. (2020). Perspectivas e desafios da agricultura 4.0 para o setor agrícola . Anais.[do] VIII Simpósio da Ciência do Agronegócio . 422–433. [ Google Scholar ]
• Liu J., Anavatti S., Garratt M., Abbass H. A. (2022). Modified continuous ant colony optimisation for multiple unmanned ground vehicle path planning . Expert Syst. Appl. 196 , 116605. 10.1016/j.eswa.2022.116605 [ CrossRef ] [ Google Scholar ]
• Lohchab V., Kumar M., Suryan G., Gautam V., Das R. K. (2018). “A review of IoT based smart farm monitoring,” in 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT) , 1620–1625. [ Google Scholar ]
• Luo Y., Li J., Yu C., Xu B., Li Y., Hsu L.-T., et al.. (2019). Research on time-correlated errors using allan variance in a Kalman filter applicable to vector-tracking-based GNSS software-defined receiver for autonomous ground vehicle navigation . Remote Sens. 11 , 1–39. 10.3390/rs11091026 [ CrossRef ] [ Google Scholar ]
• Mac T. T., Copot C., Tran D. T., De Keyser R. (2017). A hierarchical global path planning approach for mobile robots based on multi-objective particle swarm optimization . Appl. Soft Comput. 59 , 68–76. 10.1016/j.asoc.2017.05.012 [ CrossRef ] [ Google Scholar ]
• Mammarella M., Comba L., Biglia A., Dabbene F., Gay P. (2020). “Cooperative agricultural operations of aerial and ground unmanned vehicles,” in 2020 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor) , 224–229. [ Google Scholar ]
• Megeto G. A. S., Silva A. G., Bulgarelli R. F., Bublitz C. F., Valente A. C., Costa D. A. G. (2021). Artificial intelligence applications in the agriculture 4.0 . Revista Ciência Agronômica , (Universidade Federal do Ceará) 51, e20207701. 10.5935/1806-6690.20200084 [ CrossRef ] [ Google Scholar ]
• Mengoli D., Eusebi A., Rossi S., Tazzari R., Marconi L. (2021). “Robust autonomous row-change maneuvers for agricultural robotic platform,” in 2021 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor) , 390–395. [ Google Scholar ]
• Miranda J., Ponce P., Molina A., Wright P. (2019). Sensing, smart and sustainable technologies for Agri-Food 4.0 . Comput. Industry 108 , 21–36. 10.1016/j.compind.2019.02.002 [ CrossRef ] [ Google Scholar ]
• Mooney P. (2020). La insostenible agricultura 4.0 digitalización y poder corporativo en la cadena alimentaria . [ Google Scholar ]
• Mostari A., Benabdeli K., Ferah T. (2021). Assessment of the impact of urbanisation on agricultural and forest areas in the coastal zone of Mostaganem (Western Algeria) . Ekologia 40 , 230–239. 10.2478/eko-2021-0025 [ CrossRef ] [ Google Scholar ]
• Nazarahari M., Khanmirza E., Doostie S. (2019). Multi-objective multi-robot path planning in continuous environment using an enhanced genetic algorithm . Expert Syst. Appl. 115 , 106–120. 10.1016/j.eswa.2018.08.008 [ CrossRef ] [ Google Scholar ]
• Nerlekar V., Mamtura T., Parihar S. (2022). “Implementation of A* algorithm for optimal path planning for mobile robots,” in 2022 4th International Conference on Smart Systems and Inventive Technology (ICSSIT) , 382–390. 10.1109/ICSSIT53264.2022.971649 [ CrossRef ] [ Google Scholar ]
• Ojeda-Beltran A. (2022). Plataformas tecnologicas en la agricultura 4.0: una mirada al desarrollo en colombia . Comput. Electron. Sci. Theory Appl. 3 , 9–18. 10.17981/cesta.03.01.2022.02 [ CrossRef ] [ Google Scholar ]
• Ozdogan B., Gacar A., Aktas H. (2017). Digital agriculture practices in the context of agriculture 4.0 . JEFA 4 , 184–191. 10.17261/Pressacademia.2017.448 [ CrossRef ] [ Google Scholar ]
• Page M. J., McKenzie J. E., Bossuyt P. M., Boutron I., Hoffmann T. C., Mulrow C. D., et al.. (2021). Declaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas . Revista Española de Cardiología 74 , 790–799. 10.1016/j.rec.2021.07.010 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pak J., Kim J., Park Y., Son H. I. (2022). Field evaluation of path-planning algorithms for autonomous mobile robot in smart farms . IEEE Access 10 , 60253–60266. 10.1109/ACCESS.2022.3181131 [ CrossRef ] [ Google Scholar ]
• Potena C., Khanna R., Nieto J., Siegwart R., Nardi D., Pretto A. (2019). AgriColMap: aerial-ground collaborative 3D mapping for precision farming . IEEE Robot. Automat. Lett. 4 , 1085–1092. 10.1109/LRA.2019.2894468 [ CrossRef ] [ Google Scholar ]
• Prakash R., Dheer D. K., Kumar M. (2020). “Path planning of UGV using sampling-based method and PSO in 2D map configuration: a comparative analysis,” in 2020 International Conference on Emerging Frontiers in Electrical and Electronic Technologies (ICEFEET) , 1–6. [ Google Scholar ]
• Pylianidis C., Osinga S., Athanasiadis L. N. (2021). Introducing digital twins to agriculture . Comput. Electron. Agric. 184 , 105942. 10.1016/j.compag.2020.105942 [ CrossRef ] [ Google Scholar ]
• Quaglia G., Cavallone P., Visconte C. (2018). “Agri_q: agriculture UGV for monitoring and drone landing,” in IFToMM Symposium on Mechanism Design for Robotics (Cham: Springer International Publishing; ), 413–423. [ Google Scholar ]
• Radmanesh M., Sharma B., Kumar M., French D. (2021). PDE solution to UAV/UGV trajectory planning problem by spatio-temporal estimation during wildfires . Chin. J. Aeronaut. 34 , 601–616. 10.1016/j.cja.2020.11.002 [ CrossRef ] [ Google Scholar ]
• Rahmadian R., Widyartono M. (2020). “Autonomous robotic in agriculture: a review,” in 2020 Third International Conference on Vocational Education and Electrical Engineering (ICVEE) (IEEE: ), 1–6. 10.1109/ICVEE50212.2020.9243253 [ CrossRef ] [ Google Scholar ]
• Raj M., Gupta S., Chamola V., Elhence A., Garg T., Atiquzzaman M., et al.. (2021). A survey on the role of internet of things for adopting and promoting agriculture 4.0 . J. Netw. Comput. Appl. 187 , 103107. 10.1016/j.jnca.2021.103107 [ CrossRef ] [ Google Scholar ]
• Rajeshwari T., Vardhini P. H., Reddy K. M. K., Priya K. K., Sreeja K. (2021). “Smart agriculture implementation using IoT and leaf disease detection using logistic regression,” in 2021 4th International Conference on Recent Developments in Control, Automation & Power Engineering (RDCAPE) (IEEE: ), 619–623. 10.1109/RDCAPE52977.2021.9633608 [ CrossRef ] [ Google Scholar ]
• Ratnaparkhi S., Khan S., Arya C., Khapre S., Singh P., Diwakar M., et al.. (2020). “Withdrawn: smart agriculture sensors in IoT: a review,” in Materials Today: Proceedings . [ Google Scholar ]
• Romeo L., Petitti A., Colella R., Valecce G., Boccadoro P., Milella A., et al.. (2020). “Automated deployment of IoT networks in outdoor scenarios using an unmanned ground vehicle,” in 2020 IEEE International Conference on Industrial Technology (ICIT) (IEEE: ), 369–374. 10.1109/ICIT45562.2020.9067099 [ CrossRef ] [ Google Scholar ]
• Rucco A., Sujit P., Aguiar A. P., De Sousa J. B., Pereira F. L. (2017). Optimal rendezvous trajectory for unmanned aerial-ground vehicles . IEEE Trans. Aerospace Electron. Syst. 54 , 834–847. 10.1109/TAES.2017.2767958 [ CrossRef ] [ Google Scholar ]
• Ryan S. J., Carlson C. J., Tesla B., Bonds M. H., Ngonghala C. N., Mordecai E. A., et al.. (2021). Warming temperatures could expose more than 1.3 billion new people to Zika virus risk by 2050 . Glob. Change Biol. 27 , 84–93. 10.1111/gcb.15384 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Selmani A., Oubehar H., Outanoute M., Ed-Dahhak A., Guerbaoui M., Lachhab A., et al.. (2019). Agricultural cyber-physical system enabled for remote management of solar-powered precision irrigation . Biosyst. Eng. 177 , 18–30. 10.1016/j.biosystemseng.2018.06.007 [ CrossRef ] [ Google Scholar ]
• Shafi U., Mumtaz R., García-Nieto J., Hassan S. A., Zaidi S. A. R., Iqbal N. (2019). Precision agriculture techniques and practices: from considerations to applications . Sensors 19 , 1–25. 10.3390/s19173796 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Shamshirband S., Khoshnevisan B., Yousefi M., Bolandnazar E., Anuar N. B., Wahid A., et al.. (2015). A multi-objective evolutionary algorithm for energy management of agricultural systems—A case study in Iran . Renew. Sustain. Energy Rev. 44 , 457–465. 10.1016/j.rser.2014.12.038 [ CrossRef ] [ Google Scholar ]
• Sharma A., Jain A., Gupta P., Chowdary V. (2020). Machine learning applications for precision agriculture: a comprehensive review . IEEE Access 9 , 4843–4873. 10.1109/ACCESS.2020.3048415 [ CrossRef ] [ Google Scholar ]
• Shorewala S., Ashfaque A., Sidharth R., Verma U. (2021). Weed density and distribution estimation for precision agriculture using semi-supervised learning . IEEE Access 9 , 27971–27986. 10.1109/ACCESS.2021.3057912 [ CrossRef ] [ Google Scholar ]
• Sinha B. B., Dhanalakshmi R. (2022). Recent advancements and challenges of Internet of Things in smart agriculture: a survey . Future Gen. Comput. Syst. 126 , 169–184. 10.1016/j.future.2021.08.006 [ CrossRef ] [ Google Scholar ]
• Springmann M., Clark M., Mason-D'croz D., Wiebe K., Bodirsky B. L., Lassaletta L., et al.. (2018). Options for keeping the food system within environmental limits . Nature 562 , 519–525. 10.1038/s41586-018-0594-0 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Srinivas A., Sangeetha J. (2021). Smart irrigation and precision farming of paddy field using unmanned ground vehicle and internet of things system . Int. J. Adv. Comput. Sci. Appl. 12 , 407–414. 10.14569/IJACSA.2021.0121254 [ CrossRef ] [ Google Scholar ]
• Sun Y. P., Liang Y. C. (2022). “Vector field path-following control for a small unmanned ground vehicle with Kalman filter estimation,” in Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture 236 , 1885–1899. 10.1177/0954405420977347 [ CrossRef ] [ Google Scholar ]
• Tazzari R., Mengoli D., Marconi L. (2020). “Design concept and modelling of a tracked UGV for orchard precision agriculture,” in 2020 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor) , 207–212. [ Google Scholar ]
• Tsiogkas N., Lane D. M. (2018). An evolutionary algorithm for online, resource-constrained, multivehicle sensing mission planning . IEEE Robot. Automat. Lett. 3 , 1199–1206. 10.1109/LRA.2018.2794578 [ CrossRef ] [ Google Scholar ]
• Villa-Henriksen A., Edwards G. T., Pesonen L. A., Green O., Sórensen C. A. G. (2020). Internet of Things in arable farming: implementation, applications, challenges and potential . Biosyst. Eng. 191 , 60–84. 10.1016/j.biosystemseng.2019.12.013 [ CrossRef ] [ Google Scholar ]
• Wang T., Huang P., Dong G. (2021). Modeling and path planning for persistent surveillance by unmanned ground vehicle . IEEE Trans. Automat. Sci. Eng. 18 , 1615–1625. 10.1109/TASE.2020.3013288 [ CrossRef ] [ Google Scholar ]
• Xie J., Chen J. (2020). “Multi-regional coverage path planning for robots with energy constraint,” in 2020 IEEE 16th International Conference on Control & Automation (ICCA) , 1372–1377. 10.1109/ICCA51439.2020.9264472 [ CrossRef ] [ Google Scholar ]
• Xuan B. B. (2021). Consumer preference for eco-labelled aquaculture products in vietnam . Aquaculture 532 , 736111. 10.1016/j.aquaculture.2020.736111 [ CrossRef ] [ Google Scholar ]
• Yang J., Ni J., Li Y., Wen J., Chen D. (2022). The intelligent path planning system of agricultural robot via reinforcement learning . Sensors 22 , 1–19. 10.3390/s22124316 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zhai Z., Martínez-Ortega J.-F., Lucas-Martínez N., Rodríguez-Molina J. (2018). A mission planning approach fors precision farming systems based on multi-objective optimization . Sensors 18 , 1–32. 10.3390/s18061795 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zhang K., Yang Y., Fu M., Wang M. (2019). Traversability assessment and trajectory planning of unmanned ground vehicles with suspension systems on rough terrain . Sensors 19 , 1–28. 10.3390/s19204372 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zhao J., Yu Y., Lei J., Liu J. (2023). Multi-objective lower irrigation limit simulation and optimization model for Lycium barbarum based on NSGA-III and ANN . Water 15 , 1–16. 10.3390/w15040783 [ CrossRef ] [ Google Scholar ]
• Zhao Y., Ding F., Li J., Guo L., Qi W. (2019). The intelligent obstacle sensing and recognizing method based on D–S evidence theory for UGV . Future Gen. Comput. Syst. 97 , 21–29. 10.1016/j.future.2019.02.003 [ CrossRef ] [ Google Scholar ]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 08 July 2020

A mobile robotic chemist

• Benjamin Burger 1 ,
• Phillip M. Maffettone   ORCID: orcid.org/0000-0001-7173-7972 1 ,
• Vladimir V. Gusev 1 ,
• Catherine M. Aitchison   ORCID: orcid.org/0000-0003-1437-8314 1 ,
• Yang Bai 1 ,
• Xiaoyan Wang 1 ,
• Xiaobo Li 1 ,
• Ben M. Alston 1 ,
• Buyi Li 1 ,
• Rob Clowes 1 ,
• Nicola Rankin 1 ,
• Brandon Harris   ORCID: orcid.org/0000-0003-4881-6220 1 ,
• Reiner Sebastian Sprick   ORCID: orcid.org/0000-0002-5389-2706 1 &
• Andrew I. Cooper 1  

Nature volume  583 ,  pages 237–241 ( 2020 ) Cite this article

84k Accesses

638 Citations

1154 Altmetric

Metrics details

• Materials science
• Renewable energy
• Techniques and instrumentation

Technologies such as batteries, biomaterials and heterogeneous catalysts have functions that are defined by mixtures of molecular and mesoscale components. As yet, this multi-length-scale complexity cannot be fully captured by atomistic simulations, and the design of such materials from first principles is still rare 1 , 2 , 3 , 4 , 5 . Likewise, experimental complexity scales exponentially with the number of variables, restricting most searches to narrow areas of materials space. Robots can assist in experimental searches 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 but their widespread adoption in materials research is challenging because of the diversity of sample types, operations, instruments and measurements required. Here we use a mobile robot to search for improved photocatalysts for hydrogen production from water 15 . The robot operated autonomously over eight days, performing 688 experiments within a ten-variable experimental space, driven by a batched Bayesian search algorithm 16 , 17 , 18 . This autonomous search identified photocatalyst mixtures that were six times more active than the initial formulations, selecting beneficial components and deselecting negative ones. Our strategy uses a dexterous 19 , 20 free-roaming robot 21 , 22 , 23 , 24 , automating the researcher rather than the instruments. This modular approach could be deployed in conventional laboratories for a range of research problems beyond photocatalysis.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

A robotic prebiotic chemist probes long term reactions of complexifying mixtures

Silke Asche, Geoffrey J. T. Cooper, … Leroy Cronin

Automated synthesis of oxygen-producing catalysts from Martian meteorites by a robotic AI chemist

Qing Zhu, Yan Huang, … Yi Luo

Navigating phase diagram complexity to guide robotic inorganic materials synthesis

Jiadong Chen, Samuel R. Cross, … Wenhao Sun

Data availability

The implementation of the liquid-dispensing station, photolysis station and the workflow, along with three-dimensional designs for labware developed in the project, are available at https://bitbucket.org/ben_burger/kuka_workflow , the code for the robot at and the Bayesian optimizer is available at https://github.com/Taurnist/kuka_workflow_tantalus and https://github.com/CooperComputationalCaucus/kuka_optimizer . Additional design details can be obtained from the authors upon request.

Kang, K., Meng, Y. S., Bréger, J., Grey, C. P. & Ceder, G. Electrodes with high power and high capacity for rechargeable lithium batteries. Science 311 , 977–980 (2006).

Article   ADS   CAS   Google Scholar  

Woodley, S. M. & Catlow, R. Crystal structure prediction from first principles. Nat. Mater . 7 , 937–946 (2008).

Gómez-Bombarelli, R. et al. Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach. Nat. Mater . 15 , 1120–1127 (2016).

Article   ADS   Google Scholar  

Collins, C. et al. Accelerated discovery of two crystal structure types in a complex inorganic phase field. Nature 546 , 280–284 (2017).

Davies, D. W. et al. Computer-aided design of metal chalcohalide semiconductors: from chemical composition to crystal structure. Chem. Sci . 9 , 1022–1030 (2018).

CAS   PubMed   Google Scholar  

King, R. D. Rise of the robo scientists. Sci. Am . 304 , 72–77 (2011).

ADS   PubMed   Google Scholar  

Li, J. et al. Synthesis of many different types of organic small molecules using one automated process. Science 347 , 1221–1226 (2015).

ADS   CAS   PubMed   PubMed Central   Google Scholar  

Dragone, V., Sans, V., Henson, A. B., Granda, J. M. & Cronin, L. An autonomous organic reaction search engine for chemical reactivity. Nat. Commun . 8 , 15733 (2017).

ADS   PubMed   PubMed Central   Google Scholar  

Bédard, A.-C. et al. Reconfigurable system for automated optimization of diverse chemical reactions. Science 361 , 1220–1225 (2018).

Granda, J. M., Donina, L., Dragone, V., Long, D.-L. & Cronin, L. Controlling an organic synthesis robot with machine learning to search for new reactivity. Nature 559 , 377–381 (2018).

Tabor, D. P. et al. Accelerating the discovery of materials for clean energy in the era of smart automation. Nat. Rev. Mater . 3 , 5–20 (2018).

Langner, S. et al. Beyond ternary OPV: high-throughput experimentation and self-driving laboratories optimize multi-component systems. Preprint at https://arxiv.org/abs/1909.03511 (2019).

MacLeod, B. P. et al. Self-driving laboratory for accelerated discovery of thin-film materials. Preprint at https://arxiv.org/abs/1906.05398 (2019).

Steiner, S. et al. Organic synthesis in a modular robotic system driven by a chemical programming language. Science 363 , eaav2211 (2019).

Article   CAS   Google Scholar  

Wang, Z., Li, C. & Domen, K. Recent developments in heterogeneous photocatalysts for solar-driven overall water splitting. Chem. Soc. Rev . 48 , 2109–2125 (2019).

Shahriari, B., Swersky, K., Wang, Z., Adams, R. P. & Freitas, N. D. Taking the human out of the loop: a review of Bayesian optimization. Proc. IEEE 104 , 148–175 (2016).

Article   Google Scholar  

Häse, F., Roch, L. M., Kreisbeck, C. & Aspuru-Guzik, A. Phoenics: a Bayesian optimizer for chemistry. ACS Cent. Sci . 4 , 1134–1145 (2018).

Roch, L. M. et al. ChemOS: orchestrating autonomous experimentation. Sci. Robot . 3 , eaat5559 (2018).

Chen, C.-L., Chen, T.-R., Chiu, S.-H. & Urban, P. L. Dual robotic arm “production line” mass spectrometry assay guided by multiple Arduino-type microcontrollers. Sens. Actuat. B 239 , 608–616 (2017).

Fleischer, H. et al. Analytical measurements and efficient process generation using a dual-arm robot equipped with electronic pipettes. Energies 11 , 2567 (2018).

Google Scholar  

Liu, H., Stoll, N., Junginger, S. & Thurow, K. Mobile robot for life science automation. Int. J. Adv. Robot. Syst . 10 , 288 (2013).

Liu, H., Stoll, N., Junginger, S. & Thurow, K. A fast approach to arm blind grasping and placing for mobile robot transportation in laboratories. Int. J. Adv. Robot. Syst . 11 , 43 (2014).

Abdulla, A. A., Liu, H., Stoll, N. & Thurow, K. A new robust method for mobile robot multifloor navigation in distributed life science laboratories. J. Contrib. Sci. Eng . 2016 , 3589395 (2016).

MATH   Google Scholar  

Dömel, A. et al. Toward fully autonomous mobile manipulation for industrial environments. Int. J. Adv. Robot. Syst . 14 , https://doi.org/10.1177/1729881417718588 (2017).

Schweidtmann, A. M. et al. Machine learning meets continuous flow chemistry: automated optimization towards the Pareto front of multiple objectives. Chem. Eng. J . 352 , 277–282 (2018).

CAS   Google Scholar  

Zhi, L. et al. Robot-accelerated perovskite investigation and discovery (RAPID): 1. Inverse temperature crystallization. Preprint at https://doi.org/10.26434/chemrxiv.10013090.v1 (2019).

Matsuoka, S. et al. Photocatalysis of oligo ( p -phenylenes): photoreductive production of hydrogen and ethanol in aqueous triethylamine. J. Phys. Chem . 95 , 5802–5808 (1991).

Shu, G., Li, Y., Wang, Z., Jiang, J.-X. & Wang, F. Poly(dibenzothiophene- S,S -dioxide) with visible light-induced hydrogen evolution rate up to 44.2 mmol h −1 g −1 promoted by K 2 HPO 4 . Appl. Catal. B 261 , 118230 (2020).

Pellegrin, Y. & Odobel, F. Sacrificial electron donor reagents for solar fuel production. C. R. Chim . 20 , 283–295 (2017).

Sakimoto, K. K., Zhang, S. J. & Yang, P. Cysteine–cystine photoregeneration for oxygenic photosynthesis of acetic acid from CO 2 by a tandem inorganic–biological hybrid system. Nano Lett . 16 , 5883–5887 (2016).

Wang, X. et al. Sulfone-containing covalent organic frameworks for photocatalytic hydrogen evolution from water. Nat. Chem . 10 , 1180–1189 (2018).

Schwarze, M. et al. Quantification of photocatalytic hydrogen evolution. Phys. Chem. Chem. Phys . 15 , 3466–3472 (2013).

Bai, Y. et al. Accelerated discovery of organic polymer photocatalysts for hydrogen evolution from water through the integration of experiment and theory. J. Am. Chem. Soc . 141 , 9063–9071 (2019).

Zhang, J. et al. H-bonding effect of oxyanions enhanced photocatalytic degradation of sulfonamides by g -C 3 N 4 in aqueous solution. J. Hazard. Mater . 366 , 259–267 (2019).

Hutter, F., Hoos, H. H. & Leyton-Brown, K. Parallel Algorithm Configuration 55–70 (Springer, 2012).

Mynatt, C. R., Doherty, M. E. & Tweney, R. D. Confirmation bias in a simulated research environment: an experimental study of scientific inference. Q. J. Exp. Psychol . 29 , 85–95 (1977).

King, R. D. et al. Functional genomic hypothesis generation and experimentation by a robot scientist. Nature 427 , 247–252 (2004).

Pulido, A. et al. Functional materials discovery using energy–structure–function maps. Nature 543 , 657–664 (2017).

Campbell, J. E., Yang, J. & Day, G. M. Predicted energy–structure–function maps for the evaluation of small molecule organic semiconductors. J. Mater. Chem. C 5 , 7574–7584 (2017).

Fuentes-Pacheco, J., Ruiz-Ascencio, J. & Rendón-Mancha, J. M. Visual simultaneous localization and mapping: a survey. Artif. Intell. Rev . 43 , 55–81 (2015).

Rasmussen, C. E. & Williams, C. K. I. Gaussian Processes for Machine Learning (MIT Press, 2006).

Matthews, A. G. G., Rowland, M., Hron, J., Turner, R. E. & Ghahramani, Z. Gaussian process behaviour in wide deep neural networks. Preprint at https://arxiv.org/abs/1804.11271 (2018).

Millman, K. J. & Aivazis, M. Python for scientists and engineers. Comput. Sci. Eng . 13 , 9–12 (2011).

Sachs, M. et al. Understanding structure-activity relationships in linear polymer photocatalysts for hydrogen evolution. Nat. Commun . 9 , 4968 (2018).

Download references

Acknowledgements

We acknowledge financial support from the Leverhulme Trust via the Leverhulme Research Centre for Functional Materials Design, the Engineering and Physical Sciences Research Council (EPSRC) (grant number EP/N004884/1), the Newton Fund (grant number EP/R003580/1), and CSols Ltd. X.W. and Y.B. thank the China Scholarship Council for a PhD studentship. We thank KUKA Robotics for help with gripper design and the initial implementation of the robot.

Author information

Authors and affiliations.

Leverhulme Centre for Functional Materials Design, Materials Innovation Factory and Department of Chemistry, University of Liverpool, Liverpool, UK

Benjamin Burger, Phillip M. Maffettone, Vladimir V. Gusev, Catherine M. Aitchison, Yang Bai, Xiaoyan Wang, Xiaobo Li, Ben M. Alston, Buyi Li, Rob Clowes, Nicola Rankin, Brandon Harris, Reiner Sebastian Sprick & Andrew I. Cooper

You can also search for this author in PubMed   Google Scholar

Contributions

B.B. developed the workflow, developed and implemented the robot positioning approach, wrote the control software, designed the bespoke photocatalysis station and carried out experiments. P.M.M. and V.V.G. developed the optimizer and its interface to the control software. X.L. advised on the photocatalysis workflow. C.M.A., Y.B. and X.L. synthesized materials. Y.B. performed kinetic photocatalysis experiments. X.W. performed NMR analysis and synthesized materials. B.L. carried out initial scavenger screening. R.C. and N.R. helped to build the bespoke stations in the workflow. B.H. analysed the robustness of the system, assisted with the development of control software, and operated the workflow during some experiments. B.M.A. helped to supervise the automation work. R.S.S. helped to supervise the photocatalysis work. A.I.C. conceived the idea, set up the five hypotheses with B.B., and coordinated the research team. Data was interpreted by all authors and the manuscript was prepared by A.I.C., B.B., P.M.M., V.V.G. and R.S.S.

Corresponding author

Correspondence to Andrew I. Cooper .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Peer review information Nature thanks Volker Krueger, Tyler McQuade and Magda Titirici for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended data fig. 1 mobile robotic chemist..

The mobile robot used for this project, shown here performing a six-point calibration with respect to the black location cube that is attached to the bench, in this case associated with the solid cartridge station (see also Supplementary Fig. 11 and Extended Data Fig. 3a ).

Extended Data Fig. 2 Laboratory space used for the autonomous experiments.

The key locations in the workflow are labelled. Other than the black location cubes that are fixed to the benches to allow positioning (see also Extended Data Fig. 1 ), the laboratory is otherwise unmodified.

Extended Data Fig. 3 Stations in the workflow.

a , Photograph showing the robot at the solid dispensing / cartridge station. The two cartridge hotels can hold up to 20 different solids; here, four cartridges are located in the hotel on the left. The door of the Quantos dispenser is opened using custom workflow software that interfaces with the command software that is supplied with the instrument before loading the correct solid dispensing cartridge into the instrument ( Supplementary Video 3 ). Since the KUKA Mobile Robot is free-roaming and has an 820 mm reach, it would be simple to extend this modular approach to hundreds or even thousands of different solids given sufficient laboratory space. b , Photograph showing the KUKA Mobile Robot at the photolysis station (see also  Supplementary Videos 3 , 6 ). c , Photograph showing the KUKA Mobile Robot at the combined liquid handling/capping station. The robot can reach both the liquid stations and the Liverpool Inertization Capper-Crimper (LICC) station after six-point positioning, such that liquid addition, headspace inertization and capping can be carried out in a single coordinated process (see  Supplementary Videos 3 , 5 ), without any position recalibration. d , Photograph of the KUKA Mobile Robot parked at the headspace gas chromatography (GC) station. The gas chromatography instrument is a standard commercial instrument and was unmodified in this workflow.

Extended Data Fig. 4 Hydrogen evolution rates for candidate bioderived sacrificial hole scavengers.

Results of a robotic screen for sacrificial hole scavengers using the mobile robot workflow. Of the 30 bioderived molecules trialed, only cysteine was found to compete with the petrochemical amine, triethanolamine. Scavengers are labelled with the concentration of the stock solution that was used (5 ml volume; 5 mg P10). The error bars show the standard deviation.

Extended Data Fig. 5 Multipurpose gripper used in the workflow.

The gripper is shown grasping various objects. a , The empty gripper; b , gripper holding a capped sample vial (top grasp); c , gripper holding an uncapped sample vial (side grasp); d , gripper holding a solid-dispensing cartridge; and e , gripper holding a full sample rack using an outwards grasp that locks into recesses in the rack. The same gripper was also used to activate the gas chromatography instrument using a physical button press (see Supplementary Video  3 ; 1 min 52 s).

Extended Data Fig. 6 Timescales for steps in the workflow.

Average timescales for the various steps in the workflow (sample preparation, photolysis and analysis) for a batch of 16 experiments. These averages were calculated over 46 separate batches. These average times include the time taken for the loading and unloading steps (for example, the photolysis time itself was 60 min; loading and unloading takes an average of 28 min per batch). The slowest step in the workflow is the gas chromatography analysis.

Supplementary information

Supplementary information.

This file contains Supplementary Methods and Supplementary Notes. This file presents the technical specifications of the robot, the experimental stations, workflow benchmarking, the sacrificial hole scavenger screen, control experiments, in silico benchmarking of the search algorithm, experimental robustness tests, and 24/7 monitoring of the workflow.

Supplementary Data

This file contains the data that was obtained during the autonomous search. This includes the masses and volumes suggested by the optimizer, mass and volumes measured during the autonomous experiment, and the GC measurements (amounts of hydrogen evolved).

Supplementary Video 1

This video shows the autonomous system from a bird's eye view running over 48 hours with a speed up factor of 2,880.

Supplementary Video 2

This video shows the autonomous system from a bird's eye view running in the dark; speed up factor = 360.

Supplementary Video 3

This video shows a close-up of all steps in the workflow at various speeds (20x – 100x).

Supplementary Video 4

This video shows a liquid module dispensing 1 mL of water using PID control (double speed).

Supplementary Video 5

This video shows the cap crimping process (double speed).

Supplementary Video 6

This video shows the vibratory mixing used in the photolysis station (double speed).

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Burger, B., Maffettone, P.M., Gusev, V.V. et al. A mobile robotic chemist. Nature 583 , 237–241 (2020). https://doi.org/10.1038/s41586-020-2442-2

Download citation

Received : 01 November 2019

Accepted : 25 March 2020

Published : 08 July 2020

Issue Date : 09 July 2020

DOI : https://doi.org/10.1038/s41586-020-2442-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

A soft cable loop based gripper for robotic automation of chemistry.

• Sebastiano Fichera
• Paolo Paoletti

Scientific Reports (2024)

Autonomous closed-loop mechanistic investigation of molecular electrochemistry via automation

• Hongyuan Sheng
• Jingwen Sun

Nature Communications (2024)

A dynamic knowledge graph approach to distributed self-driving laboratories

• Sebastian Mosbach
• Markus Kraft

Universal chemical programming language for robotic synthesis repeatability

• Robert Rauschen
• Leroy Cronin

Nature Synthesis (2024)

Identifying general reaction conditions by bandit optimization

• Jason Y. Wang
• Jason M. Stevens
• Abigail G. Doyle

Nature (2024)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.