1. Research Background

In recent years, rapid advances in artificial intelligence have driven the widespread adoption of mobile robots, particularly in complex environments where robotic assistance is essential. Autonomous robots are in strong demand for search-and-rescue operations, missions in uninhabited areas, underground and planetary exploration, and service-robot deployment. However, existing mobile robots often require human intervention during mapping and exploration. They either fail to fully exploit information available in the operating environment or rely excessively on prior environmental knowledge. Consequently, they lack robust autonomous navigation capabilities in unknown environments. In addition, most robot-generated maps are simple topological representations containing only basic environmental information and lacking richer semantic content; therefore, they cannot satisfy the practical requirements of the aforementioned application scenarios.

The limitations identified in existing research can be summarized as three key challenges:

• Conventional simultaneous localization and mapping (SLAM) methods generally lack autonomous navigation capabilities in unknown environments and cannot construct environmental maps efficiently. An environmental exploration architecture that supports efficient autonomous mapping is therefore required.
• Existing environmental exploration algorithms either fail to make full use of information from the actual operating environment or depend excessively on prior environmental knowledge. As a result, exploration efficiency is suboptimal and robustness is limited. An exploration-policy learning algorithm is needed to enable efficient and robust exploration of unknown environments.
• Existing mapping approaches typically produce maps containing only basic environmental information or simple topological structures, providing an incomplete understanding of the surroundings. A semantic mapping algorithm is therefore required to incorporate semantic information about objects in the environment, enabling the mobile robot to acquire sufficient object-level information and navigate more effectively in unknown scenarios.

This project is intended to address the above challenges. The resulting technologies are expected to support practical applications such as autonomous driving and mobile robotics.

2. System Architecture

The proposed system employs a deep-reinforcement-learning-based autonomous mapping and exploration algorithm to build a fully autonomous mapping platform. It is designed to address the limitations of existing DRL-based navigation and mapping methods, particularly their insufficient autonomy in unknown environments and their limited mapping efficiency. As illustrated in Figure 1, the robot first interacts with the environment and acquires perceptual information through sensors such as LiDAR and cameras. The sensor data are then processed by semantic segmentation to construct a semantic topological map. After encoding, the map is used as the state representation supplied to the reinforcement learning module for decision-making. The module outputs an action—that is, a robot control command—which drives the robot toward previously unexplored areas for the next cycle of exploration and mapping.

Figure 1. Overall architecture of the autonomous mapping system.

For topological map construction, the system is designed to generate a two-dimensional map that records the semantic attributes, positions, and sizes of objects in the environment. This representation provides the mobile robot with sufficient object-level information to better guide its motion. The construction and update workflow of the semantic topological map is shown in Figure 2.

Figure 2. Semantic topological map construction and update workflow.

3. Results and Demonstration

Before developing the complete autonomous mapping system, we implemented a deep-reinforcement-learning-based local obstacle-avoidance model in both Gazebo simulation scenarios and real-world environments. This model will serve as a component of the exploration policy in the autonomous mapping system. The overall algorithmic framework is presented in Figure 3.

Figure 3. Reinforcement-learning-based obstacle-avoidance model.

As shown in Figure 3, the LiDAR observation history and the relative goal pose form the state representation. The actor network generates control actions, while the critic network evaluates the corresponding action value. Interaction transitions are stored in the experience replay buffer and sampled to update the network parameters; the navigation environment then returns the next state and reward.

The model takes LiDAR measurements and the relative pose between the navigation goal and the robot as inputs. It outputs the robot’s linear and angular velocities.

The effectiveness of the obstacle-avoidance algorithm was evaluated in the Gazebo simulation environment. Four simulation scenarios were constructed according to scene size and the number of obstacles, as shown in Figure 4.

Figure 4. Gazebo obstacle-avoidance simulation scenarios.

The test results are presented in Table 1.

Table 1. Performance evaluation of the obstacle-avoidance model.

The results indicate that, in small-scale scenarios, the algorithm achieves a very high navigation Success Rate (SR) and Success weighted by Path Length (SPL). It can therefore be used as the baseline model for subsequent research and development under this project.