From the official documents of Apollo, it is easy to know that Perception is one of the core components, but like all C + + programs, each application has a Main function entry, which leads to two problems to be explored in this paper:
- Where is the entrance to Perception?
- How does Perception start?
CyberRT
Before talking about the specific content of Perception components, it is very necessary to talk about CyberRT.
Apollo Cyber RT framework is built based on the concept of component. As a basic building block of Apollo Cyber RT framework, each component contains a specific algorithm module which process a set of data inputs and generate a set of outputs. – Apollo github.com
CyberRT is a basic framework in Apollo and is component oriented. Components are highly modular.
Each component contains a specific algorithm for processing data input and outputting the results of algorithm processing.
Students who have played ROS should be familiar with such things. CyberRT is similar to ROS. In fact, Apollo used ROS1 at first. Later, because the delay problem was not satisfied, they developed a similar one with better performance.
Back to the component problem, Perception is also a component. As described in the previous article, it receives the data of the sensor and then outputs the 3D information of the obstacle.
In CyberRT, there are instructions on how to define, implement and start components.
Component management
There are usually four steps for component development:
- Set component file structure
- Implementation component class
- Set profile
- Start component
Perception component related files
According to the official Apollo documentation, there are several documents related to a component:
Header file: common_component_example.h Source file: common_component_example.cc Build file: BUILD DAG dependency file: common.dag Launch file: common.launch
According to the prompt, we can search for relevant files in the source code.
The official Apollo 6.0 document says:
The Perception module is now capable of detecting and classifying obstacles within only one component named Detection component.
This means that the Perception module can now have only one component: detection.
Why do you say that?
Apollo is multi data fusion. It integrates Camera, Lidar and Radar targets, all of which are one component.
We can start from the general entrance and analyze it one by one.
Therefore, we have to look for Detection related components first.
Easy to find, path:
apollo/modules/perception/onboard/component
Under this directory, many perception related components are defined and implemented. This paper only focuses on Detection.
Its two related documents are:
- detection_component.h
- detection_component.cc
detection_component.h
namespace apollo {
namespace perception {
namespace onboard {
class DetectionComponent : public cyber::Component<drivers::PointCloud> {
public:
DetectionComponent() = default;
virtual ~DetectionComponent() = default;
bool Init() override;
bool Proc(const std::shared_ptr<drivers::PointCloud>& message) override;
private:
static std::atomic<uint32_t> seq_num_;
std::string sensor_name_;
// bool enable_hdmap_ = true;
float lidar_query_tf_offset_ = 20.0f;
std::string lidar2novatel_tf2_child_frame_id_;
std::string output_channel_name_;
base::SensorInfo sensor_info_;
TransformWrapper lidar2world_trans_;
std::unique_ptr<lidar::LidarObstacleDetection> detector_;
std::shared_ptr<apollo::cyber::Writer<LidarFrameMessage>> writer_;
};
CYBER_REGISTER_COMPONENT(DetectionComponent);
} // namespace onboard
} // namespace perception
} You can see that it is mainly point cloud data input.
Implementation component
detection_component.cc
bool DetectionComponent::Init() {
LidarDetectionComponentConfig comp_config;
if (!GetProtoConfig(&comp_config)) {
return false;
}
ADEBUG << "Lidar Component Configs: " << comp_config.DebugString();
output_channel_name_ = comp_config.output_channel_name();
sensor_name_ = comp_config.sensor_name();
lidar2novatel_tf2_child_frame_id_ =
comp_config.lidar2novatel_tf2_child_frame_id();
lidar_query_tf_offset_ =
static_cast<float>(comp_config.lidar_query_tf_offset());
// enable_hdmap_ = comp_config.enable_hdmap();
writer_ = node_->CreateWriter<LidarFrameMessage>(output_channel_name_);
if (!InitAlgorithmPlugin()) {
AERROR << "Failed to init detection component algorithm plugin.";
return false;
}
return true;
}
bool DetectionComponent::Proc(
const std::shared_ptr<drivers::PointCloud>& message) {
AINFO << std::setprecision(16)
<< "Enter detection component, message timestamp: "
<< message->measurement_time()
<< " current timestamp: " << Clock::NowInSeconds();
auto out_message = std::make_shared<LidarFrameMessage>();
bool status = InternalProc(message, out_message);
if (status) {
writer_->Write(out_message);
AINFO << "Send lidar detect output message.";
}
return status;
}Init and Proc are two core methods of component.
The BUILD file address is:
apollo/modules/perception/onboard/component/BUILD
The BUILD file defines the information of all component s in perception, such as camera,radar,lidar, etc. This article only focuses on Detection.
cc_library(
name = "detection_component",
srcs = ["detection_component.cc"],
hdrs = ["detection_component.h"],
deps = [
":lidar_inner_component_messages",
"//cyber/time:clock",
"//modules/common/util:string_util",
"//modules/perception/common/sensor_manager",
"//modules/perception/lib/registerer",
"//modules/perception/lidar/app:lidar_obstacle_detection",
"//modules/perception/lidar/common",
"//modules/perception/onboard/common_flags",
"//modules/perception/onboard/proto:lidar_component_config_cc_proto",
"//modules/perception/onboard/transform_wrapper",
"@eigen",
],
)Set profile
There are two types of configuration files for a Component:
- DAG
- Launch
DAG defines the dependencies of modules.
The Launch file defines the startup of the module.
Look at the Launch file first.
apollo/modules/perception/production/launch/perception_all.launch
<cyber>
<desc>cyber modules list config</desc>
<version>1.0.0</version>
<!-- sample module config, and the files should have relative path like
./bin/cyber_launch
./bin/mainboard
./conf/dag_streaming_0.conf -->
<module>
<name>perception</name>
<dag_conf>/apollo/modules/perception/production/dag/dag_streaming_perception.dag</dag_conf>
<!-- if not set, use default process -->
<process_name>perception</process_name>
<version>1.0.0</version>
</module>
<module>
<name>perception_camera</name>
<dag_conf>/apollo/modules/perception/production/dag/dag_streaming_perception_camera.dag</dag_conf>
<!-- if not set, use default process -->
<process_name>perception</process_name>
<version>1.0.0</version>
</module>
<module>
<name>perception_traffic_light</name>
<dag_conf>/apollo/modules/perception/production/dag/dag_streaming_perception_trafficlights.dag</dag_conf>
<!-- if not set, use default process -->
<process_name>perception_trafficlights</process_name>
<version>1.0.0</version>
</module>
<module>
<name>motion_service</name>
<dag_conf>/apollo/modules/perception/production/dag/dag_motion_service.dag</dag_conf>
<!-- if not set, use default process -->
<process_name>motion_service</process_name>
<version>1.0.0</version>
</module>
</cyber>We can find the dag path in the Perception module:
<dag_conf>/apollo/modules/perception/production/dag/dag_streaming_perception.dag</dag_conf>
What does it look like?
module_config {
module_library : "/apollo/bazel-bin/modules/perception/onboard/component/libperception_component_lidar.so"
components {
class_name : "DetectionComponent"
config {
name: "Velodyne128Detection"
config_file_path: "/apollo/modules/perception/production/conf/perception/lidar/velodyne128_detection_conf.pb.txt"
flag_file_path: "/apollo/modules/perception/production/conf/perception/perception_common.flag"
readers {
channel: "/apollo/sensor/lidar128/compensator/PointCloud2"
}
}
}
components {
class_name : "RecognitionComponent"
config {
name: "RecognitionComponent"
config_file_path: "/apollo/modules/perception/production/conf/perception/lidar/recognition_conf.pb.txt"
readers {
channel: "/perception/inner/DetectionObjects"
}
}
}
components {
class_name: "RadarDetectionComponent"
config {
name: "FrontRadarDetection"
config_file_path: "/apollo/modules/perception/production/conf/perception/radar/front_radar_component_conf.pb.txt"
readers {
channel: "/apollo/sensor/radar/front"
}
}
}
components {
class_name: "RadarDetectionComponent"
config {
name: "RearRadarDetection"
config_file_path: "/apollo/modules/perception/production/conf/perception/radar/rear_radar_component_conf.pb.txt"
readers {
channel: "/apollo/sensor/radar/rear"
}
}
}
components {
class_name: "FusionComponent"
config {
name: "SensorFusion"
config_file_path: "/apollo/modules/perception/production/conf/perception/fusion/fusion_component_conf.pb.txt"
readers {
channel: "/perception/inner/PrefusedObjects"
}
}
}
}
module_config {
module_library : "/apollo/bazel-bin/modules/v2x/fusion/apps/libv2x_fusion_component.so"
components {
class_name : "V2XFusionComponent"
config {
name : "v2x_fusion"
flag_file_path : "/apollo/modules/v2x/conf/v2x_fusion_tracker.conf"
readers: [
{
channel: "/perception/vehicle/obstacles"
}
]
}
}
}We found the figure of DetectionComponent. According to its configuration parameters, it is the configuration file of Velodyne 128 line Lidar. Indeed, we can see that Apollo is dominated by Lidar.
Start component
After defining a component related document, you can start it.
Command:
cyber_launch start apollo/modules/perception/production/launch/perception_all.launch
Since then, we have found the whole code process of starting a component of perception in Apollo.
summary
Because the Perception module is very large and involves three types of sensors: Camera, Radar and lidar, the data processing and model processing of each sensor will produce more than 10 tasks, and each task is designed as a component.
It's very easy to get lost in the code forest by directly reading the relevant code. Therefore, this article first explains the configuration mechanism of CyberRT, the basic framework Component. After being familiar with the basic routine, it becomes very easy to query the code when you learn the corresponding components later.
Subsequent articles will start with Camera and analyze the component basic algorithm and code implementation related to data fusion one by one. The task is a little arduous, but I believe there will be a lot of gains.