<commit_before># Apollo 2.0 Sensor Calibration Guide

This guide introduces the Apollo Sensor Calibration Service and describes the three new calibration tools in Apollo 2.0:

- Camera-to-Camera Calibration
- Camera-to-LiDAR Calibration
- Radar-to-Camera Calibration

## About This Guide

This guide provides the following information:

- Overview
- Preparation
- Using the Calibration Tools
- Obtaining Calibration Results
- Validation Methods and Results

## Overview

The new calibration tools in Apollo 2.0 (Camera-to-Camera Calibration, Camera-to-LiDAR Calibration, and Radar-to-Camera Calibration) are provided by an onboard executable program.

These new calibration tools replace the LiDAR-to-IMU Calibration feature in Apollo 1.5.

The benefit in using these tools is that they reduce the amount of work that the user must do. The user only has to start the corresponding calibration program, and the calibration work is performed and completes in real time. The user can then verify the calibration results, which are provided as `.yaml` files.

## Preparation

### Well Calibrated Intrinsics of Camera

Camera intrinsics contain focus length, principal points, distortion coefficients, and other information. Users can obtain the intrinsics from other camera calibration tools such as the [ROS Camera Calibration Tools](http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration) and the [Camera Calibration Toolbox for Matlab](http://www.vision.caltech.edu/bouguetj/calib_doc/). After the calibration is completed, ***users should convert the result to a specific `yaml` format file manually***. 

Users must ensure that the `K` and `D` data is correct:

- `K` refers to the camera matrix 
- `D` refers to the distortion parameters

The following is an example of a camera intrinsic file:

```bash
    header: 
      seq: 0
      stamp: 
      secs: 0
        nsecs: 0
      frame_id: short_camera
    height: 1080
    width: 1920
    distortion_model: plumb_bob
    D: [-0.535253, 0.259291, 0.004276, -0.000503, 0.0]
    K: [1959.678185, 0.0, 1003.592207, 0.0, 1953.786100, 507.820634, 0.0, 0.0, 1.0]
    R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
    P: [1665.387817, 0.0, 1018.703332, 0.0, 0.0, 1867.912842, 506.628623, 0.0, 0.0, 0.0, 1.0, 0.0]
    binning_x: 0
    binning_y: 0
    roi: 
      x_offset: 0
      y_offset: 0
      height: 0
      width: 0
      do_rectify: False
```

Baidu recommends that you perform the intrinsic calibration for every single camera instead of using unified intrinsic parameters for every camera. If you follow this practice, you can improve the accuracy of the extrinsic calibration results.

### Initial Extrinsic File

The tools require the user to provide an initial extrinsic value as a reference. 

The following is an example of an initial extrinsic file of Camera-to-LiDAR, where `translation` is the shift distance between the camera and LiDAR. The `rotation` is the quaternion expression form of the rotation matrix.

```bash
    header:
      seq: 0
      stamp:
        secs: 0
        nsecs: 0
      frame_id: velodyne64
    child_frame_id: short_camera
    transform:
      rotation:
        y: 0.5
        x: -0.5
        w: 0.5
        z: -0.5
      translation:
        x: 0.0
        y: 1.5
        z: 2.0
```

**NOTE:** The Camera-to-LiDAR Calibration is more dependent on initial extrinsic values. ***A large deviation can lead to calibration failure.*** Therefore, it is essential that you provide the most accurate, initial extrinsic value as conditions allow.

### Calibration Site

Because the Camera-to-LiDAR Calibration method is based on an actual scene, an ideal real-world location can significantly improve the accuracy of the calibration. It is recommended that you select a calibration site that includes objects such as trees, poles, street lights, traffic signs, stationary objects, and clear traffic lines. 

Figure 1 is an example of a good choice for a calibration site: 

  ![](images/calibration/sensor_calibration/calibration_place.png)
  <p align="center"> Figure 1. Good Choice for a Calibration Site </p>

### Required Topics

Users must confirm that all sensor topics required by the program have output messages. For more information, see: [How to Check the Sensor Output?](https://github.com/ApolloAuto/apollo/blob/master/docs/FAQs/Calibration_FAQs.md)

The sensor topics that the on-board program requires are listed in Tables 1, 2, and 3.

  **Table 1. The Required Topics of Camera-to-Camera Calibration**

| Sensor       | Topic Name                               | Topic Feq. (Hz) |
| ------------ | ---------------------------------------- | --------------- |
| Short_Camera | /apollo/sensor/camera/traffic/image_short | 9               |
| Long_Camera  | /apollo/sensor/camera/traffic/image_long | 9               |
| INS          | /apollo/sensor/gnss/odometry             | 100             |
| INS          | /apollo/sensor/gnss/ins_stat             | 2               |

  **Table 2. The Required Topics of Camera-to-LiDAR Calibration**

| Sensor         | Topic Name                               | Topic Feq. (Hz) |
| -------------- | ---------------------------------------- | --------------- |
| Short_Camera   | /apollo/sensor/camera/traffic/image_short | 9               |
| Velodyne HDL64 | /apollo/sensor/velodyne64/compensator/PointCloud2 | 10              |
| INS            | /apollo/sensor/gnss/odometry             | 100             |
| INS            | /apollo/sensor/gnss/ins_stat             | 2               |

  **Table 3. The Required Topics of Radar-to-Camera Calibration**

| Sensor       | Topic Name                               | Topic Feq. (Hz) |
| ------------ | ---------------------------------------- | --------------- |
| Short_Camera | /apollo/sensor/camera/traffic/image_short | 9               |
| INS          | /apollo/sensor/gnss/odometry             | 100             |
| INS          | /apollo/sensor/gnss/ins_stat             | 2               |

## Using the Calibration Tools

This section provides the following information to use the three calibration tools:

- Commands to run each tool
- Data collection guidelines
- Location of the configuration file
- Types of data output

Before you begin to use the tools, you must verify that the localization status is ***56*** or the calibration tools (programs) will ***not*** collect data.  

Type the following command to check localization status:

```bash
rostopic echo /apollo/sensor/gnss/ins_stat
```

### Camera-to-Camera Calibration Tool

1. Run the Camera-to-Camera Calibration Tool using these commands:

```bash
cd /apollo/scripts
bash sensor_calibration.sh lidar_camera
```

2. Run the Camera-to-Camera Calibration Tool using these commands:

  * Because the two cameras have different timestamps, they cannot be completely synchronized, so it is important to drive the vehicle very slowly when recording the data. The slow speed of the vehicle can effectively alleviate the image mismatch that is caused by the different timestamps.

  * Make sure to enable a large enough overlap of the regions of the two camera images or the tool will ***not*** be able to perform the extrinsic calibration operation.
<commit_msg>fix a document error<commit_after># Apollo 2.0 Sensor Calibration Guide

This guide introduces the Apollo Sensor Calibration Service and describes the three new calibration tools in Apollo 2.0:

- Camera-to-Camera Calibration
- Camera-to-LiDAR Calibration
- Radar-to-Camera Calibration

## About This Guide

This guide provides the following information:

- Overview
- Preparation
- Using the Calibration Tools
- Obtaining Calibration Results
- Validation Methods and Results

## Overview

The new calibration tools in Apollo 2.0 (Camera-to-Camera Calibration, Camera-to-LiDAR Calibration, and Radar-to-Camera Calibration) are provided by an onboard executable program.

These new calibration tools replace the LiDAR-to-IMU Calibration feature in Apollo 1.5.

The benefit in using these tools is that they reduce the amount of work that the user must do. The user only has to start the corresponding calibration program, and the calibration work is performed and completes in real time. The user can then verify the calibration results, which are provided as `.yaml` files.

## Preparation

### Well Calibrated Intrinsics of Camera

Camera intrinsics contain focus length, principal points, distortion coefficients, and other information. Users can obtain the intrinsics from other camera calibration tools such as the [ROS Camera Calibration Tools](http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration) and the [Camera Calibration Toolbox for Matlab](http://www.vision.caltech.edu/bouguetj/calib_doc/). After the calibration is completed, ***users should convert the result to a specific `yaml` format file manually***. 

Users must ensure that the `K` and `D` data is correct:

- `K` refers to the camera matrix 
- `D` refers to the distortion parameters

The following is an example of a camera intrinsic file:

```bash
    header: 
      seq: 0
      stamp: 
      secs: 0
        nsecs: 0
      frame_id: short_camera
    height: 1080
    width: 1920
    distortion_model: plumb_bob
    D: [-0.535253, 0.259291, 0.004276, -0.000503, 0.0]
    K: [1959.678185, 0.0, 1003.592207, 0.0, 1953.786100, 507.820634, 0.0, 0.0, 1.0]
    R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
    P: [1665.387817, 0.0, 1018.703332, 0.0, 0.0, 1867.912842, 506.628623, 0.0, 0.0, 0.0, 1.0, 0.0]
    binning_x: 0
    binning_y: 0
    roi: 
      x_offset: 0
      y_offset: 0
      height: 0
      width: 0
      do_rectify: False
```

Baidu recommends that you perform the intrinsic calibration for every single camera instead of using unified intrinsic parameters for every camera. If you follow this practice, you can improve the accuracy of the extrinsic calibration results.

### Initial Extrinsic File

The tools require the user to provide an initial extrinsic value as a reference. 

The following is an example of an initial extrinsic file of Camera-to-LiDAR, where `translation` is the shift distance between the camera and LiDAR. The `rotation` is the quaternion expression form of the rotation matrix.

```bash
    header:
      seq: 0
      stamp:
        secs: 0
        nsecs: 0
      frame_id: velodyne64
    child_frame_id: short_camera
    transform:
      rotation:
        y: 0.5
        x: -0.5
        w: 0.5
        z: -0.5
      translation:
        x: 0.0
        y: 1.5
        z: 2.0
```

**NOTE:** The Camera-to-LiDAR Calibration is more dependent on initial extrinsic values. ***A large deviation can lead to calibration failure.*** Therefore, it is essential that you provide the most accurate, initial extrinsic value as conditions allow.

### Calibration Site

Because the Camera-to-LiDAR Calibration method is based on an actual scene, an ideal real-world location can significantly improve the accuracy of the calibration. It is recommended that you select a calibration site that includes objects such as trees, poles, street lights, traffic signs, stationary objects, and clear traffic lines. 

Figure 1 is an example of a good choice for a calibration site: 

  ![](images/calibration/sensor_calibration/calibration_place.png)
  <p align="center"> Figure 1. Good Choice for a Calibration Site </p>

### Required Topics

Users must confirm that all sensor topics required by the program have output messages. For more information, see: [How to Check the Sensor Output?](https://github.com/ApolloAuto/apollo/blob/master/docs/FAQs/Calibration_FAQs.md)

The sensor topics that the on-board program requires are listed in Tables 1, 2, and 3.

  **Table 1. The Required Topics of Camera-to-Camera Calibration**

| Sensor       | Topic Name                               | Topic Feq. (Hz) |
| ------------ | ---------------------------------------- | --------------- |
| Short_Camera | /apollo/sensor/camera/traffic/image_short | 9               |
| Long_Camera  | /apollo/sensor/camera/traffic/image_long | 9               |
| INS          | /apollo/sensor/gnss/odometry             | 100             |
| INS          | /apollo/sensor/gnss/ins_stat             | 2               |

  **Table 2. The Required Topics of Camera-to-LiDAR Calibration**

| Sensor         | Topic Name                               | Topic Feq. (Hz) |
| -------------- | ---------------------------------------- | --------------- |
| Short_Camera   | /apollo/sensor/camera/traffic/image_short | 9               |
| Velodyne HDL64 | /apollo/sensor/velodyne64/compensator/PointCloud2 | 10              |
| INS            | /apollo/sensor/gnss/odometry             | 100             |
| INS            | /apollo/sensor/gnss/ins_stat             | 2               |

  **Table 3. The Required Topics of Radar-to-Camera Calibration**

| Sensor       | Topic Name                               | Topic Feq. (Hz) |
| ------------ | ---------------------------------------- | --------------- |
| Short_Camera | /apollo/sensor/camera/traffic/image_short | 9               |
| INS          | /apollo/sensor/gnss/odometry             | 100             |
| INS          | /apollo/sensor/gnss/ins_stat             | 2               |

## Using the Calibration Tools

This section provides the following information to use the three calibration tools:

- Commands to run each tool
- Data collection guidelines
- Location of the configuration file
- Types of data output

Before you begin to use the tools, you must verify that the localization status is ***56*** or the calibration tools (programs) will ***not*** collect data.  

Type the following command to check localization status:

```bash
rostopic echo /apollo/sensor/gnss/ins_stat
```

### Camera-to-Camera Calibration Tool

1. Run the Camera-to-Camera Calibration Tool using these commands:

```bash
cd /apollo/scripts
bash sensor_calibration.sh camera_camera
```

2. Follow these guidelines to collect data:

  * Because the two cameras have different timestamps, they cannot be completely synchronized, so it is important to drive the vehicle very slowly when recording the data. The slow speed of the vehicle can effectively alleviate the image mismatch that is caused by the different timestamps.

  * Make sure to enable a large enough overlap of the regions of the two camera images or the tool will ***not*** be able to perform the extrinsic calibration operation.

{'fraction_non_alphanumeric': 0.10423722955692265, 'fraction_numerical': 0.03193474804728002, 'mean_word_length': 2.5653031049778217, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7245295', 'n_tokens_mistral': 4406, 'n_tokens_neox': 3893, 'n_words': 1643}