chore: import upstream snapshot with attribution
@@ -0,0 +1,202 @@
|
||||
Detection of ArUco boards {#tutorial_aruco_board_detection}
|
||||
=========================
|
||||
|
||||
@prev_tutorial{tutorial_aruco_detection}
|
||||
@next_tutorial{tutorial_charuco_detection}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original authors | Sergio Garrido, Alexander Panov |
|
||||
| Compatibility | OpenCV >= 4.7.0 |
|
||||
|
||||
An ArUco board is a set of markers that acts like a single marker in the sense that it provides a
|
||||
single pose for the camera.
|
||||
|
||||
The most popular board is the one with all the markers in the same plane, since it can be easily printed:
|
||||
|
||||

|
||||
|
||||
However, boards are not limited to this arrangement and can represent any 2d or 3d layout.
|
||||
|
||||
The difference between a board and a set of independent markers is that the relative position between
|
||||
the markers in the board is known a priori. This allows that the corners of all the markers can be used for
|
||||
estimating the pose of the camera respect to the whole board.
|
||||
|
||||
When you use a set of independent markers, you can estimate the pose for each marker individually,
|
||||
since you dont know the relative position of the markers in the environment.
|
||||
|
||||
The main benefits of using boards are:
|
||||
|
||||
- The pose estimation is much more versatile. Only some markers are necessary to perform pose estimation.
|
||||
Thus, the pose can be calculated even in the presence of occlusions or partial views.
|
||||
- The obtained pose is usually more accurate since a higher amount of point correspondences (marker
|
||||
corners) are employed.
|
||||
|
||||
Board Detection
|
||||
---------------
|
||||
|
||||
A board detection is similar to the standard marker detection. The only difference is in the pose estimation step.
|
||||
In fact, to use marker boards, a standard marker detection should be done before estimating the board pose.
|
||||
|
||||
To perform pose estimation for boards, you should use `solvePnP()` function, as shown below
|
||||
in the `samples/cpp/tutorial_code/objectDetection/detect_board.cpp`.
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_board_full_sample
|
||||
|
||||
|
||||
The parameters are:
|
||||
|
||||
- `objPoints`, `imgPoints` object and image points, matched with `cv::aruco::GridBoard::matchImagePoints()`
|
||||
which, in turn, takes as input `markerCorners` and `markerIds` structures of detected markers from
|
||||
`cv::aruco::ArucoDetector::detectMarkers()` function.
|
||||
- `board` the `cv::aruco::Board` object that defines the board layout and its ids
|
||||
- `cameraMatrix` and `distCoeffs`: camera calibration parameters necessary for pose estimation.
|
||||
- `rvec` and `tvec`: estimated pose of the board. If not empty then treated as initial guess.
|
||||
- The function returns the total number of markers employed for estimating the board pose.
|
||||
|
||||
The drawFrameAxes() function can be used to check the obtained pose. For instance:
|
||||
|
||||

|
||||
|
||||
And this is another example with the board partially occluded:
|
||||
|
||||

|
||||
|
||||
As it can be observed, although some markers have not been detected, the board pose can still be
|
||||
estimated from the rest of markers.
|
||||
|
||||
Sample video:
|
||||
|
||||
@youtube{Q1HlJEjW_j0}
|
||||
|
||||
A full working example is included in the `detect_board.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples now take input via command line via the `cv::CommandLineParser`. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
-w=5 -h=7 -l=100 -s=10
|
||||
-v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_board_detection/gboriginal.jpg
|
||||
-c=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml
|
||||
-cd=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml
|
||||
@endcode
|
||||
Parameters for `detect_board.cpp`:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_board_keys
|
||||
|
||||
Grid Board
|
||||
----------
|
||||
|
||||
Creating the `cv::aruco::Board` object requires specifying the corner positions for each marker in the environment.
|
||||
However, in many cases, the board will be just a set of markers in the same plane and in a grid layout,
|
||||
so it can be easily printed and used.
|
||||
|
||||
Fortunately, the aruco module provides the basic functionality to create and print these types of markers
|
||||
easily.
|
||||
|
||||
The `cv::aruco::GridBoard` class is a specialized class that inherits from the `cv::aruco::Board`
|
||||
class and which represents a Board with all the markers in the same plane and in a grid layout,
|
||||
as in the following image:
|
||||
|
||||

|
||||
|
||||
Concretely, the coordinate system in a grid board is positioned in the board plane, centered in the bottom left
|
||||
corner of the board and with the Z pointing out, like in the following image (X:red, Y:green, Z:blue):
|
||||
|
||||

|
||||
|
||||
A `cv::aruco::GridBoard` object can be defined using the following parameters:
|
||||
|
||||
- Number of markers in the X direction.
|
||||
- Number of markers in the Y direction.
|
||||
- Length of the marker side.
|
||||
- Length of the marker separation.
|
||||
- The dictionary of the markers.
|
||||
- Ids of all the markers (X*Y markers).
|
||||
|
||||
This object can be easily created from these parameters using the `cv::aruco::GridBoard` constructor:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_create_board
|
||||
|
||||
- The first and second parameters are the number of markers in the X and Y direction respectively.
|
||||
- The third and fourth parameters are the marker length and the marker separation respectively.
|
||||
They can be provided in any unit, having in mind that the estimated pose for this board will be
|
||||
measured in the same units (in general, meters are used).
|
||||
- Finally, the dictionary of the markers is provided.
|
||||
|
||||
So, this board will be composed by 5x7=35 markers. The ids of each of the markers are assigned, by default,
|
||||
in ascending order starting on 0, so they will be 0, 1, 2, ..., 34.
|
||||
|
||||
After creating a grid board, we probably want to print it and use it.
|
||||
There are two ways to do this:
|
||||
1. By using the script `apps/pattern-tools/generate_pattern.py`, see @subpage tutorial_camera_calibration_pattern.
|
||||
2. By using the function `cv::aruco::GridBoard::generateImage()`.
|
||||
|
||||
The function `cv::aruco::GridBoard::generateImage()` is provided in cv::aruco::GridBoard class and
|
||||
can be called by using the following code:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/create_board.cpp aruco_generate_board_image
|
||||
|
||||
- The first parameter is the size of the output image in pixels. In this case 600x500 pixels. If this is not proportional
|
||||
to the board dimensions, it will be centered on the image.
|
||||
- `boardImage`: the output image with the board.
|
||||
- The third parameter is the (optional) margin in pixels, so none of the markers are touching the image border.
|
||||
In this case the margin is 10.
|
||||
- Finally, the size of the marker border, similarly to `generateImageMarker()` function. The default value is 1.
|
||||
|
||||
A full working example of board creation is included in the `samples/cpp/tutorial_code/objectDetection/create_board.cpp`
|
||||
|
||||
The output image will be something like this:
|
||||
|
||||

|
||||
|
||||
The samples now take input via commandline via the `cv::CommandLineParser`. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
"_output_path_/aboard.png" -w=5 -h=7 -l=100 -s=10 -d=10
|
||||
@endcode
|
||||
|
||||
Refine marker detection
|
||||
-----------------------
|
||||
|
||||
ArUco boards can also be used to improve the detection of markers. If we have detected a subset of the markers
|
||||
that belongs to the board, we can use these markers and the board layout information to try to find the
|
||||
markers that have not been previously detected.
|
||||
|
||||
This can be done using the `cv::aruco::refineDetectedMarkers()` function, which should be called
|
||||
after calling `cv::aruco::ArucoDetector::detectMarkers()`.
|
||||
|
||||
The main parameters of this function are the original image where markers were detected, the board object,
|
||||
the detected marker corners, the detected marker ids and the rejected marker corners.
|
||||
|
||||
The rejected corners can be obtained from the `cv::aruco::ArucoDetector::detectMarkers()` function and
|
||||
are also known as marker candidates. This candidates are square shapes that have been found in the
|
||||
original image but have failed to pass the identification step (i.e. their inner codification presents
|
||||
too many errors) and thus they have not been recognized as markers.
|
||||
|
||||
However, these candidates are sometimes actual markers that have not been correctly identified due to high
|
||||
noise in the image, very low resolution or other related problems that affect to the binary code extraction.
|
||||
The `cv::aruco::ArucoDetector::refineDetectedMarkers()` function finds correspondences between these
|
||||
candidates and the missing markers of the board. This search is based on two parameters:
|
||||
|
||||
- Distance between the candidate and the projection of the missing marker. To obtain these projections,
|
||||
it is necessary to have detected at least one marker of the board. The projections are obtained using the
|
||||
camera parameters (camera matrix and distortion coefficients) if they are provided. If not, the projections
|
||||
are obtained from local homography and only planar board are allowed (i.e. the Z coordinate of all the
|
||||
marker corners should be the same). The `minRepDistance` parameter in `refineDetectedMarkers()`
|
||||
determines the minimum euclidean distance between the candidate corners and the projected marker corners
|
||||
(default value 10).
|
||||
|
||||
- Binary codification. If a candidate surpasses the minimum distance condition, its internal bits
|
||||
are analyzed again to determine if it is actually the projected marker or not. However, in this case,
|
||||
the condition is not so strong and the number of allowed erroneous bits can be higher. This is indicated
|
||||
in the `errorCorrectionRate` parameter (default value 3.0). If a negative value is provided, the
|
||||
internal bits are not analyzed at all and only the corner distances are evaluated.
|
||||
|
||||
This is an example of using the `cv::aruco::ArucoDetector::refineDetectedMarkers()` function:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board.cpp aruco_detect_and_refine
|
||||
|
||||
It must also be noted that, in some cases, if the number of detected markers in the first place is
|
||||
too low (for instance only 1 or 2 markers), the projections of the missing markers can be of bad
|
||||
quality, producing erroneous correspondences.
|
||||
|
||||
See module samples for a more detailed implementation.
|
||||
|
After Width: | Height: | Size: 11 KiB |
|
After Width: | Height: | Size: 61 KiB |
|
After Width: | Height: | Size: 70 KiB |
|
After Width: | Height: | Size: 59 KiB |
|
After Width: | Height: | Size: 61 KiB |
|
After Width: | Height: | Size: 75 KiB |
@@ -0,0 +1,88 @@
|
||||
Calibration with ArUco and ChArUco {#tutorial_aruco_calibration}
|
||||
==================================
|
||||
|
||||
@prev_tutorial{tutorial_charuco_diamond_detection}
|
||||
@next_tutorial{tutorial_aruco_faq}
|
||||
|
||||
The ArUco module can also be used to calibrate a camera. Camera calibration consists in obtaining the
|
||||
camera intrinsic parameters and distortion coefficients. This parameters remain fixed unless the camera
|
||||
optic is modified, thus camera calibration only need to be done once.
|
||||
|
||||
Camera calibration is usually performed using the OpenCV `cv::calibrateCamera()` function. This function
|
||||
requires some correspondences between environment points and their projection in the camera image from
|
||||
different viewpoints. In general, these correspondences are obtained from the corners of chessboard
|
||||
patterns. See `cv::calibrateCamera()` function documentation or the OpenCV calibration tutorial for
|
||||
more detailed information.
|
||||
|
||||
Using the ArUco module, calibration can be performed based on ArUco markers corners or ChArUco corners.
|
||||
Calibrating using ArUco is much more versatile than using traditional chessboard patterns, since it
|
||||
allows occlusions or partial views.
|
||||
|
||||
As it can be stated, calibration can be done using both, marker corners or ChArUco corners. However,
|
||||
it is highly recommended using the ChArUco corners approach since the provided corners are much
|
||||
more accurate in comparison to the marker corners. Calibration using a standard Board should only be
|
||||
employed in those scenarios where the ChArUco boards cannot be employed because of any kind of restriction.
|
||||
|
||||
Calibration with ChArUco Boards
|
||||
-------------------------------
|
||||
|
||||
To calibrate using a ChArUco board, it is necessary to detect the board from different viewpoints, in the
|
||||
same way that the standard calibration does with the traditional chessboard pattern. However, due to the
|
||||
benefits of using ChArUco, occlusions and partial views are allowed, and not all the corners need to be
|
||||
visible in all the viewpoints.
|
||||
|
||||

|
||||
|
||||
The example of using `cv::calibrateCamera()` for cv::aruco::CharucoBoard:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/calibrate_camera_charuco.cpp CalibrationWithCharucoBoard1
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/calibrate_camera_charuco.cpp CalibrationWithCharucoBoard2
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/calibrate_camera_charuco.cpp CalibrationWithCharucoBoard3
|
||||
|
||||
The ChArUco corners and ChArUco identifiers captured on each viewpoint are stored in the vectors
|
||||
`allCharucoCorners` and `allCharucoIds`, one element per viewpoint.
|
||||
|
||||
The `calibrateCamera()` function will fill the `cameraMatrix` and `distCoeffs` arrays with the
|
||||
camera calibration parameters. It will return the reprojection error obtained from the calibration.
|
||||
The elements in `rvecs` and `tvecs` will be filled with the estimated pose of the camera
|
||||
(respect to the ChArUco board) in each of the viewpoints.
|
||||
|
||||
Finally, the `calibrationFlags` parameter determines some of the options for the calibration.
|
||||
|
||||
A full working example is included in the `calibrate_camera_charuco.cpp` inside the
|
||||
`samples/cpp/tutorial_code/objectDetection` folder.
|
||||
|
||||
The samples now take input via commandline via the `cv::CommandLineParser`. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
"camera_calib.txt" -w=5 -h=7 -sl=0.04 -ml=0.02 -d=10
|
||||
-v=path/img_%02d.jpg
|
||||
@endcode
|
||||
|
||||
The camera calibration parameters from `opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_charuco.yml`
|
||||
were obtained by the `img_00.jpg-img_03.jpg` placed from this
|
||||
[folder](https://github.com/opencv/opencv_contrib/tree/4.6.0/modules/aruco/tutorials/aruco_calibration/images).
|
||||
|
||||
Calibration with ArUco Boards
|
||||
-----------------------------
|
||||
|
||||
As it has been stated, it is recommended the use of ChAruco boards instead of ArUco boards for camera
|
||||
calibration, since ChArUco corners are more accurate than marker corners. However, in some special cases
|
||||
it must be required to use calibration based on ArUco boards. As in the previous case, it requires
|
||||
the detections of an ArUco board from different viewpoints.
|
||||
|
||||

|
||||
|
||||
The example of using `cv::calibrateCamera()` for cv::aruco::GridBoard:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/calibrate_camera.cpp CalibrationWithArucoBoard1
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/calibrate_camera.cpp CalibrationWithArucoBoard2
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/calibrate_camera.cpp CalibrationWithArucoBoard3
|
||||
|
||||
A full working example is included in the `calibrate_camera.cpp` inside the `samples/cpp/tutorial_code/objectDetection` folder.
|
||||
|
||||
The samples now take input via commandline via the `cv::CommandLineParser`. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
"camera_calib.txt" -w=5 -h=7 -l=100 -s=10 -d=10 -v=path/aruco_videos_or_images
|
||||
@endcode
|
||||
|
After Width: | Height: | Size: 86 KiB |
|
After Width: | Height: | Size: 75 KiB |
@@ -0,0 +1,702 @@
|
||||
Detection of ArUco Markers {#tutorial_aruco_detection}
|
||||
==========================
|
||||
|
||||
@next_tutorial{tutorial_aruco_board_detection}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original authors | Sergio Garrido, Alexander Panov |
|
||||
| Compatibility | OpenCV >= 4.7.0 |
|
||||
|
||||
Pose estimation is of great importance in many computer vision applications: robot navigation,
|
||||
augmented reality, and many more. This process is based on finding correspondences between points in
|
||||
the real environment and their 2d image projection. This is usually a difficult step, and thus it is
|
||||
common to use synthetic or fiducial markers to make it easier.
|
||||
|
||||
One of the most popular approaches is the use of binary square fiducial markers. The main benefit
|
||||
of these markers is that a single marker provides enough correspondences (its four corners)
|
||||
to obtain the camera pose. Also, the inner binary codification makes them specially robust, allowing
|
||||
the possibility of applying error detection and correction techniques.
|
||||
|
||||
The aruco module is based on the [ArUco library](https://www.uco.es/investiga/grupos/ava/portfolio/aruco/),
|
||||
a popular library for detection of square fiducial markers developed by Rafael Muñoz and Sergio Garrido @cite Aruco2014.
|
||||
|
||||
The aruco functionalities are included in:
|
||||
@code{.cpp}
|
||||
#include <opencv2/objdetect/aruco_detector.hpp>
|
||||
@endcode
|
||||
|
||||
|
||||
Markers and Dictionaries
|
||||
------------------------
|
||||
|
||||
An ArUco marker is a synthetic square marker composed by a wide black border and an inner binary
|
||||
matrix which determines its identifier (id). The black border facilitates its fast detection in the
|
||||
image and the binary codification allows its identification and the application of error detection
|
||||
and correction techniques. The marker size determines the size of the internal matrix. For instance
|
||||
a marker size of 4x4 is composed by 16 bits.
|
||||
|
||||
Some examples of ArUco markers:
|
||||
|
||||

|
||||
|
||||
It must be noted that a marker can be found rotated in the environment, however, the detection
|
||||
process needs to be able to determine its original rotation, so that each corner is identified
|
||||
unequivocally. This is also done based on the binary codification.
|
||||
|
||||
A dictionary of markers is the set of markers that are considered in a specific application. It is
|
||||
simply the list of binary codifications of each of its markers.
|
||||
|
||||
The main properties of a dictionary are the dictionary size and the marker size.
|
||||
|
||||
- The dictionary size is the number of markers that compose the dictionary.
|
||||
- The marker size is the size of those markers (the number of bits/modules).
|
||||
|
||||
The aruco module includes some predefined dictionaries covering a range of different dictionary
|
||||
sizes and marker sizes.
|
||||
|
||||
One may think that the marker id is the number obtained from converting the binary codification to
|
||||
a decimal base number. However, this is not possible since for high marker sizes the number of bits
|
||||
is too high and managing such huge numbers is not practical. Instead, a marker id is simply
|
||||
the marker index within the dictionary it belongs to. For instance, the first 5 markers in a
|
||||
dictionary have the ids: 0, 1, 2, 3 and 4.
|
||||
|
||||
More information about dictionaries is provided in the "Selecting a dictionary" section.
|
||||
|
||||
|
||||
Marker Creation
|
||||
---------------
|
||||
|
||||
Before their detection, markers need to be printed in order to be placed in the environment.
|
||||
Marker images can be generated using the `generateImageMarker()` function.
|
||||
|
||||
For example, lets analyze the following call:
|
||||
|
||||
@code{.cpp}
|
||||
cv::Mat markerImage;
|
||||
cv::aruco::Dictionary dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250);
|
||||
cv::aruco::generateImageMarker(dictionary, 23, 200, markerImage, 1);
|
||||
cv::imwrite("marker23.png", markerImage);
|
||||
@endcode
|
||||
|
||||
First, the `cv::aruco::Dictionary` object is created by choosing one of the predefined dictionaries in the aruco module.
|
||||
Concretely, this dictionary is composed of 250 markers and a marker size of 6x6 bits (`cv::aruco::DICT_6X6_250`).
|
||||
|
||||
The parameters of `cv::aruco::generateImageMarker()` are:
|
||||
|
||||
- The first parameter is the `cv::aruco::Dictionary` object previously created.
|
||||
- The second parameter is the marker id, in this case the marker 23 of the dictionary `cv::aruco::DICT_6X6_250`.
|
||||
Note that each dictionary is composed of a different number of markers. In this case, the valid ids
|
||||
go from 0 to 249. Any specific id out of the valid range will produce an exception.
|
||||
- The third parameter, 200, is the size of the output marker image. In this case, the output image
|
||||
will have a size of 200x200 pixels. Note that this parameter should be large enough to store the
|
||||
number of bits for the specific dictionary. So, for instance, you cannot generate an image of
|
||||
5x5 pixels for a marker size of 6x6 bits (and that is without considering the marker border).
|
||||
Furthermore, to avoid deformations, this parameter should be proportional to the number of bits +
|
||||
border size, or at least much higher than the marker size (like 200 in the example), so that
|
||||
deformations are insignificant.
|
||||
- The fourth parameter is the output image.
|
||||
- Finally, the last parameter is an optional parameter to specify the width of the marker black
|
||||
border. The size is specified proportional to the number of bits. For instance a value of 2 means
|
||||
that the border will have a width equivalent to the size of two internal bits. The default value
|
||||
is 1.
|
||||
|
||||
The generated image is:
|
||||
|
||||

|
||||
|
||||
A full working example is included in the `create_marker.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples now take input from the command line using cv::CommandLineParser. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
"marker23.png" -d=10 -id=23
|
||||
@endcode
|
||||
Parameters for `create_marker.cpp`:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/create_marker.cpp aruco_create_markers_keys
|
||||
|
||||
Marker Detection
|
||||
----------------
|
||||
|
||||
Given an image containing ArUco markers, the detection process has to return a list of
|
||||
detected markers. Each detected marker includes:
|
||||
|
||||
- The position of its four corners in the image (in their original order).
|
||||
- The id of the marker.
|
||||
|
||||
The marker detection process is comprised of two main steps:
|
||||
|
||||
1. Detection of marker candidates. In this step the image is analyzed in order to find square shapes
|
||||
that are candidates to be markers. It begins with an adaptive thresholding to segment the markers,
|
||||
then contours are extracted from the thresholded image and those that are not convex or do not
|
||||
approximate to a square shape are discarded. Some extra filtering is also applied (removing contours
|
||||
that are too small or too big, removing contours too close to each other, etc).
|
||||
|
||||
2. After the candidate detection, it is necessary to determine if they are actually markers by
|
||||
analyzing their inner codification. This step starts by extracting the marker bits of each marker.
|
||||
To do so, a perspective transformation is first applied to obtain the marker in its canonical form.
|
||||
Then, the canonical image is thresholded using Otsu to separate white and black bits. The image
|
||||
is divided into different cells according to the marker size and the border size. Then the number
|
||||
of black or white pixels in each cell is counted to determine if it is a white or a black bit.
|
||||
Finally, the bits are analyzed to determine if the marker belongs to the specific dictionary.
|
||||
Error correction techniques are employed when necessary.
|
||||
|
||||
|
||||
Consider the following image:
|
||||
|
||||
 { width=70% }
|
||||
|
||||
And a printout of this image in a photo:
|
||||
|
||||

|
||||
|
||||
These are the detected markers (in green). Note that some markers are rotated. The small red square
|
||||
indicates the marker’s top left corner:
|
||||
|
||||

|
||||
|
||||
And these are the marker candidates that have been rejected during the identification step (in pink):
|
||||
|
||||

|
||||
|
||||
In the aruco module, the detection is performed in the `cv::aruco::ArucoDetector::detectMarkers()`
|
||||
function. This function is the most important in the module, since all the rest of the functionality
|
||||
is based on the detected markers returned by `cv::aruco::ArucoDetector::detectMarkers()`.
|
||||
|
||||
An example of marker detection:
|
||||
|
||||
@code{.cpp}
|
||||
cv::Mat inputImage;
|
||||
// ... read inputImage ...
|
||||
std::vector<int> markerIds;
|
||||
std::vector<std::vector<cv::Point2f>> markerCorners, rejectedCandidates;
|
||||
cv::aruco::DetectorParameters detectorParams = cv::aruco::DetectorParameters();
|
||||
cv::aruco::Dictionary dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250);
|
||||
cv::aruco::ArucoDetector detector(dictionary, detectorParams);
|
||||
detector.detectMarkers(inputImage, markerCorners, markerIds, rejectedCandidates);
|
||||
@endcode
|
||||
|
||||
When you create an `cv::aruco::ArucoDetector` object, you need to pass the following parameters to the constructor:
|
||||
|
||||
- A dictionary object, in this case one of the predefined dictionaries (`cv::aruco::DICT_6X6_250`).
|
||||
- Object of type `cv::aruco::DetectorParameters`. This object includes all parameters that can be customized during the detection process.
|
||||
These parameters will be explained in the next section.
|
||||
|
||||
The parameters of `cv::aruco::ArucoDetector::detectMarkers()` are:
|
||||
|
||||
- The first parameter is the image containing the markers to be detected.
|
||||
- The detected markers are stored in the `markerCorners` and `markerIds` structures:
|
||||
- `markerCorners` is the list of corners of the detected markers. For each marker, its four
|
||||
corners are returned in their original order (which is clockwise starting with top left).
|
||||
So, the first corner is the top left corner, followed by the top right, bottom right and bottom left.
|
||||
- `markerIds` is the list of ids of each of the detected markers in `markerCorners`.
|
||||
Note that the returned `markerCorners` and `markerIds` vectors have the same size.
|
||||
- The final optional parameter, `rejectedCandidates`, is a returned list of marker candidates, i.e.
|
||||
shapes that were found and considered but did not contain a valid marker. Each candidate is also
|
||||
defined by its four corners, and its format is the same as the `markerCorners` parameter. This
|
||||
parameter can be omitted and is only useful for debugging purposes and for ‘refind’ strategies
|
||||
(see `cv::aruco::ArucoDetector::refineDetectedMarkers()`).
|
||||
|
||||
|
||||
The next thing you probably want to do after `cv::aruco::ArucoDetector::detectMarkers()` is check that
|
||||
your markers have been correctly detected. Fortunately, the aruco module provides a function to draw
|
||||
the detected markers in the input image, this function is `drawDetectedMarkers()`. For example:
|
||||
|
||||
@code{.cpp}
|
||||
cv::Mat outputImage = inputImage.clone();
|
||||
cv::aruco::drawDetectedMarkers(outputImage, markerCorners, markerIds);
|
||||
@endcode
|
||||
|
||||
- `outputImage ` is the input/output image where the markers will be drawn (it will normally be
|
||||
the same as the image where the markers were detected).
|
||||
- `markerCorners` and `markerIds` are the structures of the detected markers returned by the
|
||||
`cv::aruco::ArucoDetector::detectMarkers()` function.
|
||||
|
||||

|
||||
|
||||
Note that this function is only provided for visualization and its use can be omitted.
|
||||
|
||||
With these two functions we can create a basic marker detection loop to detect markers from our
|
||||
camera:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_detect_markers
|
||||
|
||||
Note that some of the optional parameters have been omitted, like the detection parameter object and the
|
||||
output vector of rejected candidates.
|
||||
|
||||
A full working example is included in the `detect_markers.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples now take input from the command line using cv::CommandLineParser. For this file
|
||||
the example parameters will look like:
|
||||
@code{.cpp}
|
||||
-v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_detection/images/singlemarkersoriginal.jpg -d=10
|
||||
@endcode
|
||||
Parameters for `detect_markers.cpp`:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_detect_markers_keys
|
||||
|
||||
|
||||
Pose Estimation
|
||||
---------------
|
||||
|
||||
The next thing you'll probably want to do after detecting the markers is to use them to get the camera pose.
|
||||
|
||||
To perform camera pose estimation, you need to know your camera's calibration parameters. These are
|
||||
the camera matrix and distortion coefficients. If you do not know how to calibrate your camera,
|
||||
you can take a look at the `calibrateCamera()` function and the Calibration tutorial of OpenCV.
|
||||
You can also calibrate your camera using the aruco module as explained in the **Calibration with ArUco and ChArUco**
|
||||
tutorial. Note that this only needs to be done once unless the camera optics are modified
|
||||
(for instance changing its focus).
|
||||
|
||||
As a result of the calibration, you get a camera matrix: a matrix of 3x3 elements with the
|
||||
focal distances and the camera center coordinates (a.k.a intrinsic parameters), and the distortion
|
||||
coefficients: a vector of 5 or more elements that models the distortion produced by your camera.
|
||||
|
||||
When you estimate the pose with ArUco markers, you can estimate the pose of each marker individually.
|
||||
If you want to estimate one pose from a set of markers, use ArUco Boards (see the **Detection of ArUco
|
||||
Boards** tutorial). Using ArUco boards instead of single markers allows some markers to be occluded.
|
||||
|
||||
The camera pose relative to the marker is a 3d transformation from the marker coordinate system to the
|
||||
camera coordinate system. It is specified by rotation and translation vectors. OpenCV provides
|
||||
`cv::solvePnP()` function to do that.
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_pose_estimation1
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_pose_estimation2
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_pose_estimation3
|
||||
|
||||
- The `corners` parameter is the vector of marker corners returned by the `cv::aruco::ArucoDetector::detectMarkers()` function.
|
||||
- The second parameter is the size of the marker side in meters or in any other unit. Note that the
|
||||
translation vectors of the estimated poses will be in the same units.
|
||||
- `camMatrix` and `distCoeffs` are the camera calibration parameters that were created during
|
||||
the camera calibration process.
|
||||
- The output parameters `rvecs` and `tvecs` are the rotation and translation vectors respectively,
|
||||
for each of the detected markers in `corners`.
|
||||
|
||||
The marker coordinate system that is assumed by this function is placed in the center (by default) or
|
||||
in the top left corner of the marker with the Z axis pointing out, as in the following image.
|
||||
Axis-color correspondences are X: red, Y: green, Z: blue. Note the axis directions of the rotated
|
||||
markers in this image.
|
||||
|
||||

|
||||
|
||||
OpenCV provides a function to draw the axis as in the image above, so pose estimation can be
|
||||
checked:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_draw_pose_estimation
|
||||
|
||||
- `imageCopy` is the input/output image where the detected markers will be shown.
|
||||
- `camMatrix` and `distCoeffs` are the camera calibration parameters.
|
||||
- `rvecs[i]` and `tvecs[i]` are the rotation and translation vectors respectively, for each of the detected markers.
|
||||
- The last parameter is the length of the axis, in the same unit as tvec (usually meters).
|
||||
|
||||
Sample video:
|
||||
|
||||
@youtube{IsXWrcB_Hvs}
|
||||
|
||||
A full working example is included in the `detect_markers.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples now take input from the command line using cv::CommandLineParser. For this file
|
||||
the example parameters will look like:
|
||||
@code{.cpp}
|
||||
-v=/path_to_opencv/opencv/doc/tutorials/objdetect/aruco_detection/images/singlemarkersoriginal.jpg -d=10
|
||||
-c=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml
|
||||
@endcode
|
||||
Parameters for `detect_markers.cpp`:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_markers.cpp aruco_detect_markers_keys
|
||||
|
||||
Selecting a dictionary
|
||||
----------------------
|
||||
|
||||
The aruco module provides the `Dictionary` class to represent a dictionary of markers.
|
||||
|
||||
In addition to the marker size and the number of markers in the dictionary, there is another important
|
||||
parameter of the dictionary - the inter-marker distance. The inter-marker distance is the minimum
|
||||
Hamming distance between dictionary markers that determines the dictionary's ability to detect and
|
||||
correct errors.
|
||||
|
||||
In general, smaller dictionary sizes and larger marker sizes increase the inter-marker distance and
|
||||
vice versa. However, the detection of markers with larger sizes is more difficult due to the higher
|
||||
number of bits that need to be extracted from the image.
|
||||
|
||||
For instance, if you need only 10 markers in your application, it is better to use a dictionary composed
|
||||
only of those 10 markers than using a dictionary composed of 1000 markers. The reason is that
|
||||
the dictionary composed of 10 markers will have a higher inter-marker distance and, thus, it will be
|
||||
more robust to errors.
|
||||
|
||||
As a consequence, the aruco module includes several ways to select your dictionary of markers, so that
|
||||
you can increase your system robustness:
|
||||
|
||||
### Predefined dictionaries
|
||||
|
||||
This is the easiest way to select a dictionary. The aruco module includes a set of predefined
|
||||
dictionaries in a variety of marker sizes and number of markers. For instance:
|
||||
|
||||
@code{.cpp}
|
||||
cv::aruco::Dictionary dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250);
|
||||
@endcode
|
||||
|
||||
`cv::aruco::DICT_6X6_250` is an example of predefined dictionary of markers with 6x6 bits and a total of 250
|
||||
markers.
|
||||
|
||||
From all the provided dictionaries, it is recommended to choose the smallest one that fits your application.
|
||||
For instance, if you need 200 markers of 6x6 bits, it is better to use `cv::aruco::DICT_6X6_250` than `cv::aruco::DICT_6X6_1000`.
|
||||
The smaller the dictionary, the higher the inter-marker distance.
|
||||
|
||||
The list of available predefined dictionaries can be found in the documentation for the `PredefinedDictionaryType` enum.
|
||||
|
||||
### Automatic dictionary generation
|
||||
|
||||
A dictionary can be generated automatically to adjust the desired number of markers and bits
|
||||
to optimize the inter-marker distance:
|
||||
|
||||
@code{.cpp}
|
||||
cv::aruco::Dictionary dictionary = cv::aruco::extendDictionary(36, 5);
|
||||
@endcode
|
||||
|
||||
This will generate a customized dictionary composed of 36 markers of 5x5 bits. The process can take several
|
||||
seconds, depending on the parameters (it is slower for larger dictionaries and higher numbers of bits).
|
||||
|
||||
Also you could use `aruco_dict_utils.cpp` sample inside the `opencv/samples/cpp`. This sample calculates
|
||||
the minimum Hamming distance for the generated dictionary and also allows you to create markers that are
|
||||
resistant to reflection.
|
||||
|
||||
### Manual dictionary definition
|
||||
|
||||
Finally, the dictionary can be configured manually, so that any encoding can be used. To do that,
|
||||
the `cv::aruco::Dictionary` object parameters need to be assigned manually. It must be noted that,
|
||||
unless you have a special reason to do this manually, it is preferable to use one of the previous alternatives.
|
||||
|
||||
The `cv::aruco::Dictionary` parameters are:
|
||||
|
||||
@code{.cpp}
|
||||
class Dictionary {
|
||||
public:
|
||||
|
||||
cv::Mat bytesList; // marker code information
|
||||
int markerSize; // number of bits per dimension
|
||||
int maxCorrectionBits; // maximum number of bits that can be corrected
|
||||
|
||||
...
|
||||
|
||||
}
|
||||
@endcode
|
||||
|
||||
`bytesList` is the array that contains all the information about the marker codes. `markerSize` is the size
|
||||
of each marker dimension (for instance, 5 for markers with 5x5 bits). Finally, `maxCorrectionBits` is
|
||||
the maximum number of erroneous bits that can be corrected during the marker detection. If this value is too
|
||||
high, it can lead to a high number of false positives.
|
||||
|
||||
Each row in `bytesList` represents one of the dictionary markers. However, the markers are not stored in their
|
||||
binary form, instead they are stored in a special format to simplify their detection.
|
||||
|
||||
Fortunately, a marker can be easily transformed to this form using the static method `Dictionary::getByteListFromBits()`.
|
||||
|
||||
For example:
|
||||
|
||||
@code{.cpp}
|
||||
cv::aruco::Dictionary dictionary;
|
||||
|
||||
// Markers of 6x6 bits
|
||||
dictionary.markerSize = 6;
|
||||
|
||||
// Maximum number of bit corrections
|
||||
dictionary.maxCorrectionBits = 3;
|
||||
|
||||
// Let's create a dictionary of 100 markers
|
||||
for(int i = 0; i < 100; i++)
|
||||
{
|
||||
// Assume generateMarkerBits() generates a new marker in binary format, so that
|
||||
// markerBits is a 6x6 matrix of CV_8UC1 type, only containing 0s and 1s
|
||||
cv::Mat markerBits = generateMarkerBits();
|
||||
cv::Mat markerCompressed = cv::aruco::Dictionary::getByteListFromBits(markerBits);
|
||||
|
||||
// Add the marker as a new row
|
||||
dictionary.bytesList.push_back(markerCompressed);
|
||||
}
|
||||
@endcode
|
||||
|
||||
Detector Parameters
|
||||
-------------------
|
||||
|
||||
One of the parameters of `cv::aruco::ArucoDetector` is a `cv::aruco::DetectorParameters` object. This object
|
||||
includes all the options that can be customized during the marker detection process.
|
||||
|
||||
This section describes each detector parameter. The parameters can be classified depending on
|
||||
the process in which they’re involved:
|
||||
|
||||
### Thresholding
|
||||
|
||||
One of the first steps in the marker detection process is adaptive thresholding of the input image.
|
||||
|
||||
For instance, the thresholded image for the sample image used above is:
|
||||
|
||||

|
||||
|
||||
This thresholding can be customized with the following parameters:
|
||||
|
||||
#### adaptiveThreshWinSizeMin, adaptiveThreshWinSizeMax, and adaptiveThreshWinSizeStep
|
||||
|
||||
The `adaptiveThreshWinSizeMin` and `adaptiveThreshWinSizeMax` parameters represent the interval where the
|
||||
thresholding window sizes (in pixels) are selected for the adaptive thresholding (see OpenCV
|
||||
`threshold()` and `adaptiveThreshold()` functions for more details).
|
||||
|
||||
The parameter `adaptiveThreshWinSizeStep` indicates the increments of the window size from
|
||||
`adaptiveThreshWinSizeMin` to `adaptiveThreshWinSizeMax`.
|
||||
|
||||
For instance, for the values `adaptiveThreshWinSizeMin` = 5 and `adaptiveThreshWinSizeMax` = 21 and
|
||||
`adaptiveThreshWinSizeStep` = 4, there will be 5 thresholding steps with window sizes 5, 9, 13, 17 and 21.
|
||||
On each thresholding image, marker candidates will be extracted.
|
||||
|
||||
Low values of window size can "break" the marker border if the marker size is too large, causing it to not be detected, as in the following image:
|
||||
|
||||

|
||||
|
||||
On the other hand, too large values can produce the same effect if the markers are too small, and can also
|
||||
reduce the performance. Moreover the process will tend to global thresholding, resulting in a loss of adaptive benefits.
|
||||
|
||||
The simplest case is using the same value for `adaptiveThreshWinSizeMin` and
|
||||
`adaptiveThreshWinSizeMax`, which produces a single thresholding step. However, it is usually better to use a
|
||||
range of values for the window size, although many thresholding steps can also reduce the performance considerably.
|
||||
|
||||
@see cv::aruco::DetectorParameters::adaptiveThreshWinSizeMin, cv::aruco::DetectorParameters::adaptiveThreshWinSizeMax,
|
||||
cv::aruco::DetectorParameters::adaptiveThreshWinSizeStep
|
||||
|
||||
#### adaptiveThreshConstant
|
||||
|
||||
The `adaptiveThreshConstant` parameter represents the constant value added in the thresholding operation (see OpenCV
|
||||
`threshold()` and `adaptiveThreshold()` functions for more details). Its default value is a good option in most cases.
|
||||
|
||||
@see cv::aruco::DetectorParameters::adaptiveThreshConstant
|
||||
|
||||
|
||||
### Contour filtering
|
||||
|
||||
After thresholding, contours are detected. However, not all contours
|
||||
are considered as marker candidates. They are filtered out in different steps so that contours that are
|
||||
very unlikely to be markers are discarded. The parameters in this section customize
|
||||
this filtering process.
|
||||
|
||||
It must be noted that in most cases it is a question of balance between detection capacity
|
||||
and performance. All the considered contours will be processed in the following stages, which usually have
|
||||
a higher computational cost. So, it is preferred to discard invalid candidates in this stage than in the later stages.
|
||||
|
||||
On the other hand, if the filtering conditions are too strict, the real marker contours could be discarded and,
|
||||
hence, not detected.
|
||||
|
||||
#### minMarkerPerimeterRate and maxMarkerPerimeterRate
|
||||
|
||||
These parameters determine the minimum and maximum size of a marker, specifically the minimum and
|
||||
maximum marker perimeter. They are not specified in absolute pixel values, instead they are specified
|
||||
relative to the maximum dimension of the input image.
|
||||
|
||||
For instance, a image with size 640x480 and a minimum relative marker perimeter of 0.05 will lead
|
||||
to a minimum marker perimeter of 640x0.05 = 32 pixels, since 640 is the maximum dimension of the
|
||||
image. The same applies for the `maxMarkerPerimeterRate` parameter.
|
||||
|
||||
If the `minMarkerPerimeterRate` is too low, detection performance can be significantly reduced,
|
||||
as many more contours will be considered for future stages.
|
||||
This penalization is not so noticeable for the `maxMarkerPerimeterRate` parameter, since there are
|
||||
usually many more small contours than big contours.
|
||||
A `minMarkerPerimeterRate` value of 0 and a `maxMarkerPerimeterRate` value of 4 (or more) will be
|
||||
equivalent to consider all the contours in the image, however this is not recommended for
|
||||
performance reasons.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minMarkerPerimeterRate, cv::aruco::DetectorParameters::maxMarkerPerimeterRate
|
||||
|
||||
#### polygonalApproxAccuracyRate
|
||||
|
||||
A polygonal approximation is applied to each candidate and only those that approximate to a square
|
||||
shape are accepted. This value determines the maximum error that the polygonal approximation can
|
||||
produce (see `approxPolyDP()` function for more information).
|
||||
|
||||
This parameter is relative to the candidate length (in pixels). So if the candidate has
|
||||
a perimeter of 100 pixels and the value of `polygonalApproxAccuracyRate` is 0.04, the maximum error
|
||||
would be 100x0.04=5.4 pixels.
|
||||
|
||||
In most cases, the default value works fine, but higher error values could be necessary for highly
|
||||
distorted images.
|
||||
|
||||
@see cv::aruco::DetectorParameters::polygonalApproxAccuracyRate
|
||||
|
||||
#### minCornerDistanceRate
|
||||
|
||||
Minimum distance between any pair of corners in the same marker. It is expressed relative to the marker
|
||||
perimeter. Minimum distance in pixels is Perimeter * minCornerDistanceRate.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minCornerDistanceRate
|
||||
|
||||
#### minMarkerDistanceRate
|
||||
|
||||
Minimum distance between any pair of corners from two different markers. It is expressed relative to
|
||||
the minimum marker perimeter of the two markers. If two candidates are too close, the smaller one is ignored.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minMarkerDistanceRate
|
||||
|
||||
#### minDistanceToBorder
|
||||
|
||||
Minimum distance to any of the marker corners to the image border (in pixels). Markers partially occluded
|
||||
by the image border can be correctly detected if the occlusion is small. However, if one of the corners
|
||||
is occluded, the returned corner is usually placed in a wrong position near the image border.
|
||||
|
||||
If the position of marker corners is important, for instance if you want to do pose estimation, it is
|
||||
better to discard any markers whose corners are too close to the image border. Elsewhere, it is not necessary.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minDistanceToBorder
|
||||
|
||||
### Bits Extraction
|
||||
|
||||
After candidate detection, the bits of each candidate are analyzed in order to determine if they
|
||||
are markers or not.
|
||||
|
||||
Before analyzing the binary code itself, the bits need to be extracted. To do this, perspective
|
||||
distortion is corrected and the resulting image is thresholded using Otsu threshold to separate
|
||||
black and white pixels.
|
||||
|
||||
This is an example of the image obtained after removing the perspective distortion of a marker:
|
||||
|
||||

|
||||
|
||||
Then, the image is divided into a grid with the same number of cells as the number of bits in the marker.
|
||||
In each cell, the number of black and white pixels are counted to determine the bit value assigned
|
||||
to the cell (from the majority value):
|
||||
|
||||

|
||||
|
||||
There are several parameters that can customize this process:
|
||||
|
||||
#### markerBorderBits
|
||||
|
||||
This parameter indicates the width of the marker border. It is relative to the size of each bit. So, a
|
||||
value of 2 indicates the border has the width of two internal bits.
|
||||
|
||||
This parameter needs to coincide with the border size of the markers you are using. The border size
|
||||
can be configured in the marker drawing functions such as `generateImageMarker()`.
|
||||
|
||||
@see cv::aruco::DetectorParameters::markerBorderBits
|
||||
|
||||
#### minOtsuStdDev
|
||||
|
||||
This value determines the minimum standard deviation of the pixel values to perform Otsu
|
||||
thresholding. If the deviation is low, it probably means that all the square is black (or white)
|
||||
and applying Otsu does not make sense. If this is the case, all the bits are set to 0 (or 1)
|
||||
depending on whether the mean value is higher or lower than 128.
|
||||
|
||||
@see cv::aruco::DetectorParameters::minOtsuStdDev
|
||||
|
||||
#### perspectiveRemovePixelPerCell
|
||||
|
||||
This parameter determines the number of pixels (per cell) in the obtained image after correcting perspective
|
||||
distortion (including the border). This is the size of the red squares in the image above.
|
||||
|
||||
For instance, let’s assume we are dealing with markers of 5x5 bits and border size of 1 bit
|
||||
(see `markerBorderBits`). Then, the total number of cells/bits per dimension is 5 + 2*1 = 7 (the border
|
||||
has to be counted twice). The total number of cells is 7x7.
|
||||
|
||||
If the value of `perspectiveRemovePixelPerCell` is 10, then the size of the obtained image will be
|
||||
10*7 = 70 -> 70x70 pixels.
|
||||
|
||||
A higher value of this parameter can improve the bits extraction process (up to some degree),
|
||||
however it can penalize the performance.
|
||||
|
||||
@see cv::aruco::DetectorParameters::perspectiveRemovePixelPerCell
|
||||
|
||||
#### perspectiveRemoveIgnoredMarginPerCell
|
||||
|
||||
When extracting the bits of each cell, the numbers of black and white pixels are counted. In general, it is
|
||||
not recommended to consider all the cell pixels. Instead it is better to ignore some pixels in the
|
||||
margins of the cells.
|
||||
|
||||
The reason for this is that, after removing the perspective distortion, the cells’ colors are, in general, not
|
||||
perfectly separated and white cells can invade some pixels of black cells (and vice versa). Thus, it is
|
||||
better to ignore some pixels just to avoid counting erroneous pixels.
|
||||
|
||||
For instance, in the following image:
|
||||
|
||||

|
||||
|
||||
only the pixels inside the green squares are considered. It can be seen in the right image that
|
||||
the resulting pixels contain a lower amount of noise from neighbor cells.
|
||||
The `perspectiveRemoveIgnoredMarginPerCell` parameter indicates the difference between the red and
|
||||
the green squares.
|
||||
|
||||
This parameter is relative to the total size of the cell. For instance if the cell size is 40 pixels and the
|
||||
value of this parameter is 0.1, a margin of 40*0.1=4 pixels is ignored in the cells. This means that the total
|
||||
number of pixels that would be analyzed in each cell would actually be 32x32, instead of 40x40.
|
||||
|
||||
@see cv::aruco::DetectorParameters::perspectiveRemoveIgnoredMarginPerCell
|
||||
|
||||
|
||||
### Marker identification
|
||||
|
||||
After the bits have been extracted, the next step is checking whether the extracted code belongs to the marker
|
||||
dictionary and, if necessary, error correction can be performed.
|
||||
|
||||
#### maxErroneousBitsInBorderRate
|
||||
|
||||
The bits of the marker border should be black. This parameter specifies the allowed number of erroneous
|
||||
bits in the border, i.e. the maximum number of white bits in the border. It is represented
|
||||
relative to the total number of bits in the marker.
|
||||
|
||||
@see cv::aruco::DetectorParameters::maxErroneousBitsInBorderRate
|
||||
|
||||
#### errorCorrectionRate
|
||||
|
||||
Each marker dictionary has a theoretical maximum number of bits that can be corrected (`Dictionary.maxCorrectionBits`).
|
||||
However, this value can be modified by the `errorCorrectionRate` parameter.
|
||||
|
||||
For instance, if the allowed number of bits that can be corrected (for the used dictionary) is 6 and the value of `errorCorrectionRate` is
|
||||
0.5, the real maximum number of bits that can be corrected is 6*0.5=3 bits.
|
||||
|
||||
This value is useful to reduce the error correction capabilities in order to avoid false positives.
|
||||
|
||||
@see cv::aruco::DetectorParameters::errorCorrectionRate
|
||||
|
||||
|
||||
### Corner Refinement
|
||||
|
||||
After markers have been detected and identified, the last step is performing subpixel refinement
|
||||
of the corner positions (see OpenCV `cornerSubPix()` and `cv::aruco::CornerRefineMethod`).
|
||||
|
||||
Note that this step is optional and it only makes sense if the positions of the marker corners have to
|
||||
be accurate, for instance for pose estimation. It is usually a time-consuming step and therefore is disabled by default.
|
||||
|
||||
#### cornerRefinementMethod
|
||||
|
||||
This parameter determines whether the corner subpixel process is performed or not and which method to use
|
||||
if it is being performed. It can be disabled if accurate corners are not necessary. Possible values are
|
||||
`CORNER_REFINE_NONE`, `CORNER_REFINE_SUBPIX`, `CORNER_REFINE_CONTOUR`, and `CORNER_REFINE_APRILTAG`.
|
||||
|
||||
@see cv::aruco::DetectorParameters::cornerRefinementMethod
|
||||
|
||||
#### cornerRefinementWinSize
|
||||
|
||||
This parameter determines the maximum window size for the corner refinement process.
|
||||
|
||||
High values can cause close corners of the image to be included in the window area, so that the corner
|
||||
of the marker moves to a different and incorrect location during the process. Also, it may affect performance.
|
||||
The window size may decrease if the ArUco marker is too small, check cv::aruco::DetectorParameters::relativeCornerRefinmentWinSize.
|
||||
The final window size is calculated as: min(cornerRefinementWinSize, averageArucoModuleSize*relativeCornerRefinmentWinSize),
|
||||
where averageArucoModuleSize is average module size of ArUco marker in pixels.
|
||||
|
||||
@see cv::aruco::DetectorParameters::cornerRefinementWinSize
|
||||
|
||||
#### relativeCornerRefinmentWinSize
|
||||
|
||||
Dynamic window size for corner refinement relative to Aruco module size (default 0.3).
|
||||
|
||||
The final window size is calculated as: min(cornerRefinementWinSize, averageArucoModuleSize*relativeCornerRefinmentWinSize),
|
||||
where averageArucoModuleSize is average module size of ArUco marker in pixels.
|
||||
In the case of markers located far from each other, it may be useful to increase the value of the parameter to 0.4-0.5.
|
||||
In the case of markers located close to each other, it may be useful to decrease the parameter value to 0.1-0.2.
|
||||
|
||||
@see cv::aruco::DetectorParameters::relativeCornerRefinmentWinSize
|
||||
|
||||
#### cornerRefinementMaxIterations and cornerRefinementMinAccuracy
|
||||
|
||||
These two parameters determine the stop criteria of the subpixel refinement process. The
|
||||
`cornerRefinementMaxIterations` indicates the maximum number of iterations and
|
||||
`cornerRefinementMinAccuracy` the minimum error value before stopping the process.
|
||||
|
||||
If the number of iterations is too high, it may affect the performance. On the other hand, if it is
|
||||
too low, it can result in poor subpixel refinement.
|
||||
|
||||
@see cv::aruco::DetectorParameters::cornerRefinementMaxIterations, cv::aruco::DetectorParameters::cornerRefinementMinAccuracy
|
||||
|
After Width: | Height: | Size: 5.4 KiB |
|
After Width: | Height: | Size: 31 KiB |
|
After Width: | Height: | Size: 1.2 KiB |
|
After Width: | Height: | Size: 16 KiB |
|
After Width: | Height: | Size: 28 KiB |
|
After Width: | Height: | Size: 3.2 KiB |
|
After Width: | Height: | Size: 33 KiB |
|
After Width: | Height: | Size: 79 KiB |
|
After Width: | Height: | Size: 28 KiB |
|
After Width: | Height: | Size: 38 KiB |
|
After Width: | Height: | Size: 9.2 KiB |
@@ -0,0 +1,190 @@
|
||||
Aruco module FAQ {#tutorial_aruco_faq}
|
||||
================
|
||||
|
||||
@prev_tutorial{tutorial_aruco_calibration}
|
||||
|
||||
This is a compilation of questions that can be useful for those that want to use the aruco module.
|
||||
|
||||
- I only want to label some objects, what should I use?
|
||||
|
||||
In this case, you only need single ArUco markers. You can place one or several markers with different
|
||||
ids in each of the object you want to identify.
|
||||
|
||||
|
||||
- Which algorithm is used for marker detection?
|
||||
|
||||
The aruco module is based on the original ArUco library. A full description of the detection process
|
||||
can be found in:
|
||||
|
||||
> S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez. 2014.
|
||||
> "Automatic generation and detection of highly reliable fiducial markers under occlusion".
|
||||
> Pattern Recogn. 47, 6 (June 2014), 2280-2292. DOI=10.1016/j.patcog.2014.01.005
|
||||
|
||||
|
||||
- My markers are not being detected correctly, what can I do?
|
||||
|
||||
There can be many factors that avoid the correct detection of markers. You probably need to adjust
|
||||
some of the parameters in the `cv::aruco::DetectorParameters` object. The first thing you can do is
|
||||
checking if your markers are returned as rejected candidates by the `cv::aruco::ArucoDetector::detectMarkers()`
|
||||
function. Depending on this, you should try to modify different parameters.
|
||||
|
||||
If you are using a ArUco board, you can also try the `cv::aruco::ArucoDetector::refineDetectedMarkers()` function.
|
||||
If you are [using big markers](https://github.com/opencv/opencv_contrib/issues/2811) (400x400 pixels and more), try
|
||||
increasing `cv::aruco::DetectorParameters::adaptiveThreshWinSizeMax` value.
|
||||
Also avoid [narrow borders around the ArUco marker](https://github.com/opencv/opencv_contrib/issues/2492)
|
||||
(5% or less of the marker perimeter, adjusted by `cv::aruco::DetectorParameters::minMarkerDistanceRate`)
|
||||
around markers.
|
||||
|
||||
|
||||
- What are the benefits of ArUco boards? What are the drawbacks?
|
||||
|
||||
Using a board of markers you can obtain the camera pose from a set of markers, instead of a single one.
|
||||
This way, the detection is able to handle occlusion of partial views of the Board, since only one
|
||||
marker is necessary to obtain the pose.
|
||||
|
||||
Furthermore, as in most cases you are using more corners for pose estimation, it will be more
|
||||
accurate than using a single marker.
|
||||
|
||||
The main drawback is that a Board is not as versatile as a single marker.
|
||||
|
||||
|
||||
|
||||
- What are the benefits of ChArUco boards over ArUco boards? And the drawbacks?
|
||||
|
||||
ChArUco boards combines chessboards with ArUco boards. Thanks to this, the corners provided by
|
||||
ChArUco boards are more accurate than those provided by ArUco Boards (or single markers).
|
||||
|
||||
The main drawback is that ChArUco boards are not as versatile as ArUco board. For instance,
|
||||
a ChArUco board is a planar board with a specific marker layout while the ArUco boards can have
|
||||
any layout, even in 3d. Furthermore, the markers in the ChArUco board are usually smaller and
|
||||
more difficult to detect.
|
||||
|
||||
|
||||
- I do not need pose estimation, should I use ChArUco boards?
|
||||
|
||||
No. The main goal of ChArUco boards is provide high accurate corners for pose estimation or camera
|
||||
calibration.
|
||||
|
||||
|
||||
- Should all the markers in an ArUco board be placed in the same plane?
|
||||
|
||||
No, the marker corners in a ArUco board can be placed anywhere in its 3d coordinate system.
|
||||
|
||||
|
||||
- Should all the markers in an ChArUco board be placed in the same plane?
|
||||
|
||||
Yes, all the markers in a ChArUco board need to be in the same plane and their layout is fixed by
|
||||
the chessboard shape.
|
||||
|
||||
|
||||
- What is the difference between a `cv::aruco::Board` object and a `cv::aruco::GridBoard` object?
|
||||
|
||||
The `cv::aruco::GridBoard` class is a specific type of board that inherits from `cv::aruco::Board` class.
|
||||
A `cv::aruco::GridBoard` object is a board whose markers are placed in the same plane and in a grid layout.
|
||||
|
||||
|
||||
- What are Diamond markers?
|
||||
|
||||
Diamond markers are very similar to a ChArUco board of 3x3 squares. However, contrary to ChArUco boards,
|
||||
the detection of diamonds is based on the relative position of the markers.
|
||||
They are useful when you want to provide a conceptual meaning to any (or all) of the markers in
|
||||
the diamond. An example is using one of the marker to provide the diamond scale.
|
||||
|
||||
|
||||
- Do I need to detect marker before board detection, ChArUco board detection or Diamond detection?
|
||||
|
||||
Yes, the detection of single markers is a basic tool in the aruco module. It is done using the
|
||||
`cv::aruco::DetectorParameters::detectMarkers()` function. The rest of functionalities receives
|
||||
a list of detected markers from this function.
|
||||
|
||||
|
||||
- I want to calibrate my camera, can I use this module?
|
||||
|
||||
Yes, the aruco module provides functionalities to calibrate the camera using both, ArUco boards and
|
||||
ChArUco boards.
|
||||
|
||||
|
||||
- Should I calibrate using a ChArUco board or an ArUco board?
|
||||
|
||||
It is highly recommended the calibration using ChArUco board due to the high accuracy.
|
||||
|
||||
|
||||
- Should I use a predefined dictionary or generate my own dictionary?
|
||||
|
||||
In general, it is easier to use one of the predefined dictionaries. However, if you need a bigger
|
||||
dictionary (in terms of number of markers or number of bits) you should generate your own dictionary.
|
||||
Dictionary generation is also useful if you want to maximize the inter-marker distance to achieve
|
||||
a better error correction during the identification step.
|
||||
|
||||
- I am generating my own dictionary but it takes too long
|
||||
|
||||
Dictionary generation should only be done once at the beginning of your application and it should take
|
||||
some seconds. If you are generating the dictionary on each iteration of your detection loop, you are
|
||||
doing it wrong.
|
||||
|
||||
Furthermore, it is recommendable to save the dictionary to a file with `cv::aruco::Dictionary::writeDictionary()`
|
||||
and read it with `cv::aruco::Dictionary::readDictionary()` on every execution, so you don't need
|
||||
to generate it.
|
||||
|
||||
|
||||
- I would like to use some markers of the original ArUco library that I have already printed, can I use them?
|
||||
|
||||
Yes, one of the predefined dictionary is `cv::aruco::DICT_ARUCO_ORIGINAL`, which detects the marker
|
||||
of the original ArUco library with the same identifiers.
|
||||
|
||||
|
||||
- Can I use the Board configuration file of the original ArUco library in this module?
|
||||
|
||||
Not directly, you will need to adapt the information of the ArUco file to the aruco module Board format.
|
||||
|
||||
|
||||
- Can I use this module to detect the markers of other libraries based on binary fiducial markers?
|
||||
|
||||
Probably yes, however you will need to port the dictionary of the original library to the aruco module format.
|
||||
|
||||
|
||||
- Do I need to store the Dictionary information in a file so I can use it in different executions?
|
||||
|
||||
If you are using one of the predefined dictionaries, it is not necessary. Otherwise, it is recommendable
|
||||
that you save it to file.
|
||||
|
||||
|
||||
- Do I need to store the Board information in a file so I can use it in different executions?
|
||||
|
||||
If you are using a `cv::aruco::GridBoard` or a `cv::aruco::CharucoBoard` you only need to store
|
||||
the board measurements that are provided to the `cv::aruco::GridBoard::GridBoard()` constructor or
|
||||
in or `cv::aruco::CharucoBoard` constructor. If you manually modify the marker ids of the boards,
|
||||
or if you use a different type of board, you should save your board object to file.
|
||||
|
||||
- Does the aruco module provide functions to save the Dictionary or Board to file?
|
||||
|
||||
You can use `cv::aruco::Dictionary::writeDictionary()` and `cv::aruco::Dictionary::readDictionary()`
|
||||
for `cv::aruco::Dictionary`. The data member of board classes are public and can be easily stored.
|
||||
|
||||
|
||||
- Alright, but how can I render a 3d model to create an augmented reality application?
|
||||
|
||||
To do so, you will need to use an external rendering engine library, such as OpenGL. The aruco module
|
||||
only provides the functionality to obtain the camera pose, i.e. the rotation and translation vectors,
|
||||
which is necessary to create the augmented reality effect. However, you will need to adapt the rotation
|
||||
and translation vectors from the OpenCV format to the format accepted by your 3d rendering library.
|
||||
The original ArUco library contains examples of how to do it for OpenGL and Ogre3D.
|
||||
|
||||
|
||||
- I have use this module in my research work, how can I cite it?
|
||||
|
||||
You can cite the original ArUco library:
|
||||
|
||||
> S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez. 2014.
|
||||
> "Automatic generation and detection of highly reliable fiducial markers under occlusion".
|
||||
> Pattern Recogn. 47, 6 (June 2014), 2280-2292. DOI=10.1016/j.patcog.2014.01.005
|
||||
|
||||
- Pose estimation markers are not being detected correctly, what can I do?
|
||||
|
||||
It is important to remark that the estimation of the pose using only 4 coplanar points is subject to ambiguity.
|
||||
In general, the ambiguity can be solved, if the camera is near to the marker.
|
||||
However, as the marker becomes small, the errors in the corner estimation grows and ambiguity comes
|
||||
as a problem. Try increasing the size of the marker you're using, and you can also try non-symmetrical
|
||||
(aruco_dict_utils.cpp) markers to avoid collisions. Use multiple markers (ArUco/ChArUco/Diamonds boards)
|
||||
and pose estimation with solvePnP() with the `cv::SOLVEPNP_IPPE_SQUARE` option.
|
||||
More in [this issue](https://github.com/opencv/opencv/issues/8813).
|
||||
@@ -0,0 +1,265 @@
|
||||
Detection of ChArUco Boards {#tutorial_charuco_detection}
|
||||
===========================
|
||||
|
||||
@prev_tutorial{tutorial_aruco_board_detection}
|
||||
@next_tutorial{tutorial_charuco_diamond_detection}
|
||||
|
||||
ArUco markers and boards are very useful due to their fast detection and their versatility.
|
||||
However, one of the problems of ArUco markers is that the accuracy of their corner positions is not
|
||||
too high, even after applying subpixel refinement.
|
||||
|
||||
On the contrary, the corners of chessboard patterns can be refined more accurately since each corner
|
||||
is surrounded by two black squares. However, finding a chessboard pattern is not as versatile as
|
||||
finding an ArUco board: it has to be completely visible and occlusions are not permitted.
|
||||
|
||||
A ChArUco board tries to combine the benefits of these two approaches:
|
||||
|
||||

|
||||
|
||||
The ArUco part is used to interpolate the position of the chessboard corners, so that it has the
|
||||
versatility of marker boards, since it allows occlusions or partial views. Moreover, since the
|
||||
interpolated corners belong to a chessboard, they are very accurate in terms of subpixel accuracy.
|
||||
|
||||
When high precision is necessary, such as in camera calibration, Charuco boards are a better option
|
||||
than standard ArUco boards.
|
||||
|
||||
Goal
|
||||
----
|
||||
|
||||
In this tutorial you will learn:
|
||||
|
||||
- How to create a charuco board ?
|
||||
- How to detect the charuco corners without performing camera calibration ?
|
||||
- How to detect the charuco corners with camera calibration and pose estimation ?
|
||||
|
||||
Source code
|
||||
-----------
|
||||
|
||||
You can find this code in `samples/cpp/tutorial_code/objectDetection/detect_board_charuco.cpp`
|
||||
|
||||
Here's a sample code of how to achieve all the stuff enumerated at the goal list.
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board_charuco.cpp charuco_detect_board_full_sample
|
||||
|
||||
ChArUco Board Creation
|
||||
----------------------
|
||||
|
||||
The aruco module provides the `cv::aruco::CharucoBoard` class that represents a Charuco Board and
|
||||
which inherits from the `cv::aruco::Board` class.
|
||||
|
||||
This class, as the rest of ChArUco functionalities, are defined in:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board_charuco.cpp charucohdr
|
||||
|
||||
To define a `cv::aruco::CharucoBoard`, it is necessary:
|
||||
|
||||
- Number of chessboard squares in X and Y directions.
|
||||
- Length of square side.
|
||||
- Length of marker side.
|
||||
- The dictionary of the markers.
|
||||
- Ids of all the markers.
|
||||
|
||||
As for the `cv::aruco::GridBoard` objects, the aruco module provides to create `cv::aruco::CharucoBoard`
|
||||
easily. This object can be easily created from these parameters using the `cv::aruco::CharucoBoard`
|
||||
constructor:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/create_board_charuco.cpp create_charucoBoard
|
||||
|
||||
- The first parameter is the number of squares in X and Y direction respectively.
|
||||
- The second and third parameters are the length of the squares and the markers respectively. They can
|
||||
be provided in any unit, having in mind that the estimated pose for this board would be measured
|
||||
in the same units (usually meters are used).
|
||||
- Finally, the dictionary of the markers is provided.
|
||||
|
||||
The ids of each of the markers are assigned by default in ascending order and starting on 0, like in
|
||||
`cv::aruco::GridBoard` constructor. This can be easily customized by accessing to the ids vector
|
||||
through `board.ids`, like in the `cv::aruco::Board` parent class.
|
||||
|
||||
Once we have our `cv::aruco::CharucoBoard` object, we can create an image to print it. There are
|
||||
two ways to do this:
|
||||
1. By using the script `apps/pattern-tools/generate_pattern.py`, see @subpage tutorial_camera_calibration_pattern.
|
||||
2. By using the function `cv::aruco::CharucoBoard::generateImage()`.
|
||||
|
||||
The function `cv::aruco::CharucoBoard::generateImage()` is provided in cv::aruco::CharucoBoard class
|
||||
and can be called by using the following code:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/create_board_charuco.cpp generate_charucoBoard
|
||||
|
||||
- The first parameter is the size of the output image in pixels. If this is not proportional
|
||||
to the board dimensions, it will be centered on the image.
|
||||
- The second parameter is the output image with the charuco board.
|
||||
- The third parameter is the (optional) margin in pixels, so none of the markers are touching the
|
||||
image border.
|
||||
- Finally, the size of the marker border, similarly to `cv::aruco::generateImageMarker()` function.
|
||||
The default value is 1.
|
||||
|
||||
The output image will be something like this:
|
||||
|
||||

|
||||
|
||||
A full working example is included in the `create_board_charuco.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples `create_board_charuco.cpp` now take input via commandline via the `cv::CommandLineParser`.
|
||||
For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
"_output_path_/chboard.png" -w=5 -h=7 -sl=100 -ml=60 -d=10
|
||||
@endcode
|
||||
|
||||
|
||||
ChArUco Board Detection
|
||||
-----------------------
|
||||
|
||||
When you detect a ChArUco board, what you are actually detecting is each of the chessboard corners
|
||||
of the board.
|
||||
|
||||
Each corner on a ChArUco board has a unique identifier (id) assigned. These ids go from 0 to the total
|
||||
number of corners in the board.
|
||||
The steps of charuco board detection can be broken down to the following steps:
|
||||
|
||||
- **Taking input Image**
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board_charuco.cpp inputImg
|
||||
|
||||
The original image where the markers are to be detected. The image is necessary to perform subpixel
|
||||
refinement in the ChArUco corners.
|
||||
|
||||
- **Reading the camera calibration Parameters(only for detection with camera calibration)**
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/aruco_samples_utility.hpp camDistCoeffs
|
||||
|
||||
The parameters of `readCameraParameters` are:
|
||||
- The first parameter is the path to the camera intrinsic matrix and distortion coefficients.
|
||||
- The second and third parameters are cameraMatrix and distCoeffs.
|
||||
|
||||
This function takes these parameters as input and returns a boolean value of whether the camera
|
||||
calibration parameters are valid or not. For detection of charuco corners without calibration,
|
||||
this step is not required.
|
||||
|
||||
- **Detecting the markers and interpolation of charuco corners from markers**
|
||||
|
||||
The detection of the ChArUco corners is based on the previous detected markers.
|
||||
So that, first markers are detected, and then ChArUco corners are interpolated from markers.
|
||||
The method that detect the ChArUco corners is `cv::aruco::CharucoDetector::detectBoard()`.
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board_charuco.cpp interpolateCornersCharuco
|
||||
|
||||
The parameters of detectBoard are:
|
||||
- `image` - Input image.
|
||||
- `charucoCorners` - output list of image positions of the detected corners.
|
||||
- `charucoIds` - output ids for each of the detected corners in `charucoCorners`.
|
||||
- `markerCorners` - input/output vector of detected marker corners.
|
||||
- `markerIds` - input/output vector of identifiers of the detected markers
|
||||
|
||||
If markerCorners and markerIds are empty, the function will detect aruco markers and ids.
|
||||
|
||||
If calibration parameters are provided, the ChArUco corners are interpolated by, first, estimating
|
||||
a rough pose from the ArUco markers and, then, reprojecting the ChArUco corners back to the image.
|
||||
|
||||
On the other hand, if calibration parameters are not provided, the ChArUco corners are interpolated
|
||||
by calculating the corresponding homography between the ChArUco plane and the ChArUco image projection.
|
||||
|
||||
The main problem of using homography is that the interpolation is more sensible to image distortion.
|
||||
Actually, the homography is only performed using the closest markers of each ChArUco corner to reduce
|
||||
the effect of distortion.
|
||||
|
||||
When detecting markers for ChArUco boards, and specially when using homography, it is recommended to
|
||||
disable the corner refinement of markers. The reason of this is that, due to the proximity of the
|
||||
chessboard squares, the subpixel process can produce important deviations in the corner positions and
|
||||
these deviations are propagated to the ChArUco corner interpolation, producing poor results.
|
||||
|
||||
@note To avoid deviations, the margin between chessboard square and aruco marker should be greater
|
||||
than 70% of one marker module.
|
||||
|
||||
Furthermore, only those corners whose two surrounding markers have be found are returned. If any of
|
||||
the two surrounding markers has not been detected, this usually means that there is some occlusion
|
||||
or the image quality is not good in that zone. In any case, it is preferable not to consider that
|
||||
corner, since what we want is to be sure that the interpolated ChArUco corners are very accurate.
|
||||
|
||||
After the ChArUco corners have been interpolated, a subpixel refinement is performed.
|
||||
|
||||
Once we have interpolated the ChArUco corners, we would probably want to draw them to see if their
|
||||
detections are correct. This can be easily done using the `cv::aruco::drawDetectedCornersCharuco()`
|
||||
function:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board_charuco.cpp drawDetectedCornersCharuco
|
||||
|
||||
- `imageCopy` is the image where the corners will be drawn (it will normally be the same image where
|
||||
the corners were detected).
|
||||
- The `outputImage` will be a clone of `inputImage` with the corners drawn.
|
||||
- `charucoCorners` and `charucoIds` are the detected Charuco corners from the `cv::aruco::CharucoDetector::detectBoard()`
|
||||
function.
|
||||
- Finally, the last parameter is the (optional) color we want to draw the corners with, of type `cv::Scalar`.
|
||||
|
||||
For this image:
|
||||
|
||||

|
||||
|
||||
The result will be:
|
||||
|
||||

|
||||
|
||||
In the presence of occlusion. like in the following image, although some corners are clearly visible,
|
||||
not all their surrounding markers have been detected due occlusion and, thus, they are not interpolated:
|
||||
|
||||

|
||||
|
||||
Sample video:
|
||||
|
||||
@youtube{Nj44m_N_9FY}
|
||||
|
||||
A full working example is included in the `detect_board_charuco.cpp` inside the
|
||||
`samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples `detect_board_charuco.cpp` now take input via commandline via the `cv::CommandLineParser`.
|
||||
For this file the example parameters will look like:
|
||||
@code{.cpp}
|
||||
-w=5 -h=7 -sl=0.04 -ml=0.02 -d=10 -v=/path_to_opencv/opencv/doc/tutorials/objdetect/charuco_detection/images/choriginal.jpg
|
||||
@endcode
|
||||
|
||||
ChArUco Pose Estimation
|
||||
-----------------------
|
||||
|
||||
The final goal of the ChArUco boards is finding corners very accurately for a high precision calibration
|
||||
or pose estimation.
|
||||
|
||||
The aruco module provides a function to perform ChArUco pose estimation easily. As in the
|
||||
`cv::aruco::GridBoard`, the coordinate system of the `cv::aruco::CharucoBoard` is placed in
|
||||
the board plane with the Z axis pointing in, and centered in the bottom left corner of the board.
|
||||
|
||||
@note After OpenCV 4.6.0, there was an incompatible change in the coordinate systems of the boards,
|
||||
now the coordinate systems are placed in the boards plane with the Z axis pointing in the plane
|
||||
(previously the axis pointed out the plane).
|
||||
`objPoints` in CW order correspond to the Z-axis pointing in the plane.
|
||||
`objPoints` in CCW order correspond to the Z-axis pointing out the plane.
|
||||
See PR https://github.com/opencv/opencv_contrib/pull/3174
|
||||
|
||||
|
||||
To perform pose estimation for charuco boards, you should use `cv::aruco::CharucoBoard::matchImagePoints()`
|
||||
and `cv::solvePnP()`:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_board_charuco.cpp poseCharuco
|
||||
|
||||
- The `charucoCorners` and `charucoIds` parameters are the detected charuco corners from the
|
||||
`cv::aruco::CharucoDetector::detectBoard()` function.
|
||||
- The `cameraMatrix` and `distCoeffs` are the camera calibration parameters which are necessary
|
||||
for pose estimation.
|
||||
- Finally, the `rvec` and `tvec` parameters are the output pose of the Charuco Board.
|
||||
- `cv::solvePnP()` returns true if the pose was correctly estimated and false otherwise.
|
||||
The main reason of failing is that there are not enough corners for pose estimation or
|
||||
they are in the same line.
|
||||
|
||||
The axis can be drawn using `cv::drawFrameAxes()` to check the pose is correctly estimated.
|
||||
The result would be: (X:red, Y:green, Z:blue)
|
||||
|
||||

|
||||
|
||||
A full working example is included in the `detect_board_charuco.cpp` inside the
|
||||
`samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples `detect_board_charuco.cpp` now take input via commandline via the `cv::CommandLineParser`.
|
||||
For this file the example parameters will look like:
|
||||
@code{.cpp}
|
||||
-w=5 -h=7 -sl=0.04 -ml=0.02 -d=10
|
||||
-v=/path_to_opencv/opencv/doc/tutorials/objdetect/charuco_detection/images/choriginal.jpg
|
||||
-c=/path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_charuco.yml
|
||||
@endcode
|
||||
|
After Width: | Height: | Size: 14 KiB |
|
After Width: | Height: | Size: 32 KiB |
|
After Width: | Height: | Size: 94 KiB |
|
After Width: | Height: | Size: 93 KiB |
|
After Width: | Height: | Size: 87 KiB |
|
After Width: | Height: | Size: 112 KiB |
|
After Width: | Height: | Size: 115 KiB |
@@ -0,0 +1,143 @@
|
||||
Detection of Diamond Markers {#tutorial_charuco_diamond_detection}
|
||||
==============================
|
||||
|
||||
@prev_tutorial{tutorial_charuco_detection}
|
||||
@next_tutorial{tutorial_aruco_calibration}
|
||||
|
||||
A ChArUco diamond marker (or simply diamond marker) is a chessboard composed by 3x3 squares and 4 ArUco markers inside the white squares.
|
||||
It is similar to a ChArUco board in appearance, however they are conceptually different.
|
||||
|
||||

|
||||
|
||||
In both, ChArUco board and Diamond markers, their detection is based on the previous detected ArUco
|
||||
markers. In the ChArUco case, the used markers are selected by directly looking their identifiers. This means
|
||||
that if a marker (included in the board) is found on a image, it will be automatically assumed to belong to the board. Furthermore,
|
||||
if a marker board is found more than once in the image, it will produce an ambiguity since the system won't
|
||||
be able to know which one should be used for the Board.
|
||||
|
||||
On the other hand, the detection of Diamond marker is not based on the identifiers. Instead, their detection
|
||||
is based on the relative position of the markers. As a consequence, marker identifiers can be repeated in the
|
||||
same diamond or among different diamonds, and they can be detected simultaneously without ambiguity. However,
|
||||
due to the complexity of finding marker based on their relative position, the diamond markers are limited to
|
||||
a size of 3x3 squares and 4 markers.
|
||||
|
||||
As in a single ArUco marker, each Diamond marker is composed by 4 corners and a identifier. The four corners
|
||||
correspond to the 4 chessboard corners in the marker and the identifier is actually an array of 4 numbers, which are
|
||||
the identifiers of the four ArUco markers inside the diamond.
|
||||
|
||||
Diamond markers are useful in those scenarios where repeated markers should be allowed. For instance:
|
||||
|
||||
- To increase the number of identifiers of single markers by using diamond marker for labeling. They would allow
|
||||
up to N^4 different ids, being N the number of markers in the used dictionary.
|
||||
|
||||
- Give to each of the four markers a conceptual meaning. For instance, one of the four marker ids could be
|
||||
used to indicate the scale of the marker (i.e. the size of the square), so that the same diamond can be found
|
||||
in the environment with different sizes just by changing one of the four markers and the user does not need
|
||||
to manually indicate the scale of each of them. This case is included in the `detect_diamonds.cpp` file inside
|
||||
the samples folder of the module.
|
||||
|
||||
Furthermore, as its corners are chessboard corners, they can be used for accurate pose estimation.
|
||||
|
||||
The diamond functionalities are included in `<opencv2/objdetect/charuco_detector.hpp>`
|
||||
|
||||
|
||||
ChArUco Diamond Creation
|
||||
------
|
||||
|
||||
The image of a diamond marker can be easily created using the `cv::aruco::CharucoBoard::generateImage()` function.
|
||||
For instance:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/create_diamond.cpp generate_diamond
|
||||
|
||||
This will create a diamond marker image with a square size of 200 pixels and a marker size of 120 pixels.
|
||||
The marker ids are given in the second parameter as a `cv::Vec4i` object. The order of the marker ids
|
||||
in the diamond layout are the same as in a standard ChArUco board, i.e. top, left, right and bottom.
|
||||
|
||||
The image produced will be:
|
||||
|
||||

|
||||
|
||||
A full working example is included in the `create_diamond.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples `create_diamond.cpp` now take input via commandline via the `cv::CommandLineParser`. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
"_path_/mydiamond.png" -sl=200 -ml=120 -d=10 -ids=0,1,2,3
|
||||
@endcode
|
||||
|
||||
ChArUco Diamond Detection
|
||||
------
|
||||
|
||||
As in most cases, the detection of diamond markers requires a previous detection of ArUco markers.
|
||||
After detecting markers, diamond are detected using the `cv::aruco::CharucoDetector::detectDiamonds()` function:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_diamonds.cpp detect_diamonds
|
||||
|
||||
The `cv::aruco::CharucoDetector::detectDiamonds()` function receives the original image and the previous detected marker corners and ids.
|
||||
If markerCorners and markerIds are empty, the function will detect aruco markers and ids.
|
||||
The input image is necessary to perform subpixel refinement in the ChArUco corners.
|
||||
It also receives the rate between the square size and the marker sizes which is required for both, detecting the diamond
|
||||
from the relative positions of the markers and interpolating the ChArUco corners.
|
||||
|
||||
The function returns the detected diamonds in two parameters. The first parameter, `diamondCorners`, is an array containing
|
||||
all the four corners of each detected diamond. Its format is similar to the detected corners by the `cv::aruco::ArucoDetector::detectMarkers()`
|
||||
function and, for each diamond, the corners are represented in the same order than in the ArUco markers, i.e. clockwise order
|
||||
starting with the top-left corner. The second returned parameter, `diamondIds`, contains all the ids of the returned
|
||||
diamond corners in `diamondCorners`. Each id is actually an array of 4 integers that can be represented with `cv::Vec4i`.
|
||||
|
||||
The detected diamond can be visualized using the function `cv::aruco::drawDetectedDiamonds()` which simply receives the image and the diamond
|
||||
corners and ids:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_diamonds.cpp draw_diamonds
|
||||
|
||||
The result is the same that the one produced by `cv::aruco::drawDetectedMarkers()`, but printing the four ids of the diamond:
|
||||
|
||||

|
||||
|
||||
A full working example is included in the `detect_diamonds.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples `detect_diamonds.cpp` now take input via commandline via the `cv::CommandLineParser`. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
-dp=path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/detector_params.yml -sl=0.4 -ml=0.25 -refine=3
|
||||
-v=path_to_opencv/opencv/doc/tutorials/objdetect/charuco_diamond_detection/images/diamondmarkers.jpg
|
||||
-cd=path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml
|
||||
@endcode
|
||||
|
||||
ChArUco Diamond Pose Estimation
|
||||
------
|
||||
|
||||
Since a ChArUco diamond is represented by its four corners, its pose can be estimated in the same way than in a single ArUco marker,
|
||||
i.e. using the `cv::solvePnP()` function. For instance:
|
||||
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_diamonds.cpp diamond_pose_estimation
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_diamonds.cpp draw_diamond_pose_estimation
|
||||
|
||||
The function will obtain the rotation and translation vector for each of the diamond marker and store them
|
||||
in `rvecs` and `tvecs`. Note that the diamond corners are a chessboard square corners and thus, the square length
|
||||
has to be provided for pose estimation, and not the marker length. Camera calibration parameters are also required.
|
||||
|
||||
Finally, an axis can be drawn to check the estimated pose is correct using `drawFrameAxes()`:
|
||||
|
||||

|
||||
|
||||
The coordinate system of the diamond pose will be in the center of the marker with the Z axis pointing out,
|
||||
as in a simple ArUco marker pose estimation.
|
||||
|
||||
Sample video:
|
||||
|
||||
@youtube{OqKpBnglH7k}
|
||||
|
||||
Also ChArUco diamond pose can be estimated as ChArUco board:
|
||||
@snippet samples/cpp/tutorial_code/objectDetection/detect_diamonds.cpp diamond_pose_estimation_as_charuco
|
||||
|
||||
A full working example is included in the `detect_diamonds.cpp` inside the `samples/cpp/tutorial_code/objectDetection/`.
|
||||
|
||||
The samples `detect_diamonds.cpp` now take input via commandline via the `cv::CommandLineParser`. For this file the example
|
||||
parameters will look like:
|
||||
@code{.cpp}
|
||||
-dp=path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/detector_params.yml -sl=0.4 -ml=0.25 -refine=3
|
||||
-v=path_to_opencv/opencv/doc/tutorials/objdetect/charuco_diamond_detection/images/diamondmarkers.jpg
|
||||
-cd=path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_dict.yml
|
||||
-c=path_to_opencv/opencv/samples/cpp/tutorial_code/objectDetection/tutorial_camera_params.yml
|
||||
@endcode
|
||||
|
After Width: | Height: | Size: 66 KiB |
|
After Width: | Height: | Size: 3.0 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 58 KiB |
@@ -0,0 +1,9 @@
|
||||
Object Detection (objdetect module) {#tutorial_table_of_content_objdetect}
|
||||
==========================================================
|
||||
|
||||
- @subpage tutorial_aruco_detection
|
||||
- @subpage tutorial_aruco_board_detection
|
||||
- @subpage tutorial_charuco_detection
|
||||
- @subpage tutorial_charuco_diamond_detection
|
||||
- @subpage tutorial_aruco_calibration
|
||||
- @subpage tutorial_aruco_faq
|
||||