Automatic Calibration in the Eye-in-Hand Setup (Four-Axis Robots)
This how-to guide introduces how to complete the automatic calibration for four-axis robots in the eye-in-hand (EIH) setup.
Preparation before Calibration
Before hand-eye calibration, you need to finish the following preparations:
-
Construct the Mech-Mind Vision System.
-
Prepare the materials required for calibration.
-
Adjust the effect of calibration board’s point cloud.
-
Complete the robot communication configuration.
Construct the Vision System
Construct the Mech-Mind Vision System by referring to the section Vision System Hardware Setup.
You need to use Mech-Eye Viewer, Mech-Vision and Mech-Viz during hand-eye calibration. Please ensure that they have been installed and are running the latest versions.
Prepare the Materials Required for Calibration
The automatic calibration in the EIH setup needs to use the calibration board.
Please prepare the calibration board according to the following requirements:
-
Ensure that the circles of the calibration board are clearly visible and without obvious scratches, and the board does not suffer from deformations.
-
In the EIH setup, please place the calibration board in the center of the working plane, where target objects are to be placed.
In addition, before calibration, move the robot to the starting point for calibration. For the automatic calibration in the EIH setup, the starting point is the camera’s image-capturing position (the robot will move the camera upwards during calibration).
Adjust the Effect of Calibration Board’s Point Cloud
-
Open Mech-Eye Viewer to adjust camera parameters.
-
Adjust the 2D parameters to make sure that the 2D image of the calibration board is clear and neither overexposed nor underexposed.
-
Adjust 3D parameters to make sure that the point clouds of the circles on the calibration board are complete and have clear contours. It is recommended to set Surface Smoothing and Outlier Removal to Normal in the Point Cloud Processing section to reduce the point cloud fluctuation range.
If the on-site ambient lights are not ideal and affect the quality of 2D images and point clouds, you can use shading or supplemental light to improve the lighting conditions.
-
Make sure that the images and point cloud of the calibration board are up to standard after completing preceding steps.
Normal Overexposed Underexposed 2D image
Point cloud
Complete the Robot Communication Configuration
If the robot uses the Standard Interface to communicate with the Vision System, please set up the Standard Interface communication with the robot. For details, please refer to the section Standard Interface Communication.
If the robot uses the Master-Control to communicate with the Vision System, please set up the Master-Control communication with the robot. For details, please refer to the section Master-control Communication.
Pre-calibration Configuration
If the robot uses the Standard Interface to communicate with the Vision System, complete Pre-calibration Configuration (Standard Interface).
If the robot uses the Master-Control to communicate with the Vision System, complete Pre-calibration Configuration (Master-control).
Pre-calibration Configuration (Standard Interface)
If the Standard Interface is used to communicate with the robot, follow these steps:
-
Open Mech-Vision, and click the Camera Calibration (Standard) button in the toolbar.The Configuration before Calibration window will be prompted.
-
In the Select how to calibrate window, select the New calibration radio button, and then click the Next button.
-
In the Select calibration task window, select Hand-eye calibration for listed robot from the drop-down list box, click the Select robot model button to select the robot model used by the project, and then click the Next button.
-
In the Select camera setup window, select the Eye in hand radio button, and then click the Next button.
-
In the Calibration method and robot control window, select Automatic and Standard Interface, and then click the Next button.
-
In the new window, set parameters Protocol and Host IP address, and then click the Start interface service button. The button will turn into Waiting for the robot to connect...
-
In the preceding figure, TCP Server and Hex (Little-endian) are only the Protocol setting example of the ABB robot. The protocol settings of other robot brands are different. The software will automatically set this parameter and you just need to keep the default setting.
-
The port number for the Host IP address parameter is 50000 by default. If the port number determined in the communication solution is not 50000, please change it as required.
-
-
On the robot teach pendant, select the automatic calibration program, teach the start point and run the program. For details on the calibration programs of different robots, please refer to the section Standard Interface Communication. After the program is started, the log message "Entering the calibration process, please start the calibration in Mech-Vision" will be printed in the Console log panel.
-
Return to Mech-Vision, confirm that "Connected" status message is displayed in the Connect the robot area, and click the Calibrate button. The Calibration (Eye in Hand) window will be prompted.
Pre-calibration Configuration (Master-Control)
If the Master-Control is used to communicate with the robot, follow these steps:
-
Open Mech-Vision, and click the Camera Calibration (Standard) button in the toolbar.The Configuration before Calibration window will be prompted.
-
In the Select how to calibrate window, select the New calibration radio button, and then click the Next button.
-
In the Select calibration task window, select Hand-eye calibration for listed robot from the drop-down list box, click the Select robot model button to select the robot model used by the project, and then click the Next button.
-
In the Select camera setup window, select the Eye in hand radio button, and then click the Next button.
-
In the Calibration method and robot control window, select Automatic and Master-control, and then click the Next button.
-
In the prompted window, set the Robot IP address parameter.
-
On the robot teach pendant, select and run the Master-Control program. For the instructions of different robots, please refer to the section Master-control Communication.
-
Return to Mech-Vision, and click the Connect the robot button in the Connect the robot area. The button will turn into Waiting for the robot to connect...
-
Wait until the “Connected” status message is displayed in the Connect the robot area, and then click the Calibrate button. The Calibration (Eye in Hand) window will be prompted.
Till now, you have completed the pre-calibration configuration and can start the calibration procedure.
Calibration Procedure
Connect to the Camera
-
In the Connect to camera step, select the camera to connect in the Camera ID list, and then click the
button or double-click the camera entry to connect to it.
-
After the camera is connected, click the Capture once or Capture live button.
-
In the Image viewer panel, ensure that the camera can capture images normally and click the Next button on the bottom bar.
Mount Calibration Board & Check Intrinsic Parameters
-
In the Mount calibration board & check intrinsic parameters step, select the calibration board model for the Standard calibration board model parameter according to its model nameplate in the 1 Select calibration board area.
-
In the 2 Check calibration board position and point cloud quality area, read carefully the requirements on the calibration board position and point cloud quality, and then click the Capture live button. The Capture live button will turn into Stop capturing and detect position.
-
Manually move the calibration board to the proper position, and ensure that the 2D image and depth map of the calibration board meet the requirements, and then click the Stop capturing and detect position button.
If the captured images do not meet the requirements, click the Open Mech-Eye Viewer button to open the Mech-Eye Viewer software, adjust the 2D and 3D exposure parameters and re-capture images. Please note that you need to change the Parameter group parameter to “calib” first.
-
In the 3 Check intrinsic parameters area, click the Check intrinsic parameters button.
-
Confirm the results of the camera intrinsic parameter check.
-
If the camera intrinsic parameter check passes, click the OK button in the prompted window, and then click the Next button on the bottom bar.
-
If the camera intrinsic parameter check fails, adjust detection parameters by drawing aid circles or manually adjusting the calibration circle detection parameters, and then click the Recheck intrinsic parameters button.
-
Draw an Aid Circle
-
To draw an aid circle, click the Draw an aid circle button.
-
In the right Image viewer panel, right-click the calibration board image and clear the Fit to window checkbox, and press the Ctrl key and drag the roller to adjust the image to a suitable size.
-
Move the mouse pointer to the cross center point of the calibration circle, press the left mouse button and make the aid circle completely include the calibration circle and then release it.
-
Click the Recheck intrinsic parameters button, and confirm that the camera intrinsic parameter check passes.
Manually Adjust Detection Parameters
To manually adjust detection parameters, click Calibration circle detection parameters (advanced), adjust parameter settings and click the Recheck intrinsic parameters button to confirm whether the camera intrinsic parameter check passes.
If the calibration circles are still not detected, please adjust the camera settings according to the on-site operating conditions.
Set Motion Path
-
In the Set motion path step, set the Height span parameter in the 1 Set height span of calibration poses area.
The Height span parameter should be set according to the recommended working distance range of the camera and the size of the robot’s working space.
-
Click the Generate path parameters button, and click the OK button in the prompted window.
This operation will slightly move the robot and capture images. The whole process may take 10 to 15 seconds. Please ensure that the motion path is safe. In case of emergency, please tap the emergency stop button on the robot teach pendant to stop the robot immediately. -
In the prompted Calibrate window, click the OK button.
-
Confirm that waypoints are reasonable and will not collide with obstacles in the right scene viewer panel.
-
Click the Generate motion path based on path parameters button and then click the Next button on the bottom bar.
Collect Images and Flange Poses
-
In the Collect images and flange poses step, select the Save images checkbox.
-
If the robot uses the Master-Control to communicate with the Vision System, select the Sync with real robot checkbox (selected by default, recommended), and click Details to confirm that the robot’s JPs and pose data are the same as those on the teach pendant. Clear the checkbox if you do not need to synchronize the data with the real robot.
-
Click the Auto move robot along path and capture images button, read the safety window carefully, and then click the OK button.
-
Wait until the robot finishes moving along the preset path and the camera finishes capturing images on all waypoints. In the right Image viewer panel, all the captured images are displayed.
-
Please stay away from the robot working area to keep safe.
-
Clicking the Stop Robot button can exit the calibration process. But the robot will not stop until it finishes the current waypoint. In case of emergency, please tap the emergency stop button on the robot teach pendant to stop the robot immediately (the robot needs to be reconnected after tapping the emergency stop button).
-
-
After all images are captured, click the OK button in the prompted window.
-
Confirm that the collected calibration data meets the data requirements, and then click the Next button on the bottom bar.
If the requirements are not met, please manually move the robot (through the teach pendant or Mech-Viz), select the Manually add more images checkbox, and click the Add an image and record flange pose button to add the calibration board image and enter the robot flange pose.
Calculate Extrinsic Parameters
-
In the Calculate extrinsic parameters step, click the Calculate extrinsic parameters button.
-
In the prompted window indicating calibration success, click the OK button.
To evaluate the robot absolute accuracy and the accuracy of calculated extrinsic parameters, you can click Robot absolute accuracy and repeatability and Extrinsic parameter accuracy in the “Calculate extrinsic parameters” window and use the error analysis tool to perform robot absolute accuracy check and analyze extrinsic parameter error in EIH setup. -
Calculate the Z-axis translation of the extrinsic parameters.
Due to the lack of degrees of freedom in rotation in four-axis robots, the captured images during calibration may lack the images for calibration board rotation. After calibration, you need to manually adjust the Z-axis value of the extrinsic parameters. -
Mount a tip on the proper position of the robot gripper, and set the tool center point (TCP).
-
Control the robot to let its end tip touch the cross center point of a calibration circle on the calibration board, and record the tool pose (X0, Y0, Z0) in the teach pendant.
-
Move the robot to the image-capturing position to capture the image of the calibration board, click the Apply parameters and preview point cloud button in the Adjust parameters area to send the point cloud of the calibration board to Mech-Viz.
-
In Mech-Viz, add a “Fixed-Point Move” Step, and enter the tool pose (X0, Y0, Z0) in the previous step as the target waypoint. Adjust the Z-axis value of the tool pose to make sure that the target waypoint has moved to the surface of the calibration board’s point cloud, and record the new Z-axis value (Z1). You can get the Z-axis value error ΔZ by subtracting Z1 from Z0.
-
-
Back to the Adjust parameters area of the Calculate extrinsic parameters steps, manually change the Z-axis translation parameter to the error ΔZ, and then click the Apply parameters and preview point cloud button.
-
In the 3D Simulation Area of Mech-Viz, confirm that the pose (X0, Y0, Z0) is on the surface of the calibration board’s point cloud.
-
Click the Save button on the bottom bar. In the prompted Save Calibration Files dialog box, click the OK button. The camera calibration result will be automatically saved in the “calibration” directory of the project.
Till now, the calibration process is completed.
Change the Extrinsic Parameter File
If the camera’s extrinsic parameter file has been used by a Mech-Vision project, you need to change the extrinsic parameter file used by the Mech-Vision project to a new one after you finish the hand-eye calibration.
-
Select the Capture Images from Camera Step.
-
In the Step Parameters panel, click
of the Camera Calibration Parameters parameter, and select the newly calibrated extrinsic parameter file.
Calibration-Related Status Codes
If the robot uses the Standard Interface to communicate with the Vision System during hand-eye calibration, the Vision System will return status codes in response data to indicate the execution status of the commands. The status codes indicate both the normal execution results and execution failures. You can perform further troubleshooting according to the status codes.