Deep Learning Model Package Inference

Description

This Step performs inference with single model packages exported from Mech-DLK and outputs the inference result.

It can perform model package inference in the following scenarios: text detection, text recognition, defect segmentation, unsupervised segmentation, instance segmentation, classification, and object detection.

This Step supports loading model packages (with the extension of .dlkpack) exported from Mech-DLK 2.6.1 or later versions.
The deep learning model packages for text detection, text recognition, defect segmentation, and unsupervised segmentation have been thoroughly tested in Mech-MSR 2.1.0, while other types of model packages have not been tested yet. For unknown issues, please contact Technical Support.

Workflow

The process of configuring this Step is shown below:

Configure the input. Connect the ports manually in the graphical programming workspace or select the input under Input in the parameter configuration panel.
Ensure that you have prepared your deep learning model package; otherwise, the Step cannot be used.
Use the Deep Learning Model Package Management Tool to import the model package.
Set parameters.
Select the output items under the Output section.
Run the Step and view output.

Obtain A Deep Learning Model Package

You can use the following methods to obtain a deep learning model package:

Export deep learning model packages from the Mech-DLK software (2.6.1 or later versions).
Obtain deep learning model packages from Mech-Mind Download Center.

System Requirements

The following system requirements need to be met when using this Step.

CPU: Support the AVX2 instruction set and meets any of the following conditions:
- IPC or PC without any discrete graphics card: Intel i5-12400 or higher.
- IPC or PC with a discrete graphics card: Intel i7-6700 or higher, with the graphics card not lower than GeForce GTX 1660.
  
  IPCs with Intel CPUs are fully tested while IPCs with AMD CPUs are not yet tested. Therefore, Intel CPUs are recommended.
GPU: It is recommended to use the GeForce GTX 1660 Super or above if the system is equipped with a discrete graphics card.

Description

Model Package Settings

Parameter Description

Parameter	Description
Model Manager Tool	This parameter is used to open the deep learning model package management tool and import the deep learning model package. The model package file is a `.dlkpack` file exported from the Mech-DLK software. Refer to Deep Learning Model Package Management Tool for the usage instructions.
Model Name	The parameter is for the selection of model packages for the Step.
Model Package Type	Once a Model Name is selected, the Model Package Type will be filled automatically.
Input Batch Size	Once a Model Name is selected, the Input Batch Size will be filled automatically.
GPU ID	This parameter is used to select the device ID of the GPU that will be used for the inference. Once you have selected the model name, you can select the GPU ID in the drop-down list of this parameter.

Model Manager Tool

This parameter is used to open the deep learning model package management tool and import the deep learning model package. The model package file is a .dlkpack file exported from the Mech-DLK software.

Refer to Deep Learning Model Package Management Tool for the usage instructions.

Model Name

The parameter is for the selection of model packages for the Step.

Model Package Type

Once a Model Name is selected, the Model Package Type will be filled automatically.

Input Batch Size

Once a Model Name is selected, the Input Batch Size will be filled automatically.

GPU ID

This parameter is used to select the device ID of the GPU that will be used for the inference. Once you have selected the model name, you can select the GPU ID in the drop-down list of this parameter.

Preprocessing

Parameter	Description
ROI File	This parameter is used to set or modify the ROI. Once the deep learning model package is selected, a default ROI will be applied. If you need to edit the ROI, click the Open the editor button. Edit the ROI in the pop-up window, and fill in the ROI name. Set the ROI: Hold down the left mouse button and drag to select an ROI on the image display region, and then click the left mouse button again to confirm. If needed, you can reset the ROI by clicking the left mouse button and dragging again. The coordinates of the selected ROI will be displayed in the “ROI Properties” section. Click the OK button to save and exit.

Parameter

Description

ROI File

This parameter is used to set or modify the ROI.

Once the deep learning model package is selected, a default ROI will be applied. If you need to edit the ROI, click the Open the editor button. Edit the ROI in the pop-up window, and fill in the ROI name.

Set the ROI: Hold down the left mouse button and drag to select an ROI on the image display region, and then click the left mouse button again to confirm. If needed, you can reset the ROI by clicking the left mouse button and dragging again. The coordinates of the selected ROI will be displayed in the “ROI Properties” section. Click the OK button to save and exit.

Before the inference, please check whether the ROI setting here is consistent with that in Mech-DLK. If not, the recognition result may be affected.

During the inference, the ROI set during model training, i.e., the default ROI, is usually used. If the position of the object changes in the camera’s field of view, please adjust the ROI.

If you would like to use the default ROI again, please delete the ROI file name below the Open the editor button.

Post-Process

Inference Configuration

This parameter is used to configure parameters related to model package inference. You can click the Open the editor button to open the inference configuration window. The parameters and their description included in this window are shown below:

Model Package Type

Description

Text Detection

If you need to filter detected texts, enable Result filter.

Refer to Configure Text Determination Rules to learn about how to Configure Logical Rules and Configure General Rule.

Text Recognition

Add Modification items to modify the text recognition results.

Character replace: Deletes digits, symbols, and letters in text recognition results.
Fixed-position replace: Replaces the character at the specified position with the set character.

Defect Segmentation

If you need to filter detected defects, enable Result filter.

Refer to Configure Defect Determination Rules to learn about how to Configure Logical Rules and Configure General Rules.

Unsupervised Segmentation

Drag the sliders to set the OK Threshold and NG Threshold.

If the defect confidence of an image is less than the threshold set for OK results, the image will be judged as OK.
If the defect confidence of an image is greater than the threshold set for NG results, the image will be judged as NG.
If the defect confidence of an image is greater than the threshold set for OK results and less than the threshold set for NG results, the image will be judged as Unknown.

The defect confidence refers to probability that there are defects in an image.

Instance Segmentation

Confidence threshold: This parameter is used to set the confidence threshold for instance segmentation. The results with a confidence value above this threshold will be kept.

Classification

Confidence threshold: This parameter is used to set the confidence threshold for classification. The results with a confidence value above this threshold will be kept.

Object Detection

Confidence threshold: This parameter is used to set the confidence threshold for object detection. The results with a confidence value above this threshold will be kept.

Dilation

Description: This parameter is used to expand the mask for the deep learning algorithm. When the size of the mask is smaller than that of the target object, there will be defects in the extracted point cloud, especially the edge point cloud. Therefore, it is recommended to enable “Dilation” to expand the mask and thus avoid point loss in the extracted point cloud. When this parameter is enabled, you need to set the Kernel Size (default value: 3 px) for the dilation operation. The larger the kernel size, the stronger the dilation effect.

Only visible for Instance Segmentation and Object Detection model package types.

Visualization Settings

Model Package Type	Description
Text Detection Text Recognition Instance Segmentation Classification Object Detection	Customize Font Size This parameter determines whether to customize the font size in the visualized outputs. Once this option is enabled, you should set the Font Size (0–10). Font Size (0–10) This parameter is used to set the font size of texts in the visualized outputs. Default value: 3.0
Defect Segmentation Unsupervised Segmentation	Draw Defect Mask on Image This parameter is used to determine whether to draw the defect mask on the image. Once this option is selected, the defect mask will be added on inputted images to mark detected defect regions.
Instance Segmentation	Draw Instance on Image This parameter is used to determine whether to display the segmented mask and bounding box on the image. Disabled by default. Instance Color Scheme This parameter is used to specify the instance color scheme for the visualized output result. Instances: Each detected object is displayed in a distinct color. Classes: Objects with the same label will be displayed in the same color. CentralPoint: Objects are displayed in their original color.
Object Detection	Show Obj Bounding Box This parameter is used to determine whether to display the mask and bounding box on the image. Disabled by default. Obj Bounding Box Mode This parameter is used to specify the way to visualize the output results. BoundingBox: Display the results with bounding boxes. CenterPoint: Displays the results with center points.

Model Package Type

Description

Text Detection

Text Recognition

Instance Segmentation

Classification

Object Detection

Customize Font Size

This parameter determines whether to customize the font size in the visualized outputs. Once this option is enabled, you should set the Font Size (0–10).
Font Size (0–10)

This parameter is used to set the font size of texts in the visualized outputs. Default value: 3.0

Defect Segmentation

Unsupervised Segmentation

Draw Defect Mask on Image

This parameter is used to determine whether to draw the defect mask on the image. Once this option is selected, the defect mask will be added on inputted images to mark detected defect regions.

Instance Segmentation

Draw Instance on Image

This parameter is used to determine whether to display the segmented mask and bounding box on the image. Disabled by default.
Instance Color Scheme

This parameter is used to specify the instance color scheme for the visualized output result.
- Instances: Each detected object is displayed in a distinct color.
- Classes: Objects with the same label will be displayed in the same color.
- CentralPoint: Objects are displayed in their original color.

Object Detection

Show Obj Bounding Box

This parameter is used to determine whether to display the mask and bounding box on the image. Disabled by default.
Obj Bounding Box Mode

This parameter is used to specify the way to visualize the output results.
- BoundingBox: Display the results with bounding boxes.
- CenterPoint: Displays the results with center points.

Output Description

Run the Step, and you can check the results in the data visualization area and the “Output Results” panel at the bottom of the data visualization area.

When you perform inference with a “Defect Segmentation” model package in this Step, the output Segmentation Boolean Result is of Boolean data type:

True: Indicates a defect or detects have been detected, and thus the judgment result is NG, with a corresponding Boolean value of 1.
False: Indicates that no defect has been detected, and thus the judgment result is OK, with a corresponding Boolean value of 0.

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.

I accept

I don't accept