HALCON to PCL Function Comparison

Managing Point Clouds

Creating

Create an empty point cloud

HALCON:

gen_empty_object_model_3d(cloud)

PCL:

pcl::PointCloud<pcl::PointXYZ> cloud;

PCL also has built-in smart pointer types of point clouds.

pcl::PointCloud<pcl::PointXYZ>::Ptr cloud_ptr(new pcl::PointCloud<pcl::PointXYZ>());
// or
auto cloud_ptr = std::make_shared<pcl::PointCloud<pcl::PointXYZ>();

Copy a point cloud

HALCON:

copy_object_model_3d(cloud, cloud_copy)

PCL:

pcl::PointCloud<pcl::PointXYZ> cloud_copy = cloud; // stack to stack
// or
pcl::PointCloud<pcl::PointXYZ>::Ptr cloud_ptr_copy = std::make_shared<pcl::PointCloud<pcl::PointXYZ>>(*cloud_ptr); // heap to heap
// or
pcl::PointCloud<pcl::PointXYZ>::Ptr cloud_ptr = cloud.makeShared(); // stack to heap

Create a point cloud from a set of points

HALCON:

Consumes 3 arrays of the same length: xx, yy, zz, which store the X, Y, Z coordinates of those points.

gen_object_model_3d_from_points(xx, yy, zz, cloud)

PCL:

There is no dedicated function, the programmer has to add the points with a loop. Assume xx, yy, zz are vectors of the same length.

pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>());
for (size_t i = 0; i < xx.size(); i++) {
  cloud->push(pcl::PointXYZ(xx[i], yy[i], zz[i]));
}

Combine point clouds

HALCON:

union_object_model_3d([cloud_a, cloud_b], cloud_combined)

PCL:

auto cloud_combined = cloud_a + cloud_b;

Free the memory occupied by a point cloud

HALCON:

clear_object_model_3d(cloud)

PCL:

Subject to C++ object lifetime rules.

I/O

Read a point cloud from a file

HALCON:

read_object_model_3d(filename, cloud)

Specifying file format (one of 'om3', 'dxf', 'off', 'ply', 'obj', 'stl'):

cloud = read_object_model_3d(file_name, 'm', ['file_type'], [file_format])

PCL:

pcl::PointCloud<pcl::PointXYZ> cloud;
if (pcl::io::load(file_name, cloud) == -1) perror(NULL);

Specifying file format:

// Instead of pcl::io::load(filename, cloud), use one of:
pcl::io::loadIFSFile(file_name, cloud);
pcl::io::loadOBJFile(file_name, cloud);
pcl::io::loadPCDFile(file_name, cloud);
pcl::io::loadPLYFile(file_name, cloud);

Save a point cloud to a file

HALCON:

write_object_model_3d(cloud, file_type, file_name)

file_type is one of 'om3', 'dxf', 'off', 'ply', 'ply_binary', 'obj', 'stl', 'stl_binary', 'stl_ascii'.

PCL:

// One of:
pcl::io::save(file_name, cloud); // auto detect IFS, PCD or PLY
pcl::io::saveIFSFile(file_name, cloud);
pcl::io::saveOBJFile(file_name, cloud);
pcl::io::savePCDFile(file_name, cloud);
pcl::io::savePCDFileASCII(file_name, cloud);
pcl::io::savePCDFileBinary(file_name, cloud);
pcl::io::savePCDFileBinaryCompressed(file_name, cloud);
pcl::io::savePLYFile(file_name, cloud);
pcl::io::savePLYFileASCII(file_name, cloud);
pcl::io::savePLYFileBinary(file_name, cloud);

Segmenting Point Clouds

Using Point Indices

Many segmentation algorithms in PCL emit a pcl::PointIndices as the result instead of copying the point cloud.

Most algorithms in PCL accept smart pointers of such indices via the .setIndices() method to select points from the input cloud:

alg.setInputCloud(cloud.makeShared());
alg.setIndices(std::make_shared<pcl::PointIndices>(indices));

One may also want to extract the points indexed by the indices into a new point cloud, using an extractor class:

#include <pcl/filters/extract_indices.h>

pcl::PointCloud<pcl::PointXYZ> cloud_filtered;

pcl::ExtractIndices<pcl::PointXYZ> extract;
extract.setInputCloud(cloud.makeShared());
extract.setIndices(std::make_shared<pcl::PointIndices>(indices));
extract.filter(cloud_filtered);

Or using pcl::copyPointCloud(), which assumes all indices are unique:

#include <pcl/common/io.h>

pcl::copyPointCloud(cloud, indices, cloud_filtered);

The latter also works when indices is a std::vector<pcl::PointIndices>.

Select points from a point cloud based on simple thresholds

HALCON:

select_points_object_model_3d(cloud, attrib, min_value, max_value, cloud_filtered)

PCL:

#include <pcl/point_types.h>
#include <pcl/filters/passthrough.h>

pcl::PassThrough<pcl::PointXYZ> pass;
pass.setInputCloud(cloud.makeShared());
pass.setFilterFieldName(attrib); (1)
pass.setFilterLimits(min_value, max_value);

pcl::PointCloud<pcl::PointXYZ> cloud_filtered;
pass.filter(cloud_filtered);
// or
pcl::PointIndices indices_filtered;
pass.filter(indices_filtered); (2)

1	`attrib` is a `std:string`, its possible values for a particular point type can be found at pcl/point_types.h. For `pcl::PointXYZ`, they are `"x"`, `"y"`, `"z"`.
2	See Using Point Indices.

If multiple thresholds, or more flexible arrangements of thresholds are desired, pcl::ConditionalRemoval can be used instead:

#include <pcl/point_types.h>
#include <pcl/filters/conditional_removal.h>

pcl::ConditionalRemoval<pcl::PointXYZ> condrem;
condrem.setInputCloud(cloud.makeShared());
condrem.setCondition(condition);

pcl::PointCloud<pcl::PointXYZ> cloud_filtered;
condrem.filter(cloud_filtered);

where condition is one of:

auto condition = std::make_shared<pcl::ConditionAnd<pcl::PointXYZ>>();
auto condition = std::make_shared<pcl::ConditionOr<pcl::PointXYZ>>();

and can be composed of multiple comparisons such as:

condition->addComparison(std::make_shared<pcl::FieldComparison<pcl::PointXYZ>>("z", pcl::ComparisonOps::GT, 0.0)); // z > 0.0
condition->addComparison(std::make_shared<pcl::FieldComparison<pcl::PointXYZ>>("z", pcl::ComparisonOps::LE, 0.8)); // z <= 0.8

See pcl/filters/conditional_removal.h for more comparison types.

Segment a point cloud into clusters

HALCON:

connection_object_model_3d(cloud, feature, value, clusters)

feature can either be:

'distance_3d', which separates points whose euclidean distances are value units apart.
'angle', which separates points whose normal angles are value radians apart.

PCL (segmenting based on euclidean distances):

#include <pcl/segmentation/extract_clusters.h>

std::vector<pcl::PointIndices> cluster_indices; // result vector (2)

pcl::EuclideanClusterExtraction<pcl::PointXYZ> ec;
ec.setInputCloud(cloud.makeShared());
ec.setClusterTolerance(distance);
ec.setMinClusterSize(min_pts); (1) (3)
ec.setMaxClusterSize(max_pts); (1) (3)
ec.extract(cluster_indices);
// or use an existing function:
auto tree = std::make_shared<pcl::search::KdTree<pcl::PointXYZ>>();
tree->setInputCloud(cloud.makeShared());
pcl::extractEuclideanClusters(
  cloud,
  indices.indices, // input indices (1) (2)
  tree,
  distance,
  cluster_indices,
  min_pts, (1) (3)
  max_pts (1) (3)
);

1	Optional.
2	For how to use a `pcl::PointIndices`, see Using Point Indices.
3	`min_pts` and `max_pts` define the minimum and maximum points a cluster can have.

PCL (segmenting based on euclidean distances as well as angles):

#include <pcl/segmentation/extract_clusters.h>

std::vector<pcl::PointIndices> cluster_indices; // result vector

auto tree = std::make_shared<pcl::search::KdTree<pcl::PointXYZ>>();
tree->setInputCloud(cloud.makeShared());
pcl::extractEuclideanClusters(
  cloud,
  cloud_normals, (2)
  indices.indices, // input indices (1)
  tree,
  distance,
  cluster_indices,
  angle, // in radians
  min_pts, (1)
  max_pts (1)
);

1	Optional.
2	A `pcl::PointCloud<pcl::Normal>` containing normals of `cloud`, obtained with Calculate the 3D surface normals of a point cloud.

Fit a point cloud into primitive shapes

HALCON:

fit_primitives_object_model_3d(cloud, ['primitive_type'], [primitive_type], cloud_fitted)

get_object_model_3d_params(cloud_fitted, ['primitive_parameter'], [primitive_parameter])

primitive_type can be 'cylinder', 'sphere', 'plane'. There is no corresponding functionality in PCL to 'all' or a list of primitive types.

PCL:

PCL uses Sample Consensus algorithms to fit primitives, instead of the least squares algorithm used by HALCON.

#include <pcl/sample_consensus/ransac.h> (2)

auto cloud_ptr = cloud.makeShared();

auto model = std::make_shared<pcl::SampleConsensusModel...<pcl::PointXYZ>>(cloud_ptr); (1)

pcl::RandomSampleConsensus<pcl::PointXYZ> ransac(model); (2)
ransac.setDistanceThreshold(0.01);
ransac.computeModel();

pcl::PointIndices inliers;
ransac.getInliers(inliers.indices); (3)

Eigen::VectorXf model_coefficients;
ransac.getModelCoefficients(model_coefficients);

1	Ellipsis should be replaced with one of the primitive types: `Plane`, `Line`, `Circle2D`, `Circle3D`, `Sphere`, `Cylinder`, `Cone` and more. See PCL documentation for a list, and what their resultant `model_coefficients` represent.
2	Instead of `RandomSampleConsensus`, alternatives exist per PCL documentation.
3	See Using Point Indices for how to use this output data.

Transforming Point Clouds

Apply a 3D transformation to a point cloud

HALCON:

affine_trans_object_model_3d(cloud, matrix, cloud_transformed) * Affine transformation described by matrix
* or
rigid_trans_object_model_3d(cloud, pose, cloud_transformed) * Rigid transformation described by pose
* or
projective_trans_object_model_3d(cloud, matrix, cloud_transformed) * Projective transformation described by matrix

PCL:

pcl::PointCloud<pcl::PointXYZ> cloud_transformed;

// and then one of:
pcl::transformPointCloud(cloud, cloud_transformed, matrix); (1)
pcl::transformPointCloud(cloud, indices, cloud_transformed, matrix); (1) (2)
pcl::transformPointCloud(cloud, cloud_transformed, offset, rotation); (3)

1	`matrix` is of type `Eigen::Transform` or `Eigen::Matrix`, see Transformations for details.
2	`indices` is of type `pcl::PointIndices`, see Using Point Indices for detail.
3	This performs a rigid transformation described by a `Eigen::Vector3f` `offset` and a `Eigen::Quaternion` `rotation`.

Smooth points in a point cloud

HALCON:

smooth_object_model_3d(cloud, 'mls', ['mls_kNN', 'mls_order'], [mls_kNN, mls_order], cloud_smoothed)

PCL:

#include <pcl/point_types.h>
#include <pcl/search/kdtree.h>
#include <pcl/surface/mls.h>

pcl::PointCloud<pcl::PointNormal> cloud_smoothed;

pcl::MovingLeastSquares<pcl::PointXYZ, pcl::PointNormal> mls;
mls.setInputCloud(cloud.makeShared());
mls.setComputeNormals(true);
mls.setPolynomialOrder(2); // corresponds to mls_order
mls.setSearchMethod(std::make_shared<pcl::search::KdTree<pcl::PointXYZ>>());
mls.setSearchRadius(0.03); // radius used for determining mls_kNN

mls.process(cloud_smoothed);

Calculate the 3D surface normals of a point cloud

HALCON:

surface_noramls_object_model_3d(cloud, 'mls', ['mls_kNN', 'mls_order'], [mls_kNN, mls_order], cloud_with_normals)

PCL (with MLS smoothing):

See Smooth points in a point cloud.

PCL (without smoothing):

#include <pcl/point_types.h>
#include <pcl/features/normal_3d.h>

pcl::PointCloud<pcl::Normal> normals;

pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> ne;
ne.setInputCloud(cloud.makeShared());
ne.setSearchMethod(std::make_shared<pcl::search::KdTree<pcl::PointXYZ>>());
// one of:
ne.setKSearch(20); // 20 nearest points
ne.setRadiusSearch(0.03); // points inside sphere of radius 0.03 units

ne.compute(normals);

pcl::PointCloud<pcl::PointNormal> cloud_with_normals;
pcl::concatenateFields(cloud, normals, cloud_with_normals);

Calculate the convex hull of a point cloud

HALCON:

convex_hull_object_model_3d(cloud, cloud_hull)

PCL:

#include <pcl/point_types.h>
#include <pcl/surface/convex_hull.h>

pcl::PointCloud<pcl::PointXYZ> cloud_hull;

pcl::ConvexHull<pcl::PointXYZ> chull;
chull.setInputCloud(cloud.makeShared());
chull.reconstruct(cloud_hull);

Obtain a triangulated mesh of a point cloud (greedy)

HALCON:

triangulate_object_model_3d(cloud, 'greedy', ['greedy_kNN', 'greedy_radius_type', 'greedy_radius_value', 'greedy_neigh_orient_tol', 'greedy_neigh_orient_consistent'], [greedy_kNN, 'fixed', greedy_radius_value, greedy_neigh_orient_tol, greedy_neigh_orient_consistent], triangulated_mesh, information)

PCL:

#include <pcl/point_types.h>
#include <pcl/kdtree/kdtree_flann.h>
#include <pcl/surface/gp3.h>

// First, obtain cloud_with_normals as described in (1)

pcl::PolygonMesh triangulated_mesh;

pcl::GreedyProjectionTriangulation<pcl::PointNormal> gp3;
gp3.setSearchRadius(greedy_radius_value);
gp3.setMu(2.5);
gp3.setMaximumNearestNeighbors(greedy_kNN);
gp3.setMaximumSurfaceAngle(greedy_neigh_orient_tol / 180 * M_PI);
gp3.setMinimumAngle(M_PI / 18); // 10 degrees
gp3.setMaximumAngle(2 * M_PI / 3); // 120 degrees
gp3.setNormalConsistency(greedy_neigh_orient_consistent);

gp3.setInputCloud(cloud_with_normals.makeShared());
gp3.setSearchMethod(std::make_shared<pcl::search::KdTree<pcl::PointNormal>>());
gp3.reconstruct(triangulated_mesh);

1	Calculate the 3D surface normals of a point cloud

Obtain a triangulated mesh of a point cloud (Poisson)

HALCON:

triangulate_object_model_3d(cloud, 'implicit', ['implicit_octree_depth', 'implicit_min_num_samples'], [implicit_octree_depth, implicit_min_num_samples], triangulated_mesh, information)

PCL:

#include <pcl/point_types.h>
#include <pcl/kdtree/kdtree_flann.h>
#include <pcl/surface/poisson.h>

pcl::PolygonMesh triangulated_mesh;

pcl::Poisson<pcl::PointNormal> poisson;
poisson.setDepth(implicit_octree_depth);
poisson.setSamplesPerNode(implicit_min_num_samples);
poisson.setOutputPolygons(true);

poisson.setInputCloud(cloud_with_normals.makeShared());
poisson.setSearchMethod(std::make_shared<pcl::search::KdTree<pcl::PointNormal>>());
poisson.performReconstruction(triangulated_mesh);

Downsample a point cloud

HALCON:

sample_object_model_3d(cloud, 'fast', distance, [], [], cloud_sampled)

PCL:

While not the same algorithm, it does perform sampling.

#include <pcl/point_types.h>
#include <pcl/filters/voxel_grid.h>

pcl::VoxelGrid<pcl::PointXYZ> sor;
sor.setInputCloud(cloud.makeShared());
sor.setLeafSize(distance, distance, distance);

pcl::PointCloud<pcl::PointXYZ> cloud_sampled;
sor.filter(cloud_sampled);

Transformations

HALCON makes the distinction between pixel-centered and edge-centered Cartesian coordinate systems, functions may expect and output one coordinate system or the other (see documentation). Failure to conform may result in off-by-0.5-pixel errors. PCL does not make this distinction, it’s up to the programmer to ensure consistent use of coordinate systems.

PCL uses the Eigen library to handle matrices. Eigen is not part of PCL. For the completeness of this document, its use is still provided under PCL code examples to provide parity with HALCON.

In PCL, affine transformations are described by data types Eigen::Affine2f and Eigen::Affine3f, while projective transformations are described by data types Eigen::Projective2f and Eigen::Projective3f. They are all instantiations of the template class Eigen::Transform. Many of the PCL code examples in this chapter use the affine types, they are also applicable to projective transformations.

2D Homogeneous Transformation Matrices

Create a 2D identity transformation matrix

\[\text{matrix} = I_3\]

HALCON:

hom_mat2d_identity(matrix)

PCL:

Eigen::Affine2f matrix = Eigen::Affine2f::Identity();
// or, for existing matrix:
matrix.setIdentity();

Invert a 2D transformation matrix

\[\text{matrix_inverted} = \text{matrix}^{-1}\]

HALCON:

hom_mat2d_invert(matrix, matrix_inverted)

PCL:

Eigen::Affine2f matrix_inverted = matrix.inverse();

Transpose a 2D transformation matrix

\[\text{matrix_transposed} = \text{matrix}^T\]

HALCON:

hom_mat2d_transpose(matrix, matrix_transposed)

PCL:

Eigen::Projective2f matrix_transposed(matrix.matrix().transpose());

Compute the determinant of a 2D transformation matrix

\[\text{determinant} = | \text{matrix} |\]

HALCON:

hom_mat2d_determinant(matrix, determinant)

PCL:

float determinant = matrix.matrix().determinant();

Multiply 2 2D transformation matrices

\[\text{product} = \text{matrix_a} \cdot \text{matrix_b}\]

HALCON:

hom_mat2d_compose(matrix_a, matrix_b, product)

PCL:

Eigen::Affine2f product = matrix_a * matrix_b;

Add rotation to a 2D transformation matrix

\[\text{matrix_rotated} = \text{matrix} \cdot \begin{bmatrix} \cos(\phi) & -\sin(\phi) & 0 \\ \sin(\phi) & \cos(\phi) & 0 \\ 0 & 0 & 1 \end{bmatrix}\]

phi is the angle of rotation in radians.

HALCON:

hom_mat2d_rotate_local(matrix, phi, matrix_rotated)

PCL:

Eigen::Affine2f matrix_rotated = matrix;
matrix_rotated.rotate(phi);

Add scaling to a 2D transformation matrix

\[\text{matrix_scaled} = \text{matrix} \cdot \begin{bmatrix} s_x & 0 & 0 \\ 0 & s_y & 0 \\ 0 & 0 & 1 \end{bmatrix}\]

sx and sy are scaling factors on the X and Y dimensions.

HALCON:

hom_mat2d_scale_local(matrix, sx, sy, matrix_scaled)

PCL:

Eigen::Affine2f matrix_scaled = matrix;
matrix_scaled.scale(Eigen::Vector2f(sx, sy));
// or when sx == sy:
matrix_scaled.scale(sx);

Add translation to a 2D transformation matrix

\[\text{matrix_translated} = \text{matrix} \cdot \begin{bmatrix} 1 & 0 & t_x \\ 0 & 1 & t_y \\ 0 & 0 & 1 \end{bmatrix}\]

HALCON:

hom_mat2d_translate_local(matrix, tx, ty, matrix_translated)

PCL:

Eigen::Affine2f matrix_translated = matrix;
matrix_translated.translate(Eigen::Vector2f(tx, ty));

Add reflection to a 2D transformation matrix

\[\begin{gather} \text{reflection} = \begin{bmatrix} M & \begin{matrix}0 \\ 0\end{matrix} \\ \begin{matrix}0 & 0\end{matrix} & 1 \end{bmatrix} \cdot \text{matrix} \\ M = I - \frac{2}{v^T v} v v^T \qquad v = \begin{bmatrix} -p_y \\ p_x \end{bmatrix} \end{gather}\]

HALCON:

hom_mat2d_reflect_local(matrix, px, py, reflection)

PCL:

Eigen does not have a built-in function for applying reflections. One has to perform reflection on their own.

Eigen::Vector2f v(-py, px);
Eigen::Matrix2f m = Eigen::Matrix2f::Identity() - 2.0f / (v.transpose() * v) * (v * v.transpose());
Eigen::Affine2f reflection = Eigen::Affine2f(m) * matrix;

Add slant to a 2D transformation matrix

\[\begin{aligned} \text{matrix_slanted} &= \text{matrix} \cdot \begin{bmatrix} \cos(\theta) & 0 & 0 \\ \sin(\theta) & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} && \text{axis = X} \\ \text{matrix_slanted} &= \text{matrix} \cdot \begin{bmatrix} 1 & -\sin(\theta) & 0 \\ 0 & \cos(\theta) & 0 \\ 0 & 0 & 1 \end{bmatrix} && \text{axis = Y} \end{aligned}\]

theta is the angle of slant in radians.

HALCON:

axis is 'x' or 'y'.

hom_mat2d_slant_local(matrix, theta, axis, matrix_slanted)

PCL:

Eigen does not have a built-in function for applying slants. One has to perform slanting on their own.

// When axis = X:
Eigen::Affine2f matrix_slanted = matrix * Eigen::Affine2f(Eigen::Matrix2f{{std::cos(theta), 0}, {std::sin(theta), 1}});
// Or when axis = Y:
Eigen::Affine2f matrix_slanted = matrix * Eigen::Affine2f(Eigen::Matrix2f{{1, -std::sin(theta)}, {0, std::cos(theta)}});

3D Homogeneous Transformation Matrices

Create a 3D identity transformation matrix

\[\text{matrix} = I_4\]

HALCON:

hom_mat3d_identity(matrix)

PCL:

Eigen::Affine3f matrix = Eigen::Affine3f::Identity();
// or, for existing matrix:
matrix.setIdentity();

Invert a 3D transformation matrix

\[\text{matrix_inverted} = \text{matrix}^{-1}\]

HALCON:

hom_mat3d_invert(matrix, matrix_inverted)

PCL:

Eigen::Affine3f matrix_inverted = matrix.inverse();

Transpose a 3D transformation matrix

\[\text{matrix_transposed} = \text{matrix}^T\]

HALCON:

hom_mat3d_transpose(matrix, matrix_transposed)

PCL:

Eigen::Projective3f matrix_transposed(matrix.matrix().transpose());

Compute the determinant of a 3D transformation matrix

\[\text{determinant} = | \text{matrix} |\]

HALCON:

hom_mat3d_determinant(matrix, determinant)

PCL:

float determinant = matrix.matrix().determinant();

Multiply 2 3D transformation matrices

\[\text{product} = \text{matrix_a} \cdot \text{matrix_b}\]

HALCON:

hom_mat3d_compose(matrix_a, matrix_b, product)

PCL:

Eigen::Affine3f product = matrix_a * matrix_b;

Add rotation to a 3D transformation matrix

\[\text{matrix_rotated} = \text{matrix} \cdot \begin{bmatrix} R & \begin{matrix}0\\0\\0\end{matrix} \\ \begin{matrix}0 & 0 & 0\end{matrix} & 1 \end{bmatrix}\]

\[\begin{align} R_x &= \begin{bmatrix} 1 & 0 & 0 \\ 0 & \cos(\phi) & -\sin(\phi) \\ 0 & \sin(\phi) & \cos(\phi) \end{bmatrix} && \text{axis = X} \\ R_y &= \begin{bmatrix} \cos(\phi) & 0 & \sin(\phi) \\ 0 & 1 & 0 \\ -\sin(\phi) & 0 & \cos(\phi) \end{bmatrix} && \text{axis = Y} \\ R_z &= \begin{bmatrix} \cos(\phi) & -\sin(\phi) & 0 \\ \sin(\phi) & \cos(\phi) & 0 \\ 0 & 0 & 1 \end{bmatrix} && \text{axis = Z} \\ R_u &= \cos(\phi) I + \sin(\phi) [u]_\times + (1 - \cos(\phi))(u u^T) && \text{axis is a unit vector $u$} \end{align}\]

\[u = \begin{pmatrix} u_x \\ u_y \\ u_z \end{pmatrix} \qquad [u]_\times = \begin{bmatrix} 0 & -u_z & u_y \\ u_z & 0 & -u_x \\ -u_y & u_x & 0 \end{bmatrix}\]

phi is the angle of rotation in radians.

HALCON:

axis is one of 'x', 'y', 'z', or a unit vector [ux, uy, uz] as a tuple.

hom_mat3d_rotate_local(matrix, phi, axis, matrix_rotated)

PCL:

axis is one of:

Eigen::Vector3f::UnitX()
Eigen::Vector3f::UnitY()
Eigen::Vector3f::UnitZ()
Eigen::Vector3f(ux, uy, uz)

axis must be normalized, else rotation is invalid.

Eigen::Affine3f matrix_rotated = matrix;
matrix_rotated.rotate(Eigen::AngleAxisf(phi, axis));

Add scaling to a 3D transformation matrix

\[\text{matrix_scaled} = \text{matrix} \cdot \begin{bmatrix} s_x & 0 & 0 & 0 \\ 0 & s_y & 0 & 0 \\ 0 & 0 & s_z & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}\]

sx, sy, sz are scaling factors on the X, Y, Z dimensions.

HALCON:

hom_mat3d_scale_local(matrix, sx, sy, sz, matrix_scaled)

PCL:

Eigen::Affine3f matrix_scaled = matrix;
matrix_scaled.scale(Eigen::Vector3f(sx, sy, sz));
// or when sx == sy == sz:
matrix_scaled.scale(sx);

Add translation to a 3D transformation matrix

\[\text{matrix_translated} = \text{matrix} \cdot \begin{bmatrix} 1 & 0 & 0 & t_x \\ 0 & 1 & 0 & t_y \\ 0 & 0 & 1 & t_z \\ 0 & 0 & 0 & 1 \end{bmatrix}\]

HALCON:

hom_mat3d_translate_local(matrix, tx, ty, matrix_translated)

PCL:

Eigen::Affine3f matrix_translated = matrix;
matrix_translated.translate(Eigen::Vector3f(tx, ty, tz));

Poses

PCL does not make the distinction between affine and rigid transformations. This also means there is no separate data type that represent rigid transformations, they are also represented by affine transformation matrices.

Create a 3D transformation matrix from rotation angles and translations

tx, ty, tz are translation along the X, Y, Z axes; rx, ry, rz are rotation angles around the X, Y, Z axes in radians. order_of_rotation dictates the interpretation of the rotation angles.

\[\begin{aligned} \text{matrix} &= \begin{bmatrix} 1 & 0 & 0 & t_x \\ 0 & 1 & 0 & t_y \\ 0 & 0 & 1 & t_z \\ 0 & 0 & 0 & 1 \end{bmatrix} \cdot R_x \cdot R_y \cdot R_z && \text{order_of_rotation = 'gba'} \\ \text{matrix} &= \begin{bmatrix} 1 & 0 & 0 & t_x \\ 0 & 1 & 0 & t_y \\ 0 & 0 & 1 & t_z \\ 0 & 0 & 0 & 1 \end{bmatrix} \cdot R_z \cdot R_y \cdot R_x && \text{order_of_rotation = 'abg'} \end{aligned}\]

\[\begin{aligned} R_x &= \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & \cos(r_x) & -\sin(r_x) & 0 \\ 0 & \sin(r_x) & \cos(r_x) & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} \\ R_y &= \begin{bmatrix} \cos(r_y) & 0 & \sin(r_y) & 0 \\ 0 & 1 & 0 & 0 \\ -\sin(r_y) & 0 & \cos(r_y) & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} \\ R_z &= \begin{bmatrix} \cos(r_z) & -\sin(r_z) & 0 & 0 \\ \sin(r_z) & \cos(r_z) & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} \end{aligned}\]

HALCON:

create_pose(tx, ty, tz, deg(rx), deg(ry), deg(rz), 'Rp+T', order_of_rotation, 'point', pose)
pose_to_hom_mat3d(pose, matrix)

PCL:

Eigen::Affine3f matrix = Eigen::Affine3f::Identity();
matrix.rotate(Eigen::AngleAxisf(rz, Eigen::Vector3f::UnitZ())); (1)
matrix.rotate(Eigen::AngleAxisf(ry, Eigen::Vector3f::UnitY())); (1)
matrix.rotate(Eigen::AngleAxisf(rx, Eigen::Vector3f::UnitX())); (1)
matrix.translate(Eigen::Vector3f(tx, ty, tz));

1	This example corresponds to HALCON’s behavior when `order_of_rotation` is `'abg'`. To recreate the behavior when `order_of_rotation` is `'gba'`, the order of the rotations should be reversed.

Visualization

Creating Visualisations

Displaying the raw PCD file data

Using PCL-tools on ubuntu you are able to easily visualize any PCD file from the terminal using the below function. This is a simple way to check PCD data files before you implement them into long visualization processes.

$ pcl_viewer PCD_file.pcd

Installing PCL-tools is as simple as the following terminal command line in ubuntu.

$ sudo apt install pcl-tools

Displaying Point Cloud Data From IDE

HALCON

disp_object_model_3d( : : WindowHandle, ObjectModel3D, CamParam, Pose, GenParamName, GenParamValue :)

In PCL the visualisations are created mainly thorugh the format of a dedicated visualisation function. These visualisations can be manipulated and changed to suit the users needs. The visualisation function below creates a visualizer window called "New Cloud". Following this the cloud data, 'img', with a name for viewport identification, most examples use '0' for the veiwport number/identification.

PCL:

#include <pcl/visualization/pcl_visualizer.h>
pcl::visualization::PCLVisualizer::Ptr viewer(new pcl::visualization::visualizer ("New Cloud"));
viewer->addPointCloud(new PointCloud(img)), 0);

The '0' in the above line states the viewport identification. Can be any int for the viewport.

Cloud Point Properties (Colour and Size)

For colour and size we can adjust these through the set viewer model similar to the addPointCloud model. However, these require some inputs. - Size Can be a number an int from 1.0 and above, the larger the number the larger the point size. - r, g, b, are the colours Red, Green and Blue which can have a lightness or darkness level between 0.0 (dark) and 1.0 (light) inclusive.

PCL:

viewer->setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_POINT_SIZE, size, cloud_name)

viewer->setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_COLOR, r, g, b, cloud_name)

There are also some basic colourmaps premade for use by PCL, these can be found at the link provided under rendering properties.

Line Width

Line Width uses the same paramater styles as the Point Size method above, integers from 1.0 and above. PCL:

viewer->setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_LINE_WIDTH, size, cloud_name)

Background Properties

PCL:

viewer->setBackgroundColor(r, g, b, cloud_name)

Text

Note: For all of the PCL code below the examples are all coded within a visualizer format (displayed below for reference)

#include <pcl/visualization/pcl_visualizer.h>

pcl::visualization::PCLVisualizer::Ptr simpleVis (pcl::PointCloud<pcl::PointXYZ>::ConstPtr cloud){
	pcl::visualization::PCLVisualizer::Ptr viewer (new pcl::PCLVisualizer ("3D viewer"));
	viewer->addPointCloud<pcl::PointXYZ> (cloud, "sample data");
	// Viewer extras (i.e. point size, colour of background and data, text)
}

Adding 2D Text

HALCON:

disp_text( : : WindowHandle, String, CoordSystem, Row, Column, Color, GenParamName, GenParamValue : )

PCL:

viewer->addText ("String", X_position, Y_position, id, Viewport);

String is the text you would like to display. The X_position and Y_position are where you would like the text to be displayed, if X and Y = 10, the text will be shown in the bottom left corner. id is the text object, equal to the text parameter. This text is locked onto the WINDOWs X and Y coordinates. It will not travel with the 3D point cloud data.

HALCON:

new_line( : : WindowHandle : )

PCL:

viewer->addText ("First line \n Second line", X_position, Y_position, id, Viewport);

We can use the addText function as before but simply adding a newline marker creates a secondary line in the window. Dont forgt to change your X and Y positions for the text appropriately.

HALCON:

qery_font( : : WindowHandle : Font)

Queries the fonts available for the text output window. They can be set with the operator set_font

Example of Simple Adjustments

pcl::visualization::PCLVisualizer::Ptr simpleVis (pcl::PointCloud<pcl::PointXYZ>::ConstPtr cloud)
{
  pcl::visualization::PCLVisualizer::Ptr viewer (new pcl::visualization::PCLVisualizer ("3D Viewer"));

  viewer->setBackgroundColor (0, 0, 0);
  viewer->addPointCloud<pcl::PointXYZ> (cloud, "sample cloud");

  viewer->setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 2, "sample cloud");
  viewer->setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_COLOR, 1, 0.5, 0.5, "sample cloud");

  viewer->addCoordinateSystem (1.0);
  viewer->initCameraParameters ();

  viewer->addText ("First Line \n Second Line", 10, 20, "v1 text", 0);

  return (viewer);
}

Adding 3D Text (Point Specified/Locked)

Adding 3D text is best for addressing specific cloud arrays. This can be implemented with the below functions. The PCL:

#include <pcl/visualization/pcl_visualizer.h>

 pcl::visualization::PCLVisualizer::addText3D (
   const std::string &text, // This is the char[] string that will be displayed to the position specified
   const PointT& position,  // This is the position pointer that can be set up in one of the following ways (below)
   double orientation[3],
   double textScale = 1.0,
   double r = 1.0,
   double g = 1.0,
   double b = 1.0,
   const std::string &id = 0,
   int viewport)

To set up a pointer for a set value at say the origin of your cloud point you can use this technique

pcl::PointXYZ textPosition; // Creating a pointer for PCL
textPosition.x = 0;         // Sets the x to origin
textPosition.y = 0;         // Sets the y to origin
textPosition.z = 0;         // Sets the z to origin

viewer.addText3D<pcl::PointXYZ> ("Text to show", textPosition, 0.2, 1.0, 0.0, 0.0, "3DText", 0)

This 3D text addition will always face the viewer but will be dreagged around the visualisation with respect to the coordinate set.

This is strictly for operations using the vpcl::visualization::PCLVisualizer, pcl::PointXYZ version on point cloud library. To adjust to another point type you will need to change the typeface and name.

Mouse

Get Mouse Pointer coordinates (Window) 'MouseEvent'

For the HALCON code below the function returns the location of the mouse pointer relative to the windows coordinates (row and column) of the pixels covered by the window. The button pressed on the mouse is calculated using a binary code system that once added up reveals all the numbers that wer pressed. The final value is returned to the 'Button' variable. This is also true for the 'sub_pix' variation but that returns the values with sub pixel accuracy.

HALCON:

get_mbutton( : : WindowHandle : Row, Column, Button)
get_mbutton_sub_pix( : : WindowHandle : Row, Column, Button)

In PCL there is a class for this called 'MouseEvent'. This class captures the mouse location on the window, the button pressed and also captures certain keys on teh keyboard to assess for 'special' commands. MouseEvent class saves the following inputs: - type Event type (Pressed or Released) - button The Button that causes the event - x X position of mouse pointer at that time where event got fired - y Y position of mouse pointer at that time where event got fired - alt Whether the ALT key was pressed at that time where event got fired - ctrl Whether the CTRL key was pressed at that time where event got fired - shift Whether the Shift key was pressed at that time where event got fired - selection_mode Whether we are in selection mode

To aquire this information within the window you will need to implement the following code in the PCLVisualizer containing your cloud point data.

PCL:

#include <pcl/visualization/keyboard_event.h>
// Example Visualisation function
pcl::visualization::PCLVisualizer::Ptr viewer (new pcl::visualization::PCLVisualizer ("3D Viewer"));
viewer->registerMouseCallback (mouseEventOccured, (void*)&viewer);

void mouseEventOccured (const pcl::visualization::MouseEvent &event, void* viewer_void)
{
	// In this section you are able to recall the class 'MouseEvent' using the getters below.
	std::cout << event.getType() << std::endl; 					//Displays the Event Type to terminal
	std::cout << event.getButton() << std::endl;				//Displays the Button that caused event to terminal
	std::cout << event.getX() << std::endl;						//Displays the X coordinate to terminal
	std::cout << event.getY() << std::endl;						//Displays the Y coordinate to terminal
	std::cout << event.getKeyboardModifiers() << std::endl;		//Displays the Keyboard keys pressed to terminal
	std::cout << event.getSelectionMode() << std::endl;			//Displays the selection mode to terminal
}

Selecting Areas or Multiple Points

For selecting areas and points in a Point Cloud viewer there must be a selected mouse callback function ot register the input from the user. The mouse naturally drags and orientates the image when the M1 button is clicked. To register a mouse callback with a specific function you can use the above function with specified additions. To search for a mouse input option such as the 'alt' key. Following this you must decide what to do with the data once selected. This can be to copy the selected points or to adjust the colour using the coding options detailed in Graphics Visualisations of this document.

PCL:

boost::signals2::connection pcl::visualization::PCLVisualizer::registerPointPickingCallback (std::function< void(const pcl::visualization::PointPickingEvent &)> cb	) ;
// Where cb is the CallBack function

PCL:

// ALSO
// Under the class PointPickingEvent
#include <pcl/pcl_macros.h>
#include <pcl/types.h> // for pcl::indices
viewer->registerPointPickingCallback (performPointPick,(void*)&viewer)

void performPointPick()
{
	getPointIndex(idx_);
	[x,y,z] = getPoint(float &x, float &y, float &z) const;
	std::cout << "Point Picked Location: " + idx_ << std::endl;
	// displays the point index for point data.
	std::cout << getPoint(float &x, float &y, float &z) const;
	// displays the point coordinates for point.
}

Selecting single Points

Similar rules apply as the Selecting areas section. PCL:

boost::signals2::connection pcl::visualization::PCLVisualizer::registerPointPickingCallback (std::function< void(const pcl::visualization::PointPickingEvent &)> cb)
// Where cb is the CallBack function

// ALSO
// Under the class PointPickingEvent
#include <pcl/pcl_macros.h>
#include <pcl/types.h> // for pcl::indices
viewer->registerPointPickingCallback (performAreaPick,(void*)&viewer)

void performAreaPick()

Double Click

In HALCON there is a function that allows for a double click to be registered on the window. This capability is not yet seen in the visualizer window for PCL. However they have allowed for certain keyboard inputs to be registered by the mouse event as well. These options can be seen above.

HALCON:

send_mouse_double_click_event( : : WindowHandle, Row, Column, Button : Processed)

Drag Event

HALCON:

send_mouse_drag_event( : : WindowHandle, Row, Column, Button : Processed)

PCL does not have an available drag event for personal adjustment, the current drag movement on the window allows for the point cloud data to be rotated and pivoted throughout the viewer.

This mouse button event sends the position of the mouse in the window to the 'WindowHandle' and this signals the 'Button' has been released. The 'Button' is a previously coded mouse event. See Mouse Event for more details.

Send Mouse Up/Down Event

HALCON:

send_mouse_up_event( : : WindowHandle, Row, Column, Button : Processed)
send_mouse_down_event( : : WindowHandle, Row, Column, Button : Processed)

The send mouse up and down event in halcon is similiar to the PCL get.point caller for PCL. It takes the pointers location from the window and states the current position in the window or function specified.

PCL:

	std::cout << event.getX() << std::endl;						//Displays the X coordinate to terminal
	std::cout << event.getY() << std::endl;						//Displays the Y coordinate to terminal