SHREC 2022: Fitting and recognition of simple geometric primitives on point clouds

This paper presents the methods that have participated in the SHREC 2022 track on the fitting and recognition of simple geometric primitives on point clouds. As simple primitives we mean the classical surface primitives derived from constructive solid geometry, i.e., planes, spheres, cylinders, cones and tori. The aim of the track is to evaluate the quality of automatic algorithms for fitting and recognising geometric primitives on point clouds. Specifically, the goal is to identify, for each point cloud, its primitive type and some geometric descriptors. For this purpose, we created a synthetic dataset, divided into a training set and a test set, containing segments perturbed with different kinds of point cloud artifacts. Among the six participants to this track, two are based on direct methods, while four are either fully based on deep learning or combine direct and neural approaches. The performance of the methods is evaluated using various classification and approximation measures.


Introduction
Fitting and recognition are of paramount importance in multiple application domains, such as reverse engineering and data compression. In most real-world scenarios, acquisition methods tend to produce data affected by noise and potentially by other imperfections (e.g., non-uniform or low sampling density, misalignment and missing data). The literature on methods for fitting and recognition of simple primitives is quite consolidated [1] and recently we have seen the growth of data-driven methods, see for instance [2,3,4,5,6] resulting in a growing demand for appropriate datasets for training and their evaluation. It is therefore worth analyzing and comparing the outcome of simple primitive fitting methods on a controlled dataset with different kinds of perturbations and primitive variations and in terms of different evaluation measures.

Related datasets
Given the strong interest in the problem of fitting geometric primitives, several datasets are becoming available but, generally, they are not specifically designed for simple geometric primitives, possibly affected by perturbations, missing points and local variations. A first collection of one million CAD models targeted to data-driven problems was provided by [7], in which each object is associated to a ground truth for different quantities, as for example the indices of the patch segmentation and the analytic representation of surfaces and curves. A lot of surface types are considered and the models are represented as triangle meshes. Then, it is worth mentioning [8], a benchmark of point clouds representing CAD objects aimed at evaluating methods for detecting simple geometric primitives. Despite having some features similar to our purposes, such as sub-sampled clouds and missing parts, it is not big enough and was designed to test model segmentation methods. A dataset of mechanical components is proposed by [2], which generated the point samples from some models of the American National Standards Institute (ANSI) provided by TraceParts. In this collection, four types of primitives are considered (plane, sphere, cylinder, cone). Another synthetic dataset is presented in [9], but it proposes only cylinders, spheres and cubes as surface types. These existing datasets are not suitable for our purposes since our intent is to analyze the different methods for fitting geometric primitives on point clouds that present different types of specific artifacts. Indeed, in our dataset all point clouds have been artificially generated from exact surfaces and then optionally perturbed by simulating uniform or Gaussian noise of different intensity, undersampling, missing parts, small deformations, or a combination of such point cloud artifacts. In addition, we are not interested in the whole segmentation process, but we are focused on a sub-issue of it, that is the problem of classification and recognition of segments.

Contribution
The aim of this SHREC track is to evaluate the quality of automatic algorithms for fitting and recognising geometric primitives in point clouds. The goal is to identify, for each point cloud, its primitive type (i.e., planes, spheres, cylinders, cones and tori) and some geometric descriptors.
To this aim, a dataset of 3D segments, corresponding to parts of primitives and represented as point clouds, divided in training set and test set was generated. For each segment in the training set, its primitive type and an analytical representation of it were made available to the track participants. For each point cloud in the test set, the goal of the contest is twofold: to recognise the primitive type from the data and to provide an analytical representation of the primitive recognised. The relevance or non-relevance of the simple primitives has been evaluated on the basis of a ground truth that derived from the knowledge of the primitive that originated that segment. Finally, the performance of the methods is discussed using classification, recognition and approximation measures that quantify the adherence to the original primitives and the goodness of fit.

Organization
The remainder of this paper is organized as follows. Section 2 describes the characteristics of the proposed benchmark: the dataset, the ground truth, and the quality measures chosen to evaluate classification, approximation and recognition of primitives. Section 3 presents a description of the six automatic algorithms that participate to this SHREC track: two of them are fully-based on neural networks, two are direct methods, while the remaining ones use deep learning in an initial classification step and proceed by applying direct methods. Finally, Section 4 provides an accurate comparative analysis of the results obtained by the six methods.

Dataset and ground truth
The dataset is composed of 46, 925 three-dimensional segments represented as point clouds; it is divided into a training set and a test set that contain, respectively, 46, 000 and 925 point clouds. Each point cloud is provided in a TXT file listing one triplet per line.
The point clouds (see Figure 1) are generated by sampling classical surface primitives derived from constructive solid geometry (i.e., planes, cylinders, spheres, cones and tori), by means of the following procedure. First, point clouds are sampled by using parametric equations in their canonical form, i.e., centered at the origin of the coordinate axes and with rotational axis aligned with the z−axis; the geometric quantities that define each primitive (i.e., amplitudes and radii, if any) are assigned randomly. Second, to obtain segments of different shapes, several cuts are enforced by exploiting random planes. Third, we randomly apply translations and/or rotations to recover primitives in their general position. Lastly, each point cloud is (potentially) processed so that it can be: • A0 -Clean. No perturbation is applied.
• A1 -Perturbed by uniform noise of different intensities. The noise is obtained by sampling uniform distributions of the form U(− 1 n , 1 n ), being 3 ≤ n ≤ 20 random, and adding such perturbations to a random percentage of the points.
• A2 -Perturbed by Gaussian noise of different intensities. The noise is obtained by sampling normal distributions among N (− 1 n , 4 n 2 ), being 10 ≤ n ≤ 30 random, and adding such perturbations to a random percentage of the points.
• A3 -Clean but affected by undersampling. We randomly select a percentage of points to be removed.
• A4 -Clean but affected by missing parts. We randomly choose a point of the point cloud; then, we remove all points contained inside the sphere having the selected point as center and radius assigned randomly.
• A5 -Perturbed by uniform noise of different intensities and undersampled.
• A6 -Perturbed by Gaussian noise of different intensities and undersampled.
• A7 -Perturbed by uniform noise of different intensities and affected by missing parts.
• A8 -Perturbed by Gaussian noise of different intensities and affected by missing parts.
• A9 -Clean but with local deformations. We randomly select a point of the point cloud and we apply a bivariate Gaussian centered at the selected point with a random covariance matrix.
For the sake of brevity, we will often refer to these point cloud artifacts as perturbation types. For each perturbation type A0, . . . , A8, training set and test set contain, respectively, 5, 000 and 100 point clouds; for perturbation type A9, training set and test set count, respectively, 1, 000 and 25 point clouds.
• Cylinder. We list the primitive type (for cylinders, codified as 2), the radius r, the unit vector determining the rotational axis a, and a point P sampled on the axis. In this case, the file contains the vector v = [2, r, a t , P t ] t .
• Sphere. We list the primitive type (for spheres, codified as 3), the radius r and the center C. The ground truth corresponds to the vector v = [3, r, C t ] t .
• Cone. We list the primitive type (for cones, codified as 4), half the aperture 2 α, the unit vector determining the rotational axis a and the vertex C. Then, the file provides the vector v = [4, α, a t , C t ] t .
• Torus. We list the primitive type (for tori, codified as 5), the major and minor radii, respectively r and R, the unit vector determining the rotational axis a and the center C. In this case, the file contains the vector v = [5, r, R, a t , C t ] t .
For each point cloud, the track participants were required to return a TXT file per point cloud (of the test set), with the same syntax employed for the ground truth files of the training set.

Evaluation measures
Three tasks are considered in the evaluation phase: • Classification task. Methods are evaluated on the basis of their capability for identifying, for each point cloud of the test set, its primitive type.
• Recognition task. Methods are studied in terms of their ability to recover the geometric descriptors (e.g, radii of a torus, vertex of a cone) behind each segment.
• Fitting task. Regardless of the primitive type, methods' performance is studied with respect to the approximation of each point cloud.
To quantify the performance of automatic algorithms in classifying, recognizing and fitting simple geometric primitives on point clouds under different conditions, we therefore consider a variety of classification and approximation measures. This analysis will be further specified according to the type of geometric primitive and point cloud artifacts.

Classification measures
The confusion matrix [10] is a popular way to visualize the classification performance of a method. A confusion matrix CM is a square matrix whose order equals the number of classes in the dataset under study. Diagonal elements count true positives, i.e., all those items which have been correctly labeled as members of their ground truth classes: precisely, CM(i, i) is the number of items that have been correctly predicted as elements of class i. Off-diagonal elements give the numbers of items mislabeled by the classifier; precisely, CM(i, j), with j = i, is the number of elements wrongly labeled as belonging to class j rather than to class i. Ideals classifiers have diagonal classification matrices.
True Positive and Negative Rates True Positive Rate (TPR) refers to the method's ability to correctly identify positives (e.g., the percentage of cats correctly classified as cats). Likewise, True Negative Rate (TNR) measures the classifier's ability to identify negatives (e.g., the percentage of non-cats correctly classified as non-cats). In statistics, TPR and TNR are also called sensitivity and specificity. A perfect classifier is 100% sensitive and 100% specific.
Positive and negative predicted values Besides TPR and TNR, to quantify the likelihood that a method returns a true positive (or true negative) rather than a false-positive (or a false-negative) we consider the Positive Predictive Value (PPV) and the Negative Predictive Value (NPV). PPV is the ratio of true positives to the number of items labeled as positives by the classifier. Similarly, NPV is the ratio of true negatives to the number of items classified as negatives.
Accuracy Accuracy describes how often a classifier is correct in its predictions: precisely, it is defined as the proportion of predictions that the classifier got right.
Macro-average Macro-average permits to evaluate the overall performance of the classifier treating all classes equally, it is defined as the arithmetic mean of one of the metrics mentioned above, computed independently for each class.

Fitting and recognition measures
To measure the approximation accuracy of a specific primitive, we exploit the geometric descriptors predicted by each method and the corresponding parametric representation to sample densely the recognised surface primitive, say S P . Let us consider a point cloud P to be evaluated; we use the following two measures to evaluate the approximation accuracy: • Mean Fitting Error (MFE): where d is the minimum Euclidean distance between x and every point in S P , and l is the diagonal of the minimum bounding box containing P.
• Directed Hausdorff distance: with d the Euclidean distance.
Finally, the recognition accuracy is evaluated in terms of L 2 norm between the column vectors of the ground truth and that obtained by each method. Specifically, following the notation introduced in Section 2.1, we compare the geometric descriptors recognised with those of the ground truth considering the L 2 distance: where v G and v P here denote the ground truth and the prediction vectors, respectively, where we remove the first entry and, for planes and cylinders, the point (since they are not comparable).

Description of the methods
Ten groups initially subscribed to this track, with a final number of six methods submitted. In the remainder of this section we describe the methods, here simply referred to as M1, M2, . . . , M6. The six methods can be grouped as: • Fully direct methods: accelerated Hough Transform (M1), and Histograms of local features + parameter fitting (M2).

Overall strategy
The proposed approach is based on an accelerated version of the Hough transform, a well known technique in the pattern recognition community, which have been recently extended to algebraic objects, see [12]. The general (algebraic) HT-framework deals with the problem of finding a (hyper)surface, within a family of (hyper)surfaces dependent on a set of unknown parameters, that best approximates an input shape. More precisely, given an input point cloud and a family of parametrized primitives with some unknown parameters, the core idea is to consider the parameter space corresponding to the selected family of primitives, to discretize it into cells, and to vote a cell every time the Hough transform of an input point intersects it; the most voted cell uniquely identifies the optimal solution of the recognition problem. Despite the robustness to various point cloud artifacts (e.g., noise, outliers, missing data and uneven sampling), a severe limitation of this paradigm is due to the curse of dimensionality; indeed, the dimension of the parameter space corresponds to the number of parameters that are meant to be estimated through the voting procedure. Only a few families of primitives can be considered in their general position, i.e., planes and spheres, but even for ellipsoids it is required that the cloud is centred in the origin of the Cartesian axes [13] to make the problem computationally affordable.
The proposed method takes in input a point cloud P and, for each family of primitives F (among planes, cylinders, spheres, cones and tori), it works in three main steps.
• Pre-processing. Independently from the primitive type, the normal vectors of the points are computed using the Hough transform technique and the family of planes in the Hesse normal form, as described in [14]. The adoption of a voting strategy makes this computation robust to most point cloud artifacts. Once the candidate tangent planes have been estimated, the process is specialized for each type of primitive. Specifically, the rotations and/or translations necessary to bring the point cloud in standard form are based on the geometric properties of the primitive type in question. The result of this process is a new point cloud P and an initial estimates of the parameters of the primitive to be recognised. These parameters differ according to the type of family selected; for example in case of cylinders, tori or spheres, they correspond to the radii, while in case of cones they coincide with half the aperture.
• HT-based surface recognition. The new set of points P is the input of the classical HT-based recognition algorithm. The estimates obtained in the pre-processing step allow to localize the search in a dimensionally-reduced parameter space. The method returns the parameters of the surface that best fits P with respect to the given family of primitives. From these parameters, the geometric descriptors required by the track are obtained.
• Evaluation of the approximation accuracy. To evaluate the recognition accuracy of a specific primitive, the recognised surface is sampled from the parametric equation and the Mean Fitting Error as defined in Equation 1 is calculated between P and the sampled surface. Finally, the initial roto-translation is applied backwards in order to obtain the geometric descriptors for the point cloud P in its original position.
The three steps here described are repeated for each family of surfaces (that is, planes, cylinders, spheres, cones and tori). Then, the primitive type and the geometric descriptors of the surface with the lowest MFE is returned by the method. Figure 2 provides a graphical abstract of method M1.

Computational aspects
The tests were performed on a desktop equipped with an Intel Core i9 processor (at 3.6 GHz), in a Windows system 64 bits. The method was implemented in MATLAB. The execution time for each point cloud of the test set is, on average, 123 seconds.

Overall strategy
The proposed method pre-processes the input point cloud normalizing it in a unit cube. Then, the task is addressed with two main modules: a point cloud classification with k−nearest neighbours (k-NN) using a histogram of local features (Module 1), divided into several steps, and a shape parameter fitting (Module 2).
In the first step of Module 1, all objects are classified into two types: planar and non-planar segments. The latter are then classified into subcategories following two strategies: one considers cone and cylinder as two categories (Strategy 1), while the other on considers them as one category (Strategy 2). Both of them consider sphere and torus as one category. Finally, the primitive types are further specialized (to distinguish: tori from spheres for Strategy 1; cones, cylinders, tori and spheres for Strategy 2).
• Module 1: object shape classification. In the first step, both strategies search for all planes.
The Histogram of Oriented Normal Vectors (HONV, see [15]) is used to represent a point cloud, specifically, a 2D histogram (25 x 25). Each cell with a value greater than or equal to 80% of the maximum value in the histogram is shifted to the center of the 2D histogram. In this way, a multiple 2D histogram representation of a single object is created and the k−NN, with k = 3, is used to classify planar and non-planar objects. If a point cloud is classified as plane, the PCA is applied to further verify its planar property and to remove false-positive cases.
In the second step, the aim is to classify non-planar instances. Once the categories are defined accordingly to the strategy mentioned before, the HONV feature is used to represent the point cloud and the k−NN method is applied with k = 5. To classify sphere and torus, the Surflet-pair-relation histograms [16] are used in place of HONVs: in this way, each object can be represented with only one histogram. Finally, the k−NN with k = 15 is used to distinguish spheres from tori.
In Strategy 2, an additional step is required to classify cylinder and cone. In particular, the HONV feature is used to represent the point cloud and the k−NN with k = 5 is employed to differentiate between cylinder and cone.
• Module 2: finding shape parameters. For each primitive type, different approaches are considered to estimate and fine tune the shape parameters.
-Plane. The average value of all the points considered to be on the surface is calculated. Then, PCA is used to find the eigenvector corresponding to the smallest eigenvalue, which is taken as approximation of the normal vector.
-Sphere. A least squares sphere fit algorithm 3 is employed to find the center and radius of a sphere. In this case, only 30% of points are used and the process loops 100 times to find the parameters. Using the estimated parameter values, the number of points belonging to the surface of a sphere is computed, through the difference between the distance from the point to the center and the radius. If the difference is larger than 5% of the radius, that point is considered as an outlier. Finally, the set of parameter values with the most number of inliers is chosen.
-Cylinder. A cylinder fitting approach 4 is used to find the rotational axis and the radius by minimizing the common least squares loss.
-Cone. The RANSAC method proposed in [17] is followed to estimate the parameters for conic segments.
-Torus. The parameter values are initialised: the center in the origin, the rotation axis equivalent to the z−axis, the main radius 1 and the minor radius 0.1. First, the points are fitted like a 3D circle (find center, radius, axis of rotation) by optimizing the L 2 loss function. Then, the minor radius is calculated by averaging the distance of that circle to the set of points. Figure 3 provides a graphical abstract of method M2.

Computational aspects
The classification method needs at most 20 seconds on AMD Ryzen 5 3550H, when applying k−NN approach to classify all objects in each step of Strategy 1 and Strategy 2; this corresponds to an average of 0.02 seconds per model. Each point cloud might need multiple steps, and it takes at most 1 second to get the final primitive type prediction for an object. The shape parameter fitting methods take approximately 60 minutes on an Intel i5 10510U for all the 925 objects, which means about 3.89 seconds per model, on average.

Overall strategy
As for the previous method, M3 is based on a point cloud pre-processing, consisting of a centering and a scaling, and two consecutive steps: shape classification and parameter regression. This method considers the PointNet [18] architecture as backbone for all networks. A first neural network is used to return the class probabilities for the five classes: plane, cylinder, sphere, cone, and torus.
Depending on the predicted class, each point cloud is fed into one of five specific networks to obtain the primitive parameters. Networks differ in the final layer, whose outputs must depend on the primitive type; the last layer of the backend consists of 256 linear neurons, which means that the input for the last layer of the regression is a 256-dimensional vector. Each primitive parameter is thus generated by a different MLP head which consists of a linear layer. For example, the plane network has two heads in the final layer: a 256 × 3 linear layer for the normal and a 256 × 3 linear layer for the point in plane parameter.
When it comes to the classification task, the loss function used to train the network is the crossentropy loss between the one-hot activation functions from ground-truth and the network output. For the regression step, each primitive type has its own loss, defined as the sum of simpler functions: • The scalar loss L x,x compares a ground truth scalar parameters x with its predictionx, and is defined as the mean squared error. It can be used for radii and angles.
• The point loss L p,p is defined as the Euclidean distance between ground-truth point p and output pointp. It is used for points, e.g., for the sphere centers and the cone vertices.
• The vector loss controls the direction between a ground-truth vector v G and the output vector v O , and is defined as follow: It is used for normal vectors and rotational axes. The L 2 regularization between v G and v O enforces the componentwise similarity between vectors. The exponent of the cosine function is meant to improve the learning of normal directions.
The total loss of each primitive type is defined by summing the above-defined losses: Finally, the parameters are transformed with respect to the inverse translations and scaling applied in the preprocessing. The training is performed after splitting the original training set is split into two parts: a (sub-)training set (80%) and a validation set (20%). Figure 4 provides a graphical abstract of method M3.

Overall strategy
An initial preprocessing is applied to the input point clouds. First, they are translated to the origin, by subtracting the respective centroids. Following that, a normalization is applied, so that all points lie within the unit sphere. Random rotations about the x-, y-and z-axes are applied, for data augmentation purposes. The point clouds are finally converted into volumetric grids.
The backbone neural network for all models was originally proposed in the Minkowski Engine [19] as a candidate for the ModelNet classification challenge [20]. It is based on sparse voxel convolution, which minimizes the memory requirements associated with voxel based methods.
The network comprises of a voxel-wise input MLP and an input convolution. This is followed by three strided convolutions that progressively reduce the resolution of the voxel grid. After that, the output of the convolutional layers are projected on the initial grid, concatenated and finally passed through three strided convolutional layers. To extract the feature vector, Max and Mean pooling is applied to the resulting grid and the results are concatenated. All layers are followed by Batch Normalization and LeakyReLU activation function. For this track a 3-layer MLP head was used, with the number of output parameters changing according to the task.
M2's strategy involves two consequent steps: • Classification step. An instance of the model described above, trained for classification, is tasked to find which primitive the input points where sampled from.
• Fitting step. Five more copies of the same model, each one trained on a specific type of primitive, are considered for the fitting task. During this step, the primitive type is known, Figure 5: A graphical representation of the strategy adopted in method M4.
as it was predicted in the previous step. The model corresponding to that type of primitive is activated to regress the primitive's parameters. Although the network's performance is generally quite good, it can be increased even further by performing least-squares fitting. This is accomplished by using the network's prediction as an excellent initial guess, thereby making the fitting process very quick. For the least squares fitting, an existing python library geomfitty 5 is used and extended to include cones.
For the classification model, M2 minimizes the cross entropy loss. For the regression models, M4 uses a weighted combination of four basic losses, depending on the type of parameters to be predicted for a given primitive type: • Axis to axis loss. This loss is applied to predict direction vectors. Then, given two such vectors v 1 and v 2 the loss is defined as: Vectors are forced to have the same direction by minimizing the angle between them.
• Point to axis loss. This loss is used for anchor points that help to define the axes of various shapes. It is defined as the distance between the query point and the line that passes through the shape's axis.
• Point to point loss. This loss is used for the centers of spheres and tori. For a given point and a ground truth center it is defined as the Euclidean distance between them.
• Scalar to scalar loss. The remaining parameters such as radii and angles are simply scalars, so this loss is calculated via the mean squared error.
It is important to note that neural networks perform very poorly when tasked to predict numbers that are distributed over a wide range of values. The unit sphere normalization applied to the input helps with this problem, as it restricts said range to roughly (0, 1) for most types of parameters. In order to map these values back to the correct range we apply the inverse transformations that we applied to the input points.
The network was trained and evaluated on a 85% − 15% training-validation split of the given dataset and was found to perform very well, reaching over 95% validation accuracy. To further improve the performance, a backtracking strategy is implemented to deal with heavily perturbed primitives. After the first shape's parameters are regressed, the distance between the shape and the input points is measured. If the distance exceeds a pre-defined threshold, then it is assumed that Figure 6: A graphical representation of the strategy adopted in method M5.
our initial classification guess was wrong, so the network's guess with the second highest probability is taken and the process is repeated. For peculiar cases where the least-squares fitting diverges, the regressor network's predicted parameters is used as the final estimation. Figure 5 provides a graphical abstract of method M4.

Computational aspects
After experimenting on the training set, the voxel size was set to 0.05. For the minimization problem, M4 consider the optimization algorithm Adam [21].

Overall strategy
The input point clouds are pre-processed; specifically, each segment is re-sampled in order to obtain the same number of points (8100 points), as the number of points in each object is not constant. This strategy augments the data by holding two of the three (x, y, z) position and adds a small shift in the remainder coordinate to create new data points. Finally, normalization and random rotations about the z−axis are applied to increase the effectiveness of the training process. Similarly to M3, the core component for point cloud classification is the PointNet [18] architecture. This architecture has three main modules: the max pooling layer to aggregate information symmetrically from all the points symmetrically, a global and local feature combination structure and two joint alignment networks. The symmetrical max pooling layer helps to make model invariant to the orders of the input data. Input points are fed in a collection of multi-layer perceptron networks to extract global features. After computing the global point cloud feature vector, data are fed back to concatenate the global feature with each of the point features. M5's strategy works in two main steps: • Classification step. The data is trained by the above model with different parameters and the top three models are obtained with the highest accuracy in the validation test. The final label is returned after considering the majority votes from the models.
• Shape parameter fitting. For each primitive type, different ways are considered to estimate the shape parameters of each point cloud.
-Plane. The regression with least mean square loss is used to fit a 3D plane on the input point cloud.
-Sphere. The least mean square method is used to calculate the most appropriate sphere to the point cloud.
-Cylinder. The least square fitting method is used to fit the point clouds to an infinitely long cylinder, parameterized by an anchor point, the direction and the radius of the cylinder.
-Cone. Tangent planes are computed to approximate the vertex of the cone; then the circle that fits the set of points having the same distance from the vertex is used to find the rotational axis and half the aperture.
-Torus. The SciPy library is used to determine the torus that best fits the point cloud.
For the least squares fitting, the python library geomfitty is used (see method M4).
The networks are trained and evaluated by splitting the initial dataset in two parts, training set (80%) and validation set (20%); the validation accuracy is above 80%. Figure 6 provides a graphical abstract of method M5.

Computational aspects
In the experiments, a learning rate of 2.5 × 10 −4 is used, and Adam is used for optimization. The training is run on Google Colab Pro -GPU P100, for a total of 10 epochs. The average training times, per epoch, are: 50 minutes for the PointNet network and 8 min for all 3D geometry regressors. The execution time is, on average, 36 seconds for the classification of the 925 test segments, while it is 10 minutes to estimate their shape parameters.

M6: AlexNet and ISO reconstruction standards
This method is proposed by Yifan Qie and Nabil Anwer. The method implementation can be found at https://github.com/YifanQie/SHREC22_fitting_LURPA.git.

Overall strategy
Input point clouds are first preprocessed by centering them to the origin and scaling them into a unit sphere. Then, they are rotated so that the first principal axis of each point cloud lies along z−axis.
The primitive type identification of this method is based on AlexNet [22] which is a convolutional neural network architecture with 60 million parameters and 650,000 neurons. Segments are turned into images as input for training the neural network; images are collected from the principal directions for feature extraction and recognition. All the layers, except for the last three layers, are extracted form AlexNet. The updated last three layers consists of a fully connected layer, a softmax layer, and a classification output layer, where the fully connected layer is set to return as many numbers as the number of primitive types defined in contest (i.e., 5). Regarding the specificity of the input point clouds, transfer learning is used in the approach since the first layers in the pretrained networks have proved to be able to address simple pattern recognition in the images.
This strategy involves two main steps: • Classification step. The pre-trained neural network is used for the test set to provide the primitive type that identify the input point cloud.
• Fitting step. The surface reconstruction is tested using the methods considered in the ISO standards on geometrical product specifications and verification (GPS), detailed in [23]. Once the primitive type of a point cloud, that is the invariance class in ISO GPS terminology, is identified by the fine-tuned AlexNet, an ideal feature defined by a parametrized equation can be guaranteed using least square fitting to obtain a reconstructed surface, see [24].
The network was trained and evaluated on a 95% − 5% training-validation split of the given dataset, reaching over 87% validation accuracy. Figure 7 provides a graphical abstract of method M6.

Comparative analysis
The performance of each method presented in Section 3 is here quantitatively evaluated on the basis of the measures described in Section 2.2. For the sake of conciseness (and of the readers), we here report only tables referring to the whole test set; additional tables obtained by focusing on specific point cloud artifacts are provided as additional material in A and B. Figure 8 shows the confusion matrices over the whole test set. All the six methods have a high number of true positives, being cones and tori the most difficult primitive types to be recognized. A more thorough analysis is provided by Table 1, which contains the following information: Positive Predicted Value (PPV), Negative Predicted Value (NPV), True Positive Rate (TPR), True Negative Rate (TNR), Accuracy (ACC) and their macro-averages. We can notice that: • Methods M2, M3, M4 and M5 have NPV (slightly) higher than PPV: it is more likely for these methods to be right when reporting a negative rather than a positive. Methods M1 and M6 have mixed outcomes.

Classification task
• Methods M3 and M5 have TNR higher than TPR, making them more reliable in correctly finding true negatives rather than true positives. Methods M1, M2, M4 and M6 presents, for some primitive types, a TPR higher than the TNR: this means that, in some cases, the classifier is more likely to identify correctly true positives rather than true negatives.
• All methods have the accuracy over 90%, with the slightly lower scores for cylinders, cones and tori. Macro-averaged values of accuracy are always above 95%, with two of the three best scores reached by neural methods. Figure 9 clarifies the (global) impact of different point cloud artifacts upon the six methods, with respect to the classification measures. We observe that: • M1, M2, M4 and M6 reach their best performance on clean data. Surprisingly enough, M2, M4 and M6 perform best on data with small deformations: these three approaches appear to be able to extract some additional hidden information to improve their prediction capability.
• M1 suffers the most from undersampled data. On the other hand, coherently with the Hough paradigm on which the method relies on, it is not much affected by missing data.
• M3 and M4 have their lowest performance, respectively, in case of uniform and Gaussian noise. However, the grouped bar charts suggest these two methods are the least prone to misclassification. This result confirms the power of neural methods in handling classification tasks.  Table 1: Classification metrics for the whole test set, with respect to each primitive type: T1=plane, T2=cylinder, T3=sphere, T4=cone, T5=torus. Here, gold, silver and bronze identify the first, second and third best performance. The last column contains the macro averages.
• In spite of having, in most cases, the lowest classification performance, M2 has still no measure that goes below 80%. Compared to the other method using PointNet (M3), here the implementation seems to overfit way more the training data.
• M5 has a relatively high performance on undersampled data, when compared to other point cloud artifacts.
• The combination of noise and undersampling has proved to be the most challenging perturbation for M6 which, however, exhibits one of the best overall performance on small deformations.
By checking which segments are misclassified, it can be additionally observed that the segments which results being more problematic are, in many cases, the small ones. Tables 3-12 in A provide a complete overview of classification measures, specified with respect to the primitive type. Table 2 provides various statistics of the measures introduced in Section 2.2.2, when the whole test set is considered. It contains the following information: first, second and third quartiles, mean value and standard deviation. More specifically, these quartiles split a set of sorted real numbers into four parts of approximately equal cardinality: the first (Q1) and the third (Q3) quartiles are defined as the values such that, respectively, 25% and 75% of the numbers lie below them; the second quartile (Q2) is the median of the set. Quartiles' significance is due to their ability to identify possible outliers. Based on quartiles, we can notice that: Figure 9: Bar graphs of the macro averages for the classification measures, grouped by method. The legend reflects the color encoding of the perturbation types.

Recognition and fitting tasks
• The L 2 distance provides low values for all methods with regard to the three quartiles. The lowest Q1 has an order of magnitude of −14 and is obtained by method M6; the highest order of magnitude is that of method M3, and corresponds to −1. Q2 and Q3 have order of magnitudes between −3 and −1.
• With the sole exception of the third quartile of M5, quartiles of the MFEs lie under the order of magnitude of −2. The lowest values are obtained by M2.
• When it comes to quartiles, the Hausdorff distance the order of magnitude is generally (nonstrictly) lower than −1; the only exceptions fare the third quartiles of methods M3 and M5.
Based on the arithmetic mean and standard deviation, we can conclude that: • Methods M2, M5 and M6 are affected by numerical instability issues, which result in the presence of outliers; it is worth noting, however, this problem is limited to the top 25% of the errors.
• M2 presents 11 outliers corresponding to the recognition of cones in which the estimated halfangle estimate is close to zero and the estimated vertex is far away from the origin of the coordinate axes.
• M5 provides different outliers depending on the type of task. Specifically, it has 61 outliers in the parameter recognition of various cylinders, cones and tori; however, for these clouds, fitting errors are low. We therefore conclude that, in all these cases, the approximation is successful but at the cost of a poor recognition. From the fitting task point of view, MFE and directed Hausdorff distance have high values in correspondence of 18 outliers correctly identified as planes: here, the normal is correctly identified (thus, the low value of L 2 distance); on the other hand, the method fails in returning a point passing through each returned plane.
• M6 presents 30 outliers, which correspond to point clouds classified as cones; the problem these estimates suffer from are analogue to that of M2.
• M1 and M4 alternate the first and second places in terms of means and standard deviations, suggesting that direct and neural methods can both provide competitive solutions to the problem under study.
By checking the bar graphs of the log mean errors with respect to the perturbation type, see Figure 10, it is possible to note a generally low variation of the (average) performance of each method, with the exception of M2 and M6; for these results, the logarithmic scale was preferred because of the presence of outliers. It is indicative that, when the methods (i.e., all but M2 and M6) are right in predicting the primitive type, their estimates are not much influenced by the perturbation type. Moreover: • Methods M2 and M6 have better recognition and fitting performance when dealing with clean data or with point clouds having just small deformations. However, it is worth noticing once more that the poor performance mostly depends on the presence of outliers (see Table 2).
• M5 is on average better in fitting than in recognition: despite being the mean L 2 errors rather high, mean MFEs and directed Hausdorff distances are lower.
• M1 and M4 behave similarly in both recognition and fitting tasks.
As for classification measures, tables that specify recognition and fitting measures to the single point cloud artifacts are left to B (see Tables 13-22).

Concluding remarks
Six methods were submitted for evaluation to this SHREC track on the fitting and recognition of simple geometric primitives on point clouds. Two approaches -M1 and M2 -base their whole pipeline on direct methods; M3 and M4 are fully based in neural networks; M5 and M6 exploits both methodologies, that is, neural architectures for the classification task and direct methods for the fitting and recognition tasks. The dataset was specifically designed to allow the training of data-driven methods, with a training set of 46000 primitives and a test set of 925 models. Nine types of perturbations were randomly applied to each type of primitive in the ratio of 5000 training examples and 100 test samples, with the exception of local deformations that count 1000 training examples and 25 test samples. The outcome of the methods shows that, on simple primitives, data-driven methods are reaching a satisfactory level of maturity, obtaining the best scores both in primitive classification and in the parameter estimation. Direct methods remain fiercely competitive, and seem to suffer more in classification than in fitting and recognition. We believe that a similar comparative analysis will be possible in the future on models with multiple primitives.
As a possible future development, to better assess the potential of these methods in a reverse engineering context, it would be interesting to create a dataset with real data, e.g., captured with a high quality industrial 3D scanner, and derive the ground truth coming from the specifications used in the fabrication of the targets. To determine which approach (or which paradigm, between direct and neural methods) has the best generalization capability, it would be certainly worthwhile to keep the training set introduced in this track and use real data just in the test set. This direction is however rather challenging, as it requires having sufficient diversity, in terms of models to be scanned, between the primitives and parameters used for training and testing sets. Method  T1  T2  T3  T4  T5  Avg  M1 1