12 SGP Symm2Corr

12 SGP Symm2Corr
Volume 31 (2012), Number 5
Eurographics Symposium on Geometry Processing 2012
Eitan Grinspun and Niloy Mitra
(Guest Editors)
Finding Surface Correspondences
Using Symmetry Axis Curves
Tianqiang Liu, Vladimir G. Kim, and Thomas Funkhouser
Princeton University
Abstract
In this paper, we propose an automatic algorithm for finding a correspondence map between two 3D surfaces. The
key insight is that global reflective symmetry axes are stable, recognizable, semantic features of most real-world
surfaces. Thus, it is possible to find a useful map between two surfaces by first extracting symmetry axis curves,
aligning the extracted curves, and then extrapolating correspondences found on the curves to both surfaces. The
main advantages of this approach are efficiency and robustness: the difficult problem of finding a surface map is
reduced to three significantly easier problems: symmetry detection, curve alignment, and correspondence extrapolation, each of which has a robust, polynomial-time solution (e.g., optimal alignment of 1D curves is possible with
dynamic programming). We investigate of this approach on a wide range of examples, including both intrinsically
symmetric surfaces and polygon soups, and find that it is superior to previous methods in cases where two surfaces
have different overall shapes but similar reflective symmetry axes, a common case in computer graphics.
Categories and Subject Descriptors (according to ACM CCS):
1. Introduction
Finding semantically meaningful correspondences between
points on different (non-isometric) surfaces is a fundamental
problem in computer graphics with applications in morphing [Ale01], attribute transfer [KS04], and shape database
analysis [ACP03]. For most of these applications, the goal
is to find the map f : S1 → S2 between surface, S1 and S2 ,
that best aligns “semantically equivalent” points (e.g., when
given two human bodies, the map takes a point on the right
knee of one body to the equivalent point on the right knee of
the other, etc.).
Although there are many good methods to produce a
map with minimal distortion between two surfaces once a
sparse set of feature correspondences has been found or
given by a user ( [Ale01]), it is still quite challenging to
find a sparse set of feature correspondences completely automatically. Traditional methods detect a set of local features on the two surfaces and then perform a combinatorial
search of potential correspondences between them, computing for each an estimate of the distortion induced by the deformation aligning them (e.g., [ZSCO∗ 08]). This approach
is both time-consuming and error-prone when the surfaces
have significantly different local features (e.g., two different
shapes within the same object class, as shown in Figure 1a).
c 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell PublishComputer Graphics Forum ing Ltd. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ,
UK and 350 Main Street, Malden, MA 02148, USA.
Figure 1: Aligning symmetry axis curves helps find surface
correspondences. Given two 3D models, we extract symmetry axis curves from both, align the curves (left), and then
extrapolate the axis correspondences to produce a full surface map (right). In these images, points with the same color
are predicted correspondences.
The general problem is to discover stable, semanticallyequivalent features to match on significantly different surfaces.
In this paper, we investigate methods based on symmetry
analysis. Our key ideas are based on three observations: 1)
most objects in the real world have an extrinsic and/or intrin-
T. Liu & V. Kim & T. Funkhouser / Finding Surface CorrespondencesUsing Symmetry Axis Curves
sic reflective symmetry, at least approximately; 2) finding the
symmetry axis curve of a surface (the set of surface points
stationary in a symmetric map) is relatively easy, especially
for global symmetries; and 3) searching for corresponding
features on the 1D symmetry axis curves is significantly easier than on the 2D surfaces. These observations combine to
suggest an approach that leverages a symmetry axis curve
detection algorithm to find surface correspondences. Essentially, the reflective symmetry axis curve serves as a global
feature that can be detected and aligned robustly and efficiently, thereby simplifying the search for correspondences.
Once a sparse set of correspondences are found on the symmetry axis curve, they can be extrapolated to other points on
the surface with known methods.
For our investigation of this approach, we have developed a number of algorithms and experiments. First, we propose a method for detecting intrinsic reflective symmetry
axis curves based on blended intrinsic maps [KLF11]. Second, we propose an axis curve alignment algorithm based on
string matching that is efficient (polynomial-time) and robust to missing and extra parts. Third, we describe an algorithm based on blended intrinsic maps to extrapolate symmetry axis curve correspondences to surfaces with little conformal distortion. Fourth, we describe a method for fitting
a genus-zero mesh to arbitrary surface data to facilitate intrinsic symmetry detection and surface correspondence. Finally, we present results of experiments that suggest our
method out-performs the state-of-the-art on many types of
real-world surfaces. While each of these contributions is a
small research advance on its own, the combination leads to
a significant conclusion: aligning symmetry axis curves is
usually the best way to find maps between surfaces representing real-world objects in the same class.
2. Related Work
Finding correspondences between two surfaces is a classical
problem that has been studied in many fields [vKZHCO10].
In some applications, maps between surfaces can be modeled with a low-dimensional transformation, and thus finding
correspondences can be performed in polynomial-time. For
example, rigid transformations have six degrees of freedom
and can be searched efficiently with algorithms based on
Hough transformations, RANSAC, Geometric Hashing, etc.
Similar approaches have been applied for non-rigid transformations, including thin-plate splines [BR07] and conformal
maps [LF09]. The common drawback of these approaches is
that they require an analytical model for the deformation expected between two surfaces, and thus they cannot be used
when surfaces in the same class have significantly different
shapes.
Some methods find surface correspondences by embedding surfaces in a feature space where similar points have
similar coordinates and then produce a dense map based on
nearest neighbors in that space. For example, [BBK06] developed a Generalized Multidimensional Scaling (GMDS)
framework where a small number of correspondences define
an embedding of one surface onto another minimizing an approximation to the Gromov-Hausdorff distance. [OMMG10]
showed that a single correspondence can define a Heat Kernel Map, a high dimensional embedding of a surface invariant under isometry. These methods use heuristic search procedures to find initial point correspondences, and thus may
converge to a local minimum. Also, since correspondences
are formed by closest points in a high-dimensional space,
the resulting map is not guaranteed to be smooth, or even
continuous, when surfaces are not isometric.
In applications where correspondences must be found between significantly different shapes, most previous methods
rely upon detection of local features (tips of protrusions,
ridges and valleys, etc.) and then search for the permutation
of feature correspondences that provide the surface alignment with minimal deformation. For example, [ZSCO∗ 08]
extracts features at points whose average geodesic distance
to other points on the surface is locally maximal and then
performs a priority-driven search of potential correspondence sets, choosing the one for which the deformation induced in the surface by aligning the corresponding feature
points is minimal. This approach is extremely slow (the
search space is O(N!) for N feature correspondences), and
thus is only practical for finding small correspondence sets
and/or for aligning surfaces with distinctive local shape features that can be used to prune the search. In contrast, we
search for correspondences between features on 1D curves
(symmetry axis curves), which can be done efficiently using
dynamic programming, and thus we can guarantee an optimal solution in polynomial time.
Other methods have performed global shape analysis to
produce structures that are easier to match. For example,
several researchers have extracted graph structures in which
nodes represent “parts” and then reduce finding surface correspondences to a graph matching problem [HSKK01]. Unfortunately, subgraph isomorphism is still an NP-complete
problem, and finding matches where M nodes in one graph
correspond to N nodes in another is particularly difficult in
practice, and thus these methods are most effective when
the graph extraction algorithm is very robust and can generate similar topological structures for different shapes in the
same object class. Our observation is that reflective symmetry axis curves provide exactly such a structure for most object classes.
Other recent work has investigated using symmetry analysis to guide mesh processing algorithms. For example,
[MGP07] proposed an approach that can automatically enhance symmetries in a 3D shape; [PSG∗ 06] introduced a
method that captures reflectional symmetries, which can be
leveraged for alignment, matching, segmentation, and viewpoint selection; [GPF09] leveraged symmetry detection for
c 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell Publishing Ltd.
T. Liu & V. Kim & T. Funkhouser / Finding Surface CorrespondencesUsing Symmetry Axis Curves
remeshing, simplification, and beautification of 3D meshes.
Our work can be considered an extension of this line of
work to consider inter-surface correspondence. In this respect, it is related to [ZHG10], who leveraged extrinsic planar symmetry detection to guide transfer of textures between
surfaces representing human faces. Their method requires
human-specified landmark correspondences and is limited to
transfer between similar surfaces with consistent local texture features using a single extrinsic reflective symmetry.
Our method is fully automatic and works for a much broader
class of shapes.
3. Key Idea
In this paper, we take advantage of global reflective symmetries to simplify the surface correspondence problem. The
key idea is to extract curve(s) along the reflective symmetry
axis of each surface, then search for an optimal alignment
of the curves, and finally extrapolate correspondences on the
curves to the rest of the surfaces (Figure 1).
The rationale for this approach is remarkably simple.
First, almost every object in the real world is (at least approximately) symmetric – indeed, if you look around an office, it is difficult to find an object that does not have at least
one global reflective symmetry. It seems silly not to take advantage of these symmetries when searching for surface correspondences.
Second, finding a map between symmetric points is significantly easier than finding a map between points on different surfaces, both because the local geometric features
of semantic correspondences are more likely to be similar
for symmetric maps than for inter-surface maps, and because geometric invariants (angles, distances, derivatives,
etc.) must be preserved at the stationary points of a symmetry map, providing constraints that simplify map extraction.
Third, global reflective symmetry axis curves (stationary
points in the symmetric map) are usually semantically stable
features of objects – points on the symmetry axis curve of
one object usually correspond to points on the symmetry axis
curve of another. Therefore, the search for a map between
points on the symmetry axis curves must consider only correspondences between 1D sets of ordered points, rather than
between full 2D surfaces, a problem for which efficient algorithms based on dynamic programming are available.
Finally, computing a full inter-surface map from a given
set of point correspondences is a well-studied problem for
which several methods are available [Ale01], and thus we
can use point correspondences found on the symmetry axis
curve to establish a map for the entire surface automatically.
The main idea is that the difficult (intractable) problem
of finding a map between significantly different symmetric
surfaces can be reduced to a sequence of simpler problems,
each of which can be solved with well-studied polynomialtime algorithms. In particular, the most difficult problem in
c 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell Publishing Ltd.
surface mapping (finding a coarse set of semantic surface
correspondences) is reduced to a 1D string matching problem (aligning symmetry axis curves), which can be solved
in polynomial time with dynamic programming. This approach has broad applicability for classes of man-made objects with perfect extrinsic reflective symmetries (e.g., airplanes, chairs, etc.), as well as organic objects with approximate intrinsic reflective symmetries (e.g., people, animals,
etc.), as are commonly found in 3D model repositories.
4. Methods
The input to our system is a pair of manifold, genus zero
meshes, S1 and S2 , and the output is a map, m : S1 → S2 ,
which gives a correspondence on S2 for every point on S1 .
For each input S that is not manifold or genus zero, we
execute a preprocessing step that constructs such a mesh M
using a simple “shrink wrap” algorithm based on the variational level-set method described in [ZOMK00]. The algorithm starts with M on the bounding sphere of S and then
iteratively moves vertices towards closest points on S until either they lie on S or further movement would exceed a
maximum curvature threshold. M is then used for correspondence computations with other manifold, genus zero meshes.
Afterwards, correspondnences found on M can be projected
back onto the original input S using any interpolation scheme
(we use radial-basis functions and closest points).
The core of our method finds correspondences between
two manifold, genus zero meshes using the three stage processing pipeline shown in Figure 2. We first extract symmetry axis curves (2b), then align the symmetry axis curves
(2c), and finally extrapolate the symmetry axis curve correspondences to a complete surface map (2d). The following subsections describe each of these steps in detail. Since
each step leverages prior techniques, there are only minor research contributions within each subsection. Combining this
sequence of steps into a system that automatically finds surface maps is the main contribution of our work.
Figure 2: System processing pipeline.
4.1. Symmetry Axis Extraction
The first step is to extract a set of candidate symmetry axis
curves from each surface. Given a densely sampled genuszero triangle mesh, M, we aim to find a set of curves on M
T. Liu & V. Kim & T. Funkhouser / Finding Surface CorrespondencesUsing Symmetry Axis Curves
that are stationary under a reflective symmetry r, r : M → M,
where r is a nearly-isometric map that takes the entire surface to itself. For intrinsic (extrinsic) symmetries, these are
the curves on M whose points are all nearly equal geodesic
(Euclidean) distance from both p and r(p) for all points p
on M. In our case, since the input surface is genus-zero after
preprocessing, each candidate symmetry axis forms a closed
1D curve C on M, which we output as a sequence of mesh
vertices. For each candidate C, we also produce a quality
score, QAxis (C), estimating how good the curve is for surface correspondence.
Several methods are possible to extract a symmetry axis
curve on a mesh [vKZHCO10]. We have adopted an approach that first generates a symmetry map from the surface
onto itself, then computes a function measuring the distance
from a surface point p to its correspondence in that map, and
finally extracts a symmetry axis curve C as the 0-level set of
that distance function. We found this approach to be more robust that other methods (e.g., PIRS voting [XZT∗ 09]), first
because our algorithm for generating symmetry maps explicitly tries to maintain conformality and thus is more robust to
deviations from isometry, and second because it is easier to
extract a symmetry axis curve from a distance function than
from functions produced by voting, which may have very
subtle ridges on thin extremities (e.g., the tail of a cat) unless extremely large numbers of votes are cast. Moreover, the
symmetry map produced in this step can be used to evaluate
the distortion of the map, which can be used to estimate the
quality of the symmetry axis curve, QAxis (C), and it can be
used to facilitate correspondence extrapolation later in the
processing pipeline.
The main challenge then is to generate candidate symmetry maps. For extrinsic reflectional symmetries (planar reflections that map the entire surface onto itself), this problem is trivial, and several good algorithms are available (e.g.,
[OO96, PSG∗ 06]). However, for intrinsic symmetries (isometric maps from the surface onto itself), the problem is
more difficult, especially when the best symmetry map is
not perfectly isometric. So, we take a two-phase approach,
where we first check whether the input surface has approximate extrinsic reflective symmetries, and then check for approximate intrinsic reflective symmetries only if an extrinsic
one cannot be found.
To check for extrinsic reflective symmetries, we perform
a rigid transformation of the mesh so that its centroid is at
the origin and its principal axes are aligned with Cartesian
axis vectors. Then, we consider reflections across planes defined by x = 0, y = 0, z = 0 as candidates for an extrinsic,
global reflective symmetry transformation [OO96]. For each
plane, we reflect points sampled uniformly from the surface across the plane and compute the average distance δ
to the closest
p point on the mesh. If δ is less than a threshold
(ε = 0.03 Area(M)), we accept the plane reflection as an
extrinsic symmetry and create a candidate map r that takes
each vertex v to the vertex on the mesh closest to its reflection. The axis curve C is then extracted as the 0-level set of
d(p, r(p)) using topological thinning, and the axis quality is
determined by QAxis (C) = Length(C).
To find intrinsic reflective symmetry maps, we consider
conformal maps that correspond the mesh to itself. Like
[KLF11], the basis of our approach is blended intrinsic maps
(BIM) – i.e., each candidate symmetry map is a weighted
blend of conformal maps constructed from triplets of feature
points. However, using ideas in [KLCF10], we can specialize the algorithm to work more efficiently for reflectional
symmetry detection than it does in the general case by applying filters to the selection of feature point triplets that ensure consistency of symmetric spatial relationships and surface properties. Specifically, we first detect feature points
at extrema of the average geodesic distance (AGD) function [ZSCO∗ 08]. Then, we construct conformal maps for
triplets of feature point correspondences, considering each
feature point, a, paired with two others, b and c, and forming
the conformal map defined by (a → a, b → c, c → b), which
represents the hypothesis that a is on the symmetry axis and
b ↔ c are a symmetric correspondence. Then, consistent sets
of the low-distortion maps are identified by eigenanalysis
of a conformal map similarity matrix, and symmetry maps
are produced by blending conformal maps with weights provided by the top eigenvectors. Rather than producing only a
single best map, as was done in [KLF11], our method generates maps associated with all of the eigenvectors associated
with the top eigenvalues, ranked by estimates of area distortion, c(pi ), averaged over points pi sampled on the surface.
For each map r, we extract a candidate symmetry axis curve,
C, as the 0-level set of d(p, r(p)) using topological thinning,
and estimate its quality with
QAxis (C) = Length(C) ×
avg(c(pi )) − ct
1 − ct
(1)
where ct = 0.5, and discard it if QAxis (C) is negative (the
symmetry map has large distortion). The final result then is a
set of symmetry axis curves with associated symmetry maps
and quality estimates.
4.2. Symmetry Axis Alignment
Our next step finds the optimal alignment (map) between
symmetry axis curves.
In this section, we first define the axis alignment problem
and then describe an efficient algorithm to solve it. Since
symmetry axes are 1D curves, we are able to find an optimal
solution in polynomial-time.
Problem formulation. Given two sets of symmetry axis
curves, C1 = {C11 ,C12 , . . . ,C1k1 } and C2 = {C21 ,C22 , . . . ,C2k2 },
extracted from meshes M1 and M2 , our goal is to find a pair
of axis curves (C1∗ ,C2∗ ), and an optimal alignment c∗ that
j
maximize a quality measure Q(C1i ,C2 , c), where (1 ≤ i ≤ k1 )
c 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell Publishing Ltd.
T. Liu & V. Kim & T. Funkhouser / Finding Surface CorrespondencesUsing Symmetry Axis Curves
curves, A ∈ {C1i } and B ∈ {C2i }, we discretize both curves
into an ordered set of N (N = 200) uniformly-spaced sample points (A = {a1 , a2 , . . . aN } and B = {b1 , b2 , . . . bN }) and
then define the alignment quality to be the inverse of the
string edit distance of alignment c:
QAlign (A, B, c) = 1/(D(A, B, c) + ε)
(a) Symmetry map
D(A, B, c) =
(b) Symmetry axis
Figure 3: Extraction of intrinsic symmetry axis curves. a)
The left image shows a reflective symmetry map, where
corresponding points are shown in the same color. b) The
right image shows distances between symmetric correspondences, where red is small and blue is large. The symmetry
axis curve (yellow) is the 0-level set of this distance function.
Figure 4: Symmetry axis alignment. A sequence alignment
may have gaps and stretches, highlighted with red arrows
on the left. The optimal alignment is shown with correspondences shown in different colors on the right.
and (1 ≤ j ≤ k2 ):
(C1∗ ,C2∗ , c∗ ) = arg max {Q(C1i ,C2 , c)},
j
(2)
C1i ,C2j ,c
We define the quality of an alignment between any two
j
curves C1i and C2 as the product of three terms:
j
j
j
Q(C1i ,C2 , c) = QAxis (C1i ) · QAxis (C2 ) · QAlign (C1i ,C2 , c) (3)
j
The first two terms, QAxis (C1i ) and QAxis (C1 ), describe the
quality of each axis independently, as defined in Equation 1.
j
The last term, QAlign (C1i ,C2 , c), describes the quality of a
pairwise alignment c between two axis curves – its definition
and computation are the focus of this section.
j
Definition of QAlign . QAlign (C1i ,C2 , c) provides a measure
estimating how well a correspondence c aligns semantic feaj
tures of two axis curves C1i and C2 . For the classes of objects
considered in this paper (e.g., Figure 4), it should be robust
to non-isometric distortions (e.g., cat neck vs. giraffe neck),
extra or missing parts (e.g., cat tail vs. giraffe), and/or differences due to errors in symmetry axis curve extraction. Moreover, it should be defined so that its optimizer can be found
efficiently.
With these goals in mind, for each pair of symmetry axis
c 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell Publishing Ltd.
∑
γ(aik , b jk )
(4)
0≤k<K
where γ(·, ·) denotes the cost of a string edit operation.
If we use λ to denote an empty string, correspondences
and gaps can both be represented by alignment pairs, giving three possible edit operations: (1) substitutions (ai →
b j ), (2) insertions, (ai → λ), and (3) deletions, (λ → b j ).
A valid alignment, c, is a sequence of alignment pairs,
(ai0 , b j0 ), (ai1 , b j1 ), . . . , (aiK−1 , b jK−1 ), where ai ∈ {A, λ}
and bi ∈ {B, λ}, where point order in the strings is preserved
in the sequence of alignment pairs, and where points paired
with λ cannot also be paired with other points.
We define λ(·, ·) to balance efficient computations and
discriminative alignments. For each correspondence assignment with λ, we assign a constant gap penalty: γ(ai , λ) =
γ(λ, b j ) = 0.5. For assignments between each pair of points,
ai and b j , we define γ(ai , b j ) to be the L2 distance betweeen
shape descriptors computed at ai and b j . In our implementation, the shape descriptor for a point p on a symmetry axis
curve C of mesh M has 52 features in total. 32 of them represent the histogram of geodesic distances from p to all other
points on M (normalized so that each dimension has zero
mean and unit standard deviation) and 20 of them represent
the local curvature profile of M at p. The curvature profile
contains 10 features the curvature of C at p at different scales
ranging from 0.05 to 0.15 and 10 features representing the
curvature profile of M in the direction orthogonal to C at p at
the same scales. Although these shape descriptors (the curvature profiles) are not invariant to isometries or other simple
deformations, we find that they are more robust that many alternatives (e.g., HKS, Gaussian curvature, etc.) to intra-class
variations found in our data sets.
Computation of Qalign . To find the alignment c with the
minimal value of Qalign , we repeatedly employ an algorithm
based on dynamic programming. Since the algorithm is so
fast, we execute it for every pair of extracted curves using
both possible alignment directions and every possible starting point, and then return the best result.
For each possible pair of curves, alignment direction, and
starting point, the discrete optimization problem in Equation 4 can be solved optimally with Dynamic Time Warping,
a method traditionally used for speech analysis [RS80] and
recently used for shape matching [MP05]. The basic idea is
the optimal solution of the current sequences can be computed by the optimal solutions of their prefixes, and it can
be computed recursively as below (ignoring boundary con-
T. Liu & V. Kim & T. Funkhouser / Finding Surface CorrespondencesUsing Symmetry Axis Curves
ditions for brevity):
D(A1:i , B1: j ) =


 D(A1:i−1 , B1: j−1 ) + γ(ai , b j )
min D(A1:i−1 , B1: j ) + min(γ(ai , b j ), γ(ai , λ))


D(A1:i , B1: j−1 ) + min(γ(ai , b j ), γ(λ, b j ))
(5)
This approach is applied for every pair of symmetry axis
curves extracted from both meshes, C1 = {C11 ,C12 , . . . ,C1k1 }
and C2 = {C21 ,C22 , . . . ,C2k2 }, and then the best alignment is
output as the final result.
The overall complexity of this algorithm is O(K 2 N 3 ),
where K is the number of axes extracted per mesh and N is
the number of discrete points on each symmetry axis curve
(usually 200). Each dynamic program takes O(N 2 ) for each
of O(K 2 ) pairs of curves, O(N) starting points, and 2 directions. In practice, it runs in ∼ 5 seconds per pair of meshes.
It would be possible to improve the speed even further by a
factor of O(N/ log N) using a method by [MP05], but we did
not implement it since the speed of this step is fast already.
Finding vertex correspondences from the alignment. The
output of Dynamic Time Warping is a correspondence between point samples, which may contain one-to-many correspondences. To convert this one-to-many correspondences
into a one-to-one vertex correspondence, we replace the
curve segment defined by the multiple corresponding points
with its middle sample point. We then project each sample
point onto its closest vertex, and remove correspondences
where vertices are duplicated. The output is then a duplicatefree set of correspondences between vertices on two symmetry axis curves.
4.3. Correspondence Extrapolation
The next step is to extrapolate the correspondences on the
symmetry axis curves to the rest of the genus-zero triangle
mesh. Given a set of vertex correspondences, c = {ai , bi },
where ai ∈ A is a point on symmetry axis C1 of mesh M1
and bi ∈ B is on on C2 of M2 , the goal is to generate a map,
m : M1 → M2 , for all vertices in M1 .
This is a classic inter-surface mapping problem, for
which a number of solution methods have been proposed
[Ale01]. One approach is to embed the surfaces into a highdimensional space based on geodesic distances to points
with known correspondences and then to establish correspondences between closest points in the embedded space
(e.g., GMDS [BBK06]). Another approach is to map both
surfaces to a canonical domain (e.g., a sphere or a coarse
mesh), where given correspondence points align and distortion is minimal according to some metric (e.g., angle
deviations), and then interpolate the map in that domain
[KS04, PSS01, SAPH04]. We employ a two-stage hybrid algorithm that leverages both of these ideas.
Figure 5: Correspondence Extrapolation. Correspondences
on the symmetry axis curves are first extrapolated to a stable
set of feature points, and then interpolated to the rest of the
surface.
Our unique problem is that the set of correspondences
provided by the symmetry axis curve alignment, cC , usually
includes points only within central regions of the surfaces,
and thus correspondences may be extrapolated to other regions, sometimes over considerable distances, which is unstable and leads to map distortions. To address this problem,
we proceed in two phases (Figure 5). In the first phase, we
employ a linear assignment based only on axis correspondences cC to find new correspondences cF between highlydistinctive feature points far from the symmetry axis curves
(e.g., tips of protrusions). In the second phase, we compute
complete inter-surface map by approximately interpolating
the correspondences given by both cC and cF . In this way,
the correspondence set is enriched with well-spaced, stable
correspondences before it is interpolated over the entire surface. Details of this two-stage algorithm follow.
In the first phase, our goal is to extrapolate the set of correspondences cC given on the symmetry axis curves. Since
we cannot robustly compute a complete surface map directly, we focus only on stable feature points far from the
axis for which we can robustly extrapolate correspondences.
In our implementation, we extract a set of feature points,
F1 and F2 , at persistent maxima of geodesic distances from
the symmetry axis curve, using the same persistence definition as in [DLL∗ 10]. We then search for correspondences
between these points using a method based on the ideas of
Generalized Multidimensional Scaling (GMDS) [BBK06].
The feature points, F1 and F2 , are embedded into a high dimensional space where the i-th dimension represents a normalized distance to the point in the i-th correspondence of
the symmetry axis curve. Then, we conduct a linear assignment based on the distance in the embedded space, and build
a new correspondence set cF that includes pairs of feature
points ( f1 , f2 ). This procedure tends to find correspondences
that are stable, well spread-out on the surface, and semantically correct.
In the second phase, we fill in a full surface map based
on the correspondences discovered between axis and feature
points. Since the given correspondences now span the entire surface, this is a straight-forward inter-surface mapping
problem. In our implementation, we use a variant of blended
c 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell Publishing Ltd.
T. Liu & V. Kim & T. Funkhouser / Finding Surface CorrespondencesUsing Symmetry Axis Curves
intrinsic maps [KLF11], where conformal maps defined by
triplets of corresponding points are blended to form a smooth
map over the entire surface. With this approach the main design decision is to select which conformal maps to compute
and blend. If there are |cC | correspondences between points
on the symmetry axis curves and |cF | correspondences be
F|
tween feature points, then we could blend |cC |+|c
con3
formal maps, many of which would have redundant information. Our strategy is to blend conformal maps only constructed from triplets that contain two correspondences from
the axis curves and one from the feature points. Specifically,
we select M = 50 evenly spaced points on C1 , add another
point that is separated by 10% of the axis length, and form
a triplet from these two points combined with each of the
feature correspondences (for a total of M × |cF | triplets, or
approximately 500 in practice). These conformal maps are
blended using the “confidence weights” based on area distortion as described in [KLF11] to form the final map.
5. Results
We have executed a series of experiments designed to test the
performance of our method, to understand its speed, range of
applicability, and performance relative to the state-of-the-art.
Benchmark Results: Our first experiment tests how well
our algorithm performs for finding maps between nearly
isometric, genus-zero, watertight meshes. For this experiment, we follow the testing methodology prescribed by the
Surface Correspondence Benchmark described in [KLF11].
The benchmark provides pairs of surface meshes representing humans and animals taken from the SCAPE, TOSCA,
and SHREC Watertight 2007 data sets, and provides code to
evaluate and compare automatic surface mapping methods
with plots that show geodesic distance errors between predicted correspondences and ground truth correspondences
provided by people.
Figure 6 shows our results. In each plot, the horizontal
axis represents geodesic error and the vertical axis represents the fraction of ground truth correspondences within
predictions closer than that error threshold (higher curves
are better). For the sake of brevity, we show results of our
method (solid lines) only in comparison to Blended Intrinsic Maps (BIM) [KLF11] (dotted lines), which significantly
out-performs Mobius Voting [LF09], GMDS [BBK06], and
Heat Kernel Maps [OMMG10], and all other methods tested
on this data set. From the results, we see that our method
(solid lines) performs comparable to BIM (dotted lines) for
the NonRigidWorld (pink) and Human (green) data sets, and
better for the Animal (black) and SCAPE data sets (blue).
These results are encouraging because BIM is already very
good on some of these data sets (e.g., NonRigidWorld and
SCAPE). The improved result for animals best shows the
advantage of our approach – often the symmetry axis for
four-legged animals is robust to extract, align, and extrapoc 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell Publishing Ltd.
late, even in cases where the surface shapes are very different
(e.g., Figure 4).
Figure 6: Results for the Surface Correspondence Benchmark.
Impact of each step on the benchmark results: To investigate how much each step of computational pipeline
contributes to the final surface correspondence errors in
these benchmark tests, we ran a series of experiments in
which the pipeline was started from “perfect” data provided by a human at each successive stage of the pipeline.
Specifically, we compare final correspondences when a) the
pipeline runs in full starting with the original surface inputs, b) when the symmetry axis extraction step is skipped,
and the symmetry axis alignment step starts with humanextracted symmetry axes for all models, c) when the first
two steps are skipped, and the correspondence extrapolation
step starts from human-specified correspondences on every
pair of symmetry axes, and d) when all but the very last algorithm are skipped, and the system only extrapolates correspondences from human-specified correspondences on the
symmetry axes and extremal features. Results are shown for
the Animals data set of the Surface Correspondence Benchmark in Figure 7. In consideration of the fact that the magenta curve is shifted upward by ∼ 20% since 4 out of 21
ground-truth correspondences are given, the proximities of
curves to one another indicate that the main source of error
comes from the final step of the pipeline – correspondence
extrapolation – i.e., the results do not change much even if
perfect axis alignments and extremal feature points are provided by a human to the last stage of the pipeline. We attribute this to the failure of blended intrinsic maps to align
semantic features. Perhaps other methods that consider local
shape features instead of conformality and/or area preservation will produce better results for this benchmark.
Global symmetry detection benchmark results: To evaluate the quality of our symmetry detection algorithm in isolation, we compared the symmetry maps produced by the
first step of our pipeline to previous algorithms for that task
using the benchmark described in [KLCF10]. Specifically,
we predict a full map for each input model from every point
T. Liu & V. Kim & T. Funkhouser / Finding Surface CorrespondencesUsing Symmetry Axis Curves
Figure 7: Given ground truth in different stages
Figure 8: Results for SHREC Watertight 2007 pairs.
to its symmetry correspondence and then apply the metrics
of [KLCF10] to evaluate how well the predicted maps align
with human-specified ground truth. According to the “correspondence rate” metric, we extract “acceptable” correspondences for 91% of points on average over all three datasets
(vs 84% for [Kim et al 2010]). According to the “mesh rate”
statistic, we extract “acceptable” maps for 83% of meshes
(versus 77%). Our method for extrinsic symmetry axis extraction never fails according to these metrics. As such, we
believe that this algorithm could be useful by itself for symmetry map prediction in applications beyond the one proposed here.
ample, Figure 9 shows cases where BIM fails (red arrows
in right column), due to instability at thin junctions between
parts (ants), non-isometric regions (fish tails), and/or extra
parts (centaur-horse and deer-buck). In contrast, finding a
symmetric map for each model in these cases can be done robustly, since the two sides of the same surface are almost perfectly isometric. Moreover, since the symmetry axis curves
align semantic features (e.g., nose-to-nose, tail-to-tail, etc.),
and the symmetry axis curves span the surface sufficiently
for extrapolation, our algorithm has no trouble producing a
correct map in these cases (third column). We believe that
most pairs of 3D surfaces to be aligned for applications in
3D graphics fit these criteria.
SHREC Watertight 2007 results. To test our method on
a wider range of inputs, we also compared results for symmetric object classes in the SHREC Watertight 2007 Data
Set [GBP07]. Again, we show our results in solid lines and
BIM in dotted lines (Figure 8). Here there is a much bigger
difference in the results: our method provides better results
for object classes where multiple nearly-isometric mappings
are possible, and yet only one of them aligns the symmetry
axis. This is the case for Airplanes (BIM may map wings
to fuselage) and Ants (BIM maps front to back and/or top
to bottom), as shown in Figure 9. On the other hand, our
method performs worse in cases when the symmetry axes are
extracted poorly due to non-isometries in the symmetry map
(Teddy, Figure 10(a)), when extracted symmetry axis curves
are very short (Glasses, Figure 10(b)), and when the surfaces
have few feature points to guide surface map extrapolation
(e.g., Fish). Overall, our method has better results than BIM
for six object classes and worse results for three.
More difficult examples: While the results of tests on standard data sets are informative for comparison, they do not
test the full range of applicability of our method. Figures 1
and 9 show several more interesting examples. In general,
we find that our approach is most useful when the input surfaces are both symmetric and share a similar symmetry axis
curve, but vary significantly in overall shape. In these cases,
other algorithms that attempt to find a nearly isometric map
between the two surfaces will have great difficulty. For ex-
Our implementation also can handle polygon soups and
other surface inputs that present problems for other surface correspondence algorithms. In Figure 9, the deer in the
bottom row have 9 and 13 connected components, and the
mouse in Figure 2 has 209. Previous methods that solve
for correspondences based on a smooth surface deformation
model (e.g, BIM) cannot handle this type of input. By first
shrink-wrapping the model in a genus-zero triangle mesh,
we are able to employ algorithms that find correspondences
based on symmetry axes that can then be projected back on
the original surface data.
Failure Cases: Our method is not applicable for all object
types. Of course, it fails for objects that do not have a reflective symmetry, and thus it would not be useful for alignment
of partially occluded scan data, for example, unless partial
reflective symmetry axes could be identified robustly. Also,
it fails when axis curves are not extracted properly and when
symmetry axis curves are too short and few feature points
are available for extrapolation (Figure 10). These problems
could probably be addressed by using other symmetry detection algorithms and/or extracting other features to guide
extrapolation (e.g., ridges and valleys), but those are topics
for future investigation.
Timing: The computational complexity of our algorithm is
O(F 6 S log M + N 3 + NFS log M), where M ∼ 10, 000 is the
number of vertices in each genus-zero mesh, S = 128 is the
c 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell Publishing Ltd.
T. Liu & V. Kim & T. Funkhouser / Finding Surface CorrespondencesUsing Symmetry Axis Curves
Figure 10: Failure cases.
surface in its lifetime. The pairwise compute time is dominated by the correspondence extrapolation, which could be
significantly improved with other methods (we used Blended
intrinsic maps to favor ease of implementation and quality
of results). In any case, these compute times are appropriate
for automatic batch processing of models, as the system is
intended to be used.
Figure 9: Results for difficult cases. Each of the four rows
shows one example. a) A pair of input meshes is shown on
the left. b-c) Our extracted and aligned symmetry axes and
extrapolated surface maps are shown color coded by correspondence. d) Surface maps computed with Blended Intrinsic Maps (BIM) are shown for comparison on the right. Errors produced by BIM are indicated by red arrows.
number of points sampled on each mesh, F ∼ 5 − 10 is the
number of feature points on each mesh, and N = 200 is the
number of sample points on each symmetry axis curve. The
slowest steps are the ones that require computing blended intrinsic maps, as they require a log M search to find the closest
vertex for S sample points for each of ∼ 200 maps. Ironically, the problem of finding coarse point correspondences,
which traditionally is the hardest challenge in surface mapping (intractable in the general case), can be solved optimally with the least computation (O(N 3 )) of any step in our
system.
In practice, on a SunFire X4100 computer with an AMD
Opteron 275 Dual-Core 2.2GHz processor, preprocessing of
polygon soup models takes approximately one minute, extracting the symetry axis curves from an extrinsically symmetric mesh with 12,500 vertices takes around 5 seconds
or extracting intrinsic symmetry axis curves takes 1 minute,
aligning symmetry axis curves takes 5 second, extrapolating
the correspondence to intrinsic surfaces takes 3 minutes, and
transferring the map back to polygon soups takes around 2.5
minutes (if desired). The input preprocessing and symmetry
axis extraction steps must be done only once per mesh, and
so their compute times may be amortized across many executions if a surface will be mapped to more than one other
c 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell Publishing Ltd.
6. Conclusion
This paper describes a method for automatically finding a
map between surfaces based on extraction and alignment of
symmetry axis curves. Besides this main idea, the primary
research contribution is the design of a system that includes
algorithms for genus-0 mesh fitting, intrinsic reflective symmetry detection, symmetry axis curve alignment, and correspondence extrapolation. Results of experiments with this
system demonstrate that it can find surface maps at least as
well as the state-of-the-art for nearly isometric surface pairs
and better that other methods for many difficult cases, including ones where the input is a polygon soup.
Our system is an early investigation of how to use
global symmetry detection for automatic discovery of surface maps. As such, it has several limitations, which suggest topics for future work. First, it’s current algorithms
for symmetry map detection and correspondence extrapolation rely upon robust detection of feature points on and
off the symmetry axis. This is just an implementation detail of the method we’ve chosen, but it affects the final results of our system in some cases. Second, our work considers only global reflective symmetries: other global symmetry groups (e.g., rotation) might be just as valuable. In
broader terms, it would be interesting to consider other largescale shape features that are stable, easy to detect, and fast
to align. Some work has been done on extracting and matching internal symmetry axes and other part-based structures
(e.g., [HSKK01]), but topological representations are generally unstable with respect to shape variations. Finding other
stable global shapes features is an important topic in shape
analysis. At this time, we conjecture that the global reflective
symmetry axis curve is a sweet spot in this regard, but it is
T. Liu & V. Kim & T. Funkhouser / Finding Surface CorrespondencesUsing Symmetry Axis Curves
possible that better features will be discovered for alignment
in future work.
[PSG∗ 06] P ODOLAK J., S HILANE P., G OLOVINSKIY A.,
RUSINKIEWICZ S., F UNKHOUSER T.: A planar-reflective symmetry transform for 3d shapes. In ACM Transactions on Graphics
(Proc. SIGGRAPH) (2006). 2, 4
References
[PSS01] P RAUN E., S WELDENS W., S CHRÖDER P.: Consistent
mesh parameterizations. Proc. of SIGGRAPH 2001 (2001). 6
[ACP03] A LLEN B., C URLESS B., P OPOVI Ć Z.: The space
of all body shapes: reconstruction and parameterization from
range scans. ACM Transactions on Graphics (proc. SIGGRAPH)
(2003). 1
[Ale01] A LEXA M.: Recent advances in mesh morphing. Computer Graphics Forum 21, 2 (2001), 173–198. 1, 3, 6
[BBK06] B RONSTEIN A. M., B RONSTEIN M. M., K IMMEL
R.: Generalized multidimensional scaling: a framework for
isometry-invariant partial surface matching. Proc. National
Academy of Sciences (PNAS) (2006). 2, 6, 7
[BR07] B ROWN B. J., RUSINKIEWICZ S.: Global non-rigid
alignment of 3-d scans. ACM Transactions on Graphics (Proc.
SIGGRAPH) (2007). 2
[DLL∗ 10] D EY T., L I K., L UO C., R ANJAN P., S AFA I., WANG
Y.: Persistent heat signature for pose-oblivious matching of incomplete models. In Computer Graphics Forum (2010), vol. 29,
pp. 1545–1554. 6
[GBP07] G IORGI D., B IASOTTI S., PARABOSCHI L.: Shape retrieval contest 2007: Watertight models track. SHREC competition (2007). 8
[GPF09] G OLOVINSKIY A., P ODOLAK J., F UNKHOUSER T.:
Symmetry-aware mesh processing. Mathematics of Surfaces
LNCS 5654 (September 2009). 2
[HSKK01] H ILAGA M., S HINAGAWA Y., KOHMURA T., K UNII
T.: Topology matching for fully automatic similarity estimation of 3d shapes. In Proceedings of the 28th annual conference
on Computer graphics and interactive techniques (2001), ACM,
pp. 203–212. 2, 9
[RS80] R ABINER L., S CHMIDT C.: Application of dynamic time
warping to connected digit recognition. Acoustics, Speech and
Signal Processing, IEEE Transactions on 28, 4 (1980), 377–388.
5
[SAPH04] S CHREINER J., A SIRVATHAM A., P RAUN E., H OPPE
H.: Inter-surface mapping. ACM Transactions on Graphics
(Proc. SIGGRAPH) (2004). 6
[vKZHCO10] VAN K AICK O., Z HANG H., H AMARNEH G.,
C OHEN -O R D.: A survey on shape correspondence. Eurographics State-of-the-Art report (2010). 2, 4
[XZT∗ 09] X U K., Z HANG H., TAGLIASACCHI A., L IU L., L I
G., M ENG M., X IONG Y.: Partial intrinsic reflectional symmetry of 3d shapes. ACM Transactions on Graphics (TOG) 28, 5
(2009), 138. 4
[ZHG10] Z ENG W., H UA J., G U X.: Symmetric conformal mapping for surface matching and registration. International Journal
of CAD/CAM 9, 1 (2010). 3
[ZOMK00] Z HAO H., O SHER S., M ERRIMAN B., K ANG M.:
Implicit and nonparametric shape reconstruction from unorganized data using a variational level set method. Computer Vision
and Image Understanding 80, 3 (2000), 295–314. 3
[ZSCO∗ 08] Z HANG H., S HEFFER A., C OHEN -O R D., Z HOU
Q., VAN K AICK O., TAGLIASACCHI A.: Deformation-driven
shape correspondence. In Computer Graphics Forum (2008),
vol. 27, pp. 1431–1439. 1, 2, 4
[KLCF10] K IM V. G., L IPMAN Y., C HEN X., F UNKHOUSER T.:
Mobius transformations for global intrinsic symmetry analysis.
Computer Graphics Forum (Proc. of Symposium on Geometry
Processing) (2010). 4, 7, 8
[KLF11] K IM V. G., L IPMAN Y., F UNKHOUSER T.: Blended
intrinsic maps. Transactions on Graphics (Proc. of SIGGRAPH
2011) (2011). 2, 4, 7
[KS04] K RAEVOY V., S HEFFER A.: Cross-parameterization and
compatible remeshing of 3d models. ACM Transactions on
Graphics (Proc. SIGGRAPH 2004) (2004). 1, 6
[LF09] L IPMAN Y., F UNKHOUSER T.: Mobius voting for surface
correspondence. ACM Transactions on Graphics (Proc. SIGGRAPH) 28, 3 (Aug. 2009). 2, 7
[MGP07] M ITRA N. J., G UIBAS L., PAULY M.: Symmetrization. ACM Transactions on Graphics (SIGGRAPH) 26, 3 (2007),
#63, 1–8. 2
[MP05] M ARZAL A., PALAZÓN V.: Dynamic time warping of
cyclic strings for shape matching. Pattern Recognition and Image
Analysis (2005), 644–652. 5, 6
[OMMG10] OVSJANIKOV M., M ÉRIGOT Q., M ÉMOLI F.,
G UIBAS L.: One point isometric matching with the heat kernel.
In Computer Graphics Forum (2010), vol. 29, pp. 1555–1564. 2,
7
[OO96] O’M ARA D., OWENS R.: Measuring bilateral symmetry
in digital images. IEEE-TENCON - Digital Signal Processing
Applications (1996). 4
c 2012 The Author(s)
c 2012 The Eurographics Association and Blackwell Publishing Ltd.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement