Documentation of the OPENTURNS SVM module

Documentation of the OPENTURNS SVM module
OpenTURNS-otsvm – Documentation
2
Abstract
The purpose of this document is to present the OpenTURNS SVM module, which enables to realize support
vector regression and classification with Libsvm and to manipualte results with OpenTURNS.
It offers to the user the ability to:
• define a regression or a classification problem and solve it with Libsvm
• build OpenTURNS objects from Libsvm models
• extract metamodels of Libsvm as OpenTURNS metamodels
This document is organised according to the Open TURNS documentation :
• a Reference Guide which gives some theoretical basis on support vector regression.
• a Use cases Guide which details scripts in python (the Textual Interface langage of Open TURNS) and
helps the User to learn as quickly as possible the manipulation of the otsvm module.
• the User Manual which details the otsvm objects and give the list of their methods.
Contents
1 Reference Guide
1.1 Introduction to support vector machine
1.2 Linear SVM . . . . . . . . . . . . . . . .
1.2.1 Primal form . . . . . . . . . . . .
1.2.2 Dual form . . . . . . . . . . . . .
1.3 Soft margin . . . . . . . . . . . . . . . .
1.4 Nonlinear SVM . . . . . . . . . . . . . .
1.5 Classification . . . . . . . . . . . . . . .
1.6 Regression . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
3
3
3
4
4
5
5
6
6
Cases Guide
Which python modules to import ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
UC : creation of a SVM regression algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . .
UC : Creation of a Classification algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
8
8
11
3 User Manual
3.1 LibSVMRegression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 LibSVMClassification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Various . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
13
14
16
2 Use
2.1
2.2
2.3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
1
3
Reference Guide
The Libsvm library provides efficient algorithms to produce a model by Support Vector Machine.
1.1
Introduction to support vector machine
A support vector machine is a concept in statistics and computer science for a set of related supervised learning
methods that analyze data and recognize patterns, used for classification and regression analysis. The standard
SVM takes a set of input data and predicts, for each given input, which of two possible classes forms the input.
Given a set of training examples, each marked as belonging to one of two categories, a SVM training algorithm
builds a model that assigns new examples into one category or the other.
More formally, a SVM constructs a hyperplane in a high or infinite-dimensional space, which can be used for
classification or regression. A good separation is acheived by the hyperplane that has the largest distance to
the nearest training data point of any class.
Whereas the original problem may be stated in a finite dimensional space, it often happens that the sets to
discriminate are not linearly separable in that space. For this reason, it was proposed that the original finitedimensional space be mapped into a much higher-dimensional space, presumably making the separation easier
in that space. To keep the computational load reasonable, the mappings used by SVM schemes are designed
to ensure that dot products may be computed easily in terms of the variables in the original space, by defining
them in terms of a kernel function K(x, y) selected to suit the problem.
1.2
Linear SVM
Given some training data D, a set of n points of the form
D = {(xi , yi ) | xi ∈ Rp , yi ∈ {−1, 1}}ni=1
where the yi is either 1 or -1, indicating the class to which the point xi belongs. Each xi is a p-dimensional
real vector. We want to find the maximum-margin hyperplane that divides the points having yi = 1 from those
having yi = −1.
Any hyperplane can be written as the set of points x satisfying
w·x−b=0
where · denotes the dot product and w the normal vector to the hyperplane. We want to choose the w and b
to maximize the margin, or distance between the parallel hyperplanes that are as far apart as possible while
still separating the data. These hyperplanes can be described by the equations
w·x−b=1
and
w · x − b = −1
2
The distance between these two hyperplanes is ||w||
, so we want to minimize ||w||. As we also have to prevent
data points from falling into the margin, we add the following constraint : for each i either
yi (w · xi − b) ≥ 1
f or all 1 ≤ i ≤ n
(1)
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
4
So we get the optimization problem :
M in ||w||
subject to (f or any i = 1, ..., n)
yi (w · xi − b) ≥ 1
1.2.1
Primal form
The optimization problem presented in the preceding section is difficult to solve because it depends on ||w||,
which involves a square root. It is possible to alter the equation by substituing ||w|| with 12 ||w||2 without
changing the solution. This is a quadratic programming optimization problem :
1
||w||2
2
subject to (f or any i = 1, ..., n)
M in
yi (w · xi − b) ≥ 1
By introducing Lagrange multipliers α, the previous constrained problem can be expressed as
n
X
1
αi [yi (w · xi − b) − 1]}
min max{ ||w||2 −
w,b α≥0 2
i=1
This problem can now be solved by standard quadratic programming techniques and programs. The stationary
Karush-Kuhn-Tucker condition implies that the solution can be expressed as a linear combination of the training
vectors
n
X
w=
αi yi xi
i=1
Only a few αi will be greater than zero. The corresponding xi are exactly the support vectors, which lie on
the margin and satisfy yi (w · xi − b) = 1.
1.2.2
Dual form
Using the fact that ||w||2 = w · w and substituing w =
reduces to the following optimization problem :
M ax L(α) =
n
X
i=1
Pn
i=1 αi xi yi ,
one can show that the dual of the SVM
n
X
1X
1X
αi αj yi yj xTi xj =
αi −
αi αj yi yj k(xi , xj )
αi −
2
2
i,j
i=1
i,j
subject to (f or any i = 1, ..., n)
αi ≥ 0
and to the constraint f rom the minimization in b
n
X
αi yi = 0
i=1
w can be computed thanks to the α terms :
w=
X
αi yi xi
i
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
1.3
5
Soft margin
If there exists no hyperplane that can split the ”yes” and ”no” examples, the soft margin method will choose
a hyperplane that splits the examples as cleanly as possible, while still maximizing the distance to the nearest
cleanly split examples. The method introduces slack variables, ξi , which measure the degree of misclassification
of the data xi
yi (w · xi − b) ≥ 1 − ξi 1 ≤ i ≤ n
The objective function is then increased by a function which penalizes non-zero ξi and the optimization becomes
a trade off between a large margin and a small error penalty. If the penalty function is linear, the optimization
problem becomes :
n
X
1
2
min{ ||w|| + C
ξi }
w,b,ξ 2
i=1
subject to (for any i = 1, ..., n)
yi (w · xi − b) ≥ 1 − ξi
ξi ≥ 0
This constaint with the objective of minimizing ||w|| can be solved using Lagrange multipliers as done above.
One has then to solve the problem :
n
n
n
i=1
i=1
i=1
X
X
X
1
ξi −
αi [yi (w · xi − b) − 1 + ξi ] −
min max{ ||w||2 + C
βi ξi }
w,b,ξ α,β 2
with αi , βi ≥ 0
The dual form becomes :
max {L(α) =
αi
n
X
αi −
1X
αi αj yi yj k(xi , xj )}
2
i,j
i=1
subject to ( for any i = 1, ..., n)
0 ≤ αi ≤ C
and
n
X
αi yi = 0
i=1
1.4
Nonlinear SVM
The algorithm is formally similar, except that every dot product is replaced by a nonlinear kernel function. This
allows the algorithm to fit the maximum-margin hyperplane in a transformed feature space. The transformation
may be nonlinear and the transformed space high dimensional, thus though the classifier is a hyperplane in the
high-dimensional feature space, it may be nonlinear in the original input space.
Some common kernels include :
• Polynomial : k(xi , xj ) = (xi · xj + c)d
• Gaussian Radial Basis Function : k(xi , xj ) = exp(−γ||xi − xj ||2 )
• Hyperbolic tangent : k(xi , xj ) = tanh(γxi · xj + c)
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
6
The kernel is related to the transform
P ϕ(xi ) by the equation k(xi , xj ) = ϕ(xi ) · ϕ(xj ). The value w is also in
the transformed space, with w = i αi yi ϕ(xi ).
The effectiveness of SVM depends on the selection of kernel, the kernel’s parameters, and soft margin parameter
C. A common choice is a Gaussian kernel, which has a single parameter γ. Best combination of C and γ is
selected by a grid search with exponentially growing sequences of C and γ. Each combination of parameter
choices is checked using cross validation, and the parameters with best cross-validation accuracy are picked.
The final model, which is used for testing and for classifying data, is then trained on the whole training set
using the selected parameters.
1.5
Classification
Given training vectors xi in two classes and a vector y ∈ −1, 1, C-SVC ( the algorithm that we use for
classification) solves the following dual problem :
1
minα αT Qα − eT α
2
0 ≤ αi ≤ C
yT α = 0
where e is the vector of all ones, C > 0 is the upper bound, Q is an l by l positive semidefinite matrix
Qij = yi yj K(xi , xj ) and K(xi , xj ) is the kernel. The decision function is
sign(
l
X
yi αi K(xi , x) + b)
i=1
For some classification problems, numbers of data in different classes are unbalanced. We can use different
penalty parameters in the SVM formulation, the dual of C-SVC becomes :
1
minα αT Qα − eT α
2
0 ≤ αi ≤ C+ , if yi = 1
0 ≤ αi ≤ C− , if yi = −1
yT α = 0
where C+ and C− depending on yi and yj and of weights that we can fix for each class.
1.6
Regression
Up to now, we presented SVM for classification. We can use too for Regression. This is similar to the nonlinear
case. We replace the soft margin by a ε-insensitive loss function which is defined like:
|y − f (x)|ε = max(0, |y − f (x)| − ε)
where f(x) is the loss function and ε a precision parameter.
We get this optimization problem if we introduce the slack variables ξ and ξi :
n
X
||w||2
minw {
+C
(ξi + ξi∗ )}
2
i=1
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
7
subject to (for any i = 1, ..., n)
li − wxi + b ≤ ε + ξi
wxi − b − li ≤ ε + ξi∗
ξi , ξi∗ ≥ 0
with C which is a control parameter like in soft margin.
To solve this problem, we introduce a new time Lagrange multipliers and we will get this regression function :
f (x) =
n
X
(αi − αi∗ )K(xi , x) − b
i=1
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
2
8
Use Cases Guide
This section presents the main functionalities of the module otsvm in their context.
2.1
Which python modules to import ?
In order to use the functionalities described in this documentation, it is necessary to import :
• the openturns python module which gives access to the OpenTURNS functionalities.
• the otsvm module which links the openturns functionalities.
Python script for this use case :
# Load OpenTURNS
from o p e n t u r n s import ∗
#Load t h e svm module
from otsvm import ∗
2.2
UC : creation of a SVM regression algorithm
The objective of this Use Case is to create a SVM Regression algorithm in order to create a metamodel.
Otsvm enables :
• to set lists of tradeoff factors and kernel parameter with the methods setTradeoffFactor, setKernelParameter.
• to select the kernel type in this list : Linear Kernel, Polynomial Kernel, Sigmoid Kernel, RBF kernel.
• to compute the algorithm on an input and output samples.
• to compute the algorithm on an experiment plane and a function.
• to compute the algorithm on an input and output samples and an isoprobabilistic distribution.
We recommend for users to use the RBF Kernel ( the gaussian kernel ). Moreover, it is important to understand
that the selection of parameters ( kernel parameter and tradeoff factor ) is primary. If you don’t know what
to take as parameters, you must take a wide range values, for example tradeof f ∈ {10− 5, 10− 3, 10− 1...103 }
kernel parameter ∈ {10− 15, 10− 13..., 103 }. Normally, the algorithm always converges, but this can take a long
while, mostly if you have a lot of parameters to test.
Python script for this UseCase :
#c r e a t e a f u n c t i o n , h e r e we c r e a t e t h e S o b o l f u n c t i o n
d i m en si on = 3
meanTh = 1 . 0
a = NumericalPoint ( dimension )
i n p u t V a r i a b l e s = D e s c r i p t i o n ( dimension )
outputVariables = Description (1)
o u t p u t V a r i a b l e s [ 0 ] = ”y”
formula = Description (1)
formula [ 0 ] = ”1.0”
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
9
covTh = 1 . 0
f o r i in range ( dimension ) :
a [ i ] = 0.5∗ i
covTh = covTh ∗ ( 1 . 0 + 1 . 0 / ( 3 . 0 ∗ ( 1 . 0 + a [ i ] ) ∗ ∗ 2 ) )
inputVariables [ i ] = ” xi ” + str ( i )
f o r m u l a [ 0 ] = f o r m u l a [ 0 ] + ” ∗ ( ( abs ( 4 . 0 ∗ x i ” +s t r ( i ) + ” −2.0) + ” +
s t r ( a [ i ] ) + ”) / ( 1 . 0 + ” + s t r ( a [ i ] ) + ”))”
covTh = covTh −1.0
model = NumericalMathFunction ( i n p u t V a r i a b l e s , o u t p u t V a r i a b l e s , f o r m u l a )
#c r e a t e t h e i n p u t d i s t r i b u t i o n
RandomGenerator . S e t S e e d ( 0 )
marginals = D i s t r i b u t i o n C o l l e c t i o n ( dimension )
f o r i in range ( dimension ) :
m a r g i n a l s [ i ] = Uniform ( 0 . 0 , 1 . 0 )
d i s t r i b u t i o n = ComposedDistribution ( marginals )
#c r e a t e l i s t s o f k e r n e l p a r a m e t e r s and t r a d e o f f f a c t o r s
t r a d e o f f = NumericalPoint ( [ 0 . 0 1 , 0 . 1 , 1 , 1 0 , 1 0 0 , 1 0 0 0 ] )
k e r n e l = NumericalPoint ( [ 0 . 0 0 1 , 0 . 0 1 , 0 . 1 , 1 , 1 0 , 1 0 0 ] )
#f i r s t example : c r e a t e t h e problem with an i n p u t and output s a m p l e s :
#f i r s t , we c r e a t e s a m p l e s
d a t a I n = d i s t r i b u t i o n . getNumericalSample ( 2 5 0 )
dataOut = model ( d a t a I n )
#second , we c r e a t e our svm r e g r e s s i o n o b j e c t , we must s e l e c t t h e t h i r d parameter
#i n an enumerate i n t h e l i s t { NormalRBF , L i n e a r , Sigmoid , Polynomial }
R e g r e s s i o n = SVMRegression ( dataIn , dataOut , LibSVM . NormalRBF )
#t h i r d , we s e t k e r n e l parameter and t r a d e o f f f a c t o r
Regression . setTradeoffFactor ( tradeoff )
Regression . setKernelParameter ( kernel )
# Perform t h e a l g o r i t h m
R e g r e s s i o n . run ( )
# Stream out t h e r e s u l t s
SVMRegressionResult = R e g r e s s i o n . g e t R e s u l t ( )
# get the r e s i d u a l e r r o r
residual = result . getResiduals ()
# get the r e l a t i v e e r r o r
relativeError = result . getRelativeErrors ()
#s e c o nd example : c r e a t e t h e problem with an e x p e r i m e n t p l a n e :
#f i r s t , we c r e a t e t h e p l a n e
myPlane = MonteCarloExperiment ( d i s t r i b u t i o n , 2 5 0 )
myExperiment = Experiment ( myPlane , ” example ” )
#second , we c r e a t e our svm r e g r e s s i o n o b j e c t , t h e f i r s t parameter i s t h e f u n c t i o n
R e g r e s s i o n 2 = SVMRegression ( model , myExperiment ,
LibSVM . L i n e a r )
#t h i r d , we s e t k e r n e l parameter and t r a d e o f f f a c t o r
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
10
Regression2 . setTradeoffFactor ( tradeoff )
Regression2 . setKernelParameter ( kernel )
# Perform t h e a l g o r i t h m
R e g r e s s i o n 2 . run ( )
# Stream out t h e r e s u l t s
SVMRegressionResult = R e g r e s s i o n 2 . g e t R e s u l t ( )
# get the r e s i d u a l e r r o r
residual = result . getResiduals ()
# get the r e l a t i v e e r r o r
relativeError = result . getRelativeErrors ()
#t h i r d example : c r e a t e t h e problem with an i s o p r o b a b i l i s t i c d i s t r i b u t i o n
#f i r s t , we c r e a t e our d i s t r i b u t i o n
marginals = D i s t r i b u t i o n C o l l e c t i o n ( dimension )
f o r i in range ( dimension ) :
m a r g i n a l s [ i ] = Uniform ( 0 . 0 , 1 . 0 )
d i s t r i b u t i o n = ComposedDistribution ( marginals )
#second , we c r e a t e i n p u t and output s a m p l e s
d a t a I n = d i s t r i b u t i o n . getNumericalSample ( 2 5 0 )
dataOut = model ( d a t a I n )
#t h i r d , we c r e a t e our svm r e g r e s s i o n
R e g r e s s i o n 3 = SVMRegression ( dataIn , dataOut , d i s t r i b u t i o n ,
LibSVM . Polynomial )
#and t o f i n i s h , we s e t k e r n e l parameter and t r a d e o f f f a c t o r
Regression3 . setTradeoffFactor ( tradeoff )
Regression3 . setKernelParameter ( kernel )
# Perform t h e a l g o r i t h m
R e g r e s s i o n 3 . run ( )
# Stream out t h e r e s u l t s
SVMRegressionResult = R e g r e s s i o n 3 . g e t R e s u l t ( )
# get the r e s i d u a l e r r o r
residual = result . getResiduals ()
# get the r e l a t i v e e r r o r
relativeError = result . getRelativeErrors ()
#f o u r t h example i s h e r e t o p r e s e n t you t h e SVMResourceMap c l a s s .
#U s e r s can f i x o t h e r s p a r a m e t e r s l i k e t h e d e g r e e and t h e c o n s t a n t o f t h e
#Polynomial Kernel , t h e c a c h e S i z e , t h e number o f f o l d s o r t h e e p s i l o n
#f i r s t , we c r e a t e s a m p l e s
d a t a I n = d i s t r i b u t i o n . getNumericalSample ( 2 5 0 )
dataOut = model ( d a t a I n )
#second , we c r e a t e our svm r e g r e s s i o n o b j e c t
#here , we s e l e c t t h e Polynomial K e r n e l but by d e f a u l t h i s d e g r e e i s 3 . We want a
#d e g r e e o f 2
ResourceMap . S e t ( ”LibSVM−D e g r e e P o l y n o m i a l K e r n e l ” , ” 2 ” )
#now t h e d e g r e e o f t h e Polynomial k e r n e l i s 2
R e g r e s s i o n = SVMRegression ( dataIn , dataOut , LibSVM . Polynomial )
#t h i r d , we s e t k e r n e l parameter and t r a d e o f f f a c t o r
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
11
Regression . setTradeoffFactor ( tradeoff )
Regression . setKernelParameter ( kernel )
# Perform t h e a l g o r i t h m
R e g r e s s i o n . run ( )
# Stream out t h e r e s u l t s
SVMRegressionResult = R e g r e s s i o n . g e t R e s u l t ( )
# get the r e s i d u a l e r r o r
residual = result . getResiduals ()
# get the r e l a t i v e e r r o r
relativeError = result . getRelativeErrors ()
2.3
UC : Creation of a Classification algorithm
The objective of this Use Case is to create a SVM Classification algorithm in order to build a metamodel that
separates datas in 2 classes.
Otsvm enables to :
• to set lists of tradeoff factors and kernel parameter with the methods setTradeoffFactor, setKernelParameter.
• to select the kernel type in this list : Linear Kernel, Polynomial Kernel, Sigmoid Kernel, RBF kernel.
• to compute the algorithm on an input and output samples.
Python script for this UseCase :
#t h i s example u s e s a c s v f i l e with t h e d a t a s f o r t h e c l a s s i f i c a t i o n
#we r e t r e i v e t h e sample from t h e f i l e sample . c s v
path = o s . path . abspath ( o s . path . dirname (
file
))
dataInOut = NumericalSample ( ) . ImportFromCSVFile ( path + ”/ sample . c s v ” )
#we c r e a t e d a t a I n and dataOut
d a t a I n=NumericalSample ( 8 6 1 , 2 )
dataOut=NumericalSample ( 8 6 1 , 1 )
#we b u i l d t h e i n p u t Sample and t h e output Sample b e c a u s e we must s e p a r a t e dataInOut
f o r i in range ( 8 6 1 ) :
a=dataInOut [ i ]
b=N u m e r i c a l P o i n t ( 2 )
b[0]= a [ 1 ]
b[1]= a [ 2 ]
d a t a I n [ i ]=b
dataOut [ i ]= i n t ( a [ 0 ] )
#l i s t o f C parameter
cp=N u m e r i c a l P o i n t ( [ 0 . 0 0 0 0 0 1 , 0 . 0 0 0 0 1 , 0 . 0 0 0 1 , 0 . 0 0 1 , 0 . 0 1 , 0 . 1 , 1 , 1 0 , 1 0 0 ] )
#l i s t o f gamma parameter i n k e r n e l f u n c t i o n
gamma=N u m e r i c a l P o i n t ( [ 0 . 0 0 0 0 0 1 , 0 . 0 0 0 0 1 , 0 . 0 0 0 1 , 0 . 0 0 1 , 0 . 0 1 , 0 . 1 , 1 , 1 0 , 1 0 0 ] )
#c r e a t e t h e C l a s s i f i c a t i o n Problem
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
12
R e g r e s s i o n=L i b S V M C l a s s i f i c a t i o n ( dataIn , dataOut )
R e g r e s s i o n . s e t K e r n e l T y p e (LibSVM . NormalRbf )
R e g r e s s i o n . s e t T r a d e o f f F a c t o r ( cp )
R e g r e s s i o n . s e t K e r n e l P a r a m e t e r (gamma)
#compute t h e c l a s s i f i c a t i o n
R e g r e s s i o n . run ( )
p r i n t ”#######################”
p r i n t ” R e s u l t s with Samples I /O”
p r i n t ” Accuracy ( p . c . ) = ” , R e g r e s s i o n . g e t A c c u r a c y ( )
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
3
13
User Manual
This section gives an exhaustive presentation of the objects and functions provided by the otsvm module, in
the alphabetic order.
3.1
LibSVMRegression
A LibSV M Regression is an otsvm object.
Usage :
LibSV M Regression(dataIn, dataOut, kerneltype)
LibSV M Regression(inputF unction, experience, kerneltype)
LibSV M Regression(dataIn, dataOut, distribution, kerneltype)
Arguments :
dataIn : a NumericalSample, an OpenTURNS object which is the input sample.
dataOut : a NumericalSample, an OpenTURNS object which is the output sample.
kerneltype : a Libsvm enum which is the selection of the kernel. We can select (Linear, Polynomial,
NormalRbf, Sigmoid).
experience : an Experiment, an OpenTURNS object which is an experiment plane.
distribution : a Distribution, an OpenTURNS object which is the isoprobalistic distribution
Value :
a LibSVMRegression which is the object manipulated to make the regression.
Some methods :
run
Usage : run()
Argument : none.
Value : none. It makes the svm regression and builds the metamodel. To understand the algorithm,
First, we make a cross validation to determinate the best parameters. Second, we train the problem and retreive some results ( support vectors, support vectors coefficients, kernel parameters).
Third, we build the model with OpenTurns and save results in the MetaModelResult.
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
3.2
14
LibSVMClassification
A LibSV M Classif ication is an otsvm object.
Usage :
LibSV M Classif ication(dataIn, outClasses)
Arguments :
dataIn : a NumericalSample, an OpenTURNS object which is the input sample.
outClasses : an Indices, an OpenTURNS object which is the output labels ( labels must be positive ).
Value :
a LibSVMClassification which is the object manipulated to make the classification.
Some methods :
run
Usage : run()
Argument : none.
Value : none. It makes the svm classification and builds the metamodel.
classif y
Usage :
getLabel(vector)
getLabel(sample)
Argument :
vector, an OpenTURNS object which is a NumericalPoint. This is the input vector to classify.
sample, an OpenTURNS object which is a NumericalSample. This is an input sample to
classify.
Value :
an UnsignedLong for the vector, an openturns type. It predicts for an input vector , the output
label.
a Indices for the sample, an openturns object. It predicts for a sample, the output label for
each vector.
setKernelT ype
Usage : setKernelT ype(kerneltype)
Argument : kerneltype: an enum from LibSVM. We can select (Linear, Polynomial, NormalRbf,
Sigmoid).
Value : none. It set the type of the kernel.
setT radeof f F actor
Usage : setT radeof f F actor(trade)
Argument : trade : a NumericalPoint, an OpenTURNS object.
Value : none. It set the tradeoff factor.
setKernelP arameter
Usage : setKernelP arameter(kernel)
Argument : kernel : a NumericalPoint, an OpenTURNS object.
Value : none. It set the kernel parameter.
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
15
grade
Usage :
grade(vector, outClasse)
grade(sample, indices)
Argument :
vector, an OpenTURNS object which is a NumericalPoint. This is the input vector.
sample, an OpenTURNS object which is a NumericalSample. This is the input sample.
outClasse, an UnsignedLong, an OpenTURNS type. This is a potential label.
indices, a Indices, an OpenTURNS object. This is a list of potentials labels.
Value :
an UnsignedLong for the vector, an openturns type. It gives a positive integer. More the value
is high and more the given label in parameter is a good label for the vector. The maximum
of this value is number of classes - 1.
a Indices for the sample, an openturns object. It gives a list of positives integers. More the
value is high and more the given label in parameter is a good label for each vector. The
maximum of this value is number of classes - 1.
setW eight
Usage : setW eight(weigt)
Argument : weight : a NumericalPoint, an OpenTURNS object.
Value : none. For each class, we associate a penalty. It is useful for unbalances data or
assymetric misclassification cost.
c
2005-2012
Phimeca Engineering
OpenTURNS-otsvm – Documentation
3.3
16
Various
The module adds some keys to the ResourceM ap in OpenTURNS.
There is five new keys :
• LibSV M − DegreeP olynomialKernel : the degree of the polynomial kernel d. By default, it’s 3.
• LibSV M − ConstantP olynomialKernel : the constant of the polynomial kernel c. By default it’s 0.
• LibSV M − CacheSize : the cache size of the calculation for kernel trick in MB. By default, it’s 100.
• LibSV M − Epsilon : the parameter ε in the regression. By default it’s 0.001.
• LibSV M Regression − N umberOf F olds : the number of folds for the cross validation. By default it’s 3.
• LibSV M − Shrinking : a boolean. We can use this to to eliminate outlying vectors. By default it’s
False.
If we want to change the value , we must use the method ResourceM ap.Set(key, value). If we want to retrieve
the value , we must use the method ResourceM ap.Get(key).
c
2005-2012
Phimeca Engineering
Download PDF