Links between Subjective Assessments and Objective Metrics for Steering

Links between Subjective Assessments and Objective Metrics for Steering
Links between Subjective Assessments
and Objective Metrics for Steering
Xuxin He and Zhicheng Su
Master Thesis in Vehicle Engineering
Vehicle Dynamics
Aeronautical and Vehicle Engineering
Royal Institute of Technology
TRITA-AVE 2012:36
ISSN 1651-7660
Postal address
Visiting address
Telephone
Internet
Royal Institute of Technology
Teknikringen 8
+46 8 790 6000
www.ave.kth.se
Vehicle Dynamics
Stockholm
Telefax
SE-100 44 Stockholm
+46 8 790 6500
TRITA-AVE 2012:36
ISSN 1651-7660
Links between Subjective Assessments and Objective Metrics for Steering
Xuxin He and Zhicheng Su
Master's Thesis in the Master's program Vehicle Engineering
©Xuxin He and Zhicheng Su, 2012
Abstract
The characteristics of vehicle steering perception are decisive factors concerning vehicle
safety and overall pleasure behind the wheel. It is a challenge for vehicle manufacturers to
achieve these features and qualities, because usually vehicle tuning almost only relies on
subjective evaluation of test drivers, which is costly and time consuming. In order to
optimize suspension design and develop a tool that can be used to evaluate steering with
objective metrics instead of subjective assessment, links between them must be confirmed.
In this master thesis, both objective and subjective testing data of over 20 vehicles across
four different segments are introduced in linear and nonlinear analysis. Linear regression
analysis is applied to investigate simply positive or negative correlation between a pair of
subjective-objective parameters. However, even if certain linear correlations are obtained, it
is still hard to define the optimal value for objective metrics. Considering that the general
shape of a correlation function can reveal which objective range give higher subjective
rating, it is possible to define these preferred ranges with Neural Network (NN). The best
data available is adopted from three drivers who tested 15 sedans, and some interesting
results are found.
The initial results demonstrate that NN is a powerful tool to uncover and graphically
illustrate the links between objective metrics and subjective assessments, i.e., the specific
range leading to better steering feel. Given a larger sample size, more reliable and optimal
links can be defined by following the same method. These confirmed links would enable
vehicle dynamics engineers to more effectively develop new vehicles with nearly perfect
steering feel.
Keywords: Steering feel, objective measure/metric, subjective assessment, linear regression,
nonlinear correlation, Neural Network, preferred range
I
Acknowledgment
The thesis work was performed during 2012-01 to 2012-06 at Vehicle Dynamics and
Calibration (96520) of Volvo Car Corporation (VCC). We are grateful for financial and
technical support from VCC.
It is with great gratitude that we acknowledge the continuous guidance and advices of our
supervisor Mikael Nybacka, Assistant Professor at Vehicle Dynamics of KTH Royal Institute
of Technology. We also would like to thank Carl Sandberg, the supervisor at VCC, for his
support and assistance.
We are indebted to many people, Lars Drugge at KTH and Stefan Karlsson, Egbert Bakker as
well as Kenneth Ekström at VCC, for their valuable feedback and suggestions on this thesis.
Finally, our appreciation also goes to all the colleagues and friends at KTH for a wonderful
time and interesting discussions about thesis and research.
Xuxin He and Zhicheng Su
Stockholm, June 2012
III
Contents
Abstract ..................................................................................................................................... I
Acknowledgment ................................................................................................................... III
List of Symbols .................................................................................................................... VII
List of Abbreviations ......................................................................................................... VIII
1 Introduction .......................................................................................................................... 1
1.1 Background ..................................................................................................................... 1
1.2 Problem Formulation....................................................................................................... 1
1.3 Objectives ........................................................................................................................ 2
2 Method .................................................................................................................................. 3
2.1 Case Study ....................................................................................................................... 3
2.2 Linear Regression Analysis ............................................................................................. 4
2.2.1 Simple Linear ........................................................................................................... 4
2.2.2 Multiple Linear ......................................................................................................... 5
2.3 Nonlinear Regression Analysis ....................................................................................... 6
2.3.1 Artificial Neural Network ......................................................................................... 6
2.3.2 Adaptive Neuro Fuzzy Inference System (ANFIS) ................................................ 14
3 Parameter Selection and Matching .................................................................................. 21
3.1 Selection of Subjective Assessments ............................................................................ 21
3.2 Objective Measures Used in Analysis ........................................................................... 22
3.3 Initial Parameter Matching ............................................................................................ 23
4 Mathematical Modeling..................................................................................................... 25
4.1 Representation of Simple Vehicle Model ..................................................................... 25
4.2 Parameters with Time Trajectories ............................................................................... 26
5 Evaluation of Acquired Data ............................................................................................ 29
5.1 Correlation between Objective Measures ..................................................................... 29
5.2 Evaluation of Subjective Assessments .......................................................................... 32
5.3 Evaluation of Rating Tendency ..................................................................................... 35
6 Linear Regression Analysis of Subjective and Objective Data ...................................... 41
6.1 Results from Simple Linear Regression ........................................................................ 41
6.2 Results from Multiple Linear Regression ..................................................................... 45
6.2.1 Regressor Elimination Process ............................................................................... 45
6.2.2 Multiple Linear Regression .................................................................................... 48
6.2.3 Subjective Ratings vs. Objective Measures Organized by Assessment Number ... 48
6.2.4 Best Correlating Assessments ................................................................................ 49
6.2.5 Interpretation of Results ......................................................................................... 52
6.2.6 Influence of Correlated Objective Metrics in Best Assessments ........................... 52
V
6.3 Result Analysis .............................................................................................................. 54
7 Nonlinear Regression Analysis of Subjective and Objective Data ................................ 57
7.1 Initial Results from Neural Network Model ................................................................. 57
7.1.1 Data Used in NN Training ...................................................................................... 57
7.1.2 NN Training............................................................................................................ 57
7.1.3 Correlation Results ................................................................................................. 59
7.2 Result Analysis .............................................................................................................. 60
8 Recommended Procedure of Obtaining Ideal Data for Analysis .................................. 63
8.1 Track Test with Real Vehicle ........................................................................................ 63
8.1.1 Test Track ............................................................................................................... 63
8.1.2 Various Vehicles .................................................................................................... 63
8.1.3 Same Vehicle with Various Steering Characteristics ............................................. 63
8.2 Simulator Experiments .................................................................................................. 65
8.3 Suitable Test Objects ..................................................................................................... 65
8.4 Test Subjects ................................................................................................................. 66
8.5 Pre-test with a Reference Vehicle ................................................................................. 66
8.6 Normalization Work...................................................................................................... 67
8.6.1 Rating Mean Normalization ................................................................................... 67
8.6.2 Spread Normalization ............................................................................................. 67
9 Summary Results ............................................................................................................... 69
9.1 Correlation of Objective Measures and Evaluation of Rating ...................................... 69
9.2 Confirmed Subjective-objective Links for Steering...................................................... 69
10 Conclusions and Recommendations ............................................................................... 71
10.1 Conclusions ................................................................................................................. 71
10.2 Recommendations to Future Work ............................................................................. 71
Reference ............................................................................................................................... 73
Appendix A ............................................................................................................................ 75
Appendix B ............................................................................................................................ 77
Appendix C ............................................................................................................................ 79
Appendix D ............................................................................................................................ 83
List of Symbols
Lateral acceleration
Bias
Cornering stiffness of front tires
Cornering stiffness of rear tires
Frequency
Gravitational acceleration
Lateral slip force on front tires
Lateral slip force on rear tires
Steering ratio between steering wheel and front wheel angle
Inertia around the z-axis
Front axle to center of gravity distance
Rear axle to center of gravity distance
Vehicle mass
Steering wheel torque
Curve radius
Time delay
Lateral velocity
Longitudinal velocity
Weight
Front slip angle
Rear slip angle
Regression coefficient
VII
Steering angle
Steering wheel angle
Sum
Roll angle
̇
Roll rate
Yaw angle
̇
Yaw rate
List of Abbreviations
ANFIS
Adaptive Neuro Fuzzy Inference System
AVES
Alliance Vehicle Evaluation Standard
BP
Back-propagation
FIS
Fuzzy Inference System
KTH
Royal Institute of Technology, Stockholm
ME
Mean error
MF
Membership function
MIRA
Motor Industry Research Association
NN
Neural Network
OM
Objective measure
SA
Subjective assessment
SS
Sum of squares
VCC
Volvo Car Corporation
Links between Subjective Assessments and Objective Metrics for Steering
1 Introduction
This chapter will give an overview of the background for this thesis. Based
on the state-of-the-art of research in subjective-objective links as well as
requirements from Volvo Car Corporation (VCC), specific objectives are
generated.
1.1 Background
Dynamics quality of a vehicle is a crucial aspect for safety and overall experience of driving
pleasure. Nowadays, vehicle dynamics characteristics could vary a lot even with similar
components from same suppliers. It is the appropriate calibration that makes a new product,
an integration of these components, receive good evaluation about the dynamic properties
from customers. Vehicle manufacturers usually use two approaches, subjective evaluation
using test drivers and objective metrics, to evaluate the dynamics features of a vehicle.
Though some of objective measures can be applied to describe dynamics behavior of a
vehicle, they cannot indicate if these technical settings lead to good interaction between a
driver and the vehicle. Thus, the subjective perception of driving or riding a vehicle is an
effective measure, in addition to objective metrics, to evaluate the performance of a vehicle.
Subjective evaluation of a vehicle includes various assessments to judge it as a whole.
Instead of monotonous objective data, the concrete driving experience of vehicle users is
revealed by drivability, handling, steering, ride comfort and some other specific perceptions.
So far many companies in vehicle industry use subjective evaluation in the tuning process of
vehicle development and some of them develop their own evaluation standards. AVES
(Alliance Vehicle Evaluation Standard) [1] defined by Renault-Nissan Alliance have been
adopted as a major item to evaluate the vehicle quality. Approximately 350 criteria
concerning static and dynamic characteristics are listed in this evaluation system. The most
common subjective rating scale from SAE Recommended Practice J1441 [2] has been widely
applied since 1985, as seen in the figure 1 in [2].
1.2 Problem Formulation
One of the most challenging tasks for vehicle engineers is, under cost and time constraints, to
satisfy every requirement of the subjective perception of customers regarding vehicle
dynamics characteristics. Subjective evaluation has its own limitations when the vehicle tests
need to be carried out in complicated or dangerous driving situations. Moreover, the
reliability and repeatability are also somehow weaknesses for pure subjective evaluation.
However, vehicle tuning still mainly relies on purely subjective evaluation of test drivers,
owing to a limited knowledge of links between subjective assessments and objective metrics.
This is not cost-effective during development. Therefore, there is a strong need in vehicle
industry to improve the efficiency of vehicle calibration.
1
AVL List GmbH has worked with their developed tool AVL-DRIVE [3] to benchmark
vehicle longitudinal drivability. The most significant information captured by this system
includes vehicle velocity, longitudinal acceleration, engine speed, pedal position, vibrations
and others giving over 500 objective criteria. Through analysis of driving test and calibration
both research associations and vehicle industries, such as MIRA (Motor Industry Research
Association) [4] and Scania CV AB [5], have already explored subjective-objective
correlations in vehicle steering. Links found from such work cannot only save efforts on
testing or calibration, but also help to lead to wanted subjective perception of steering feel for
general customers. In this thesis the related work is used to validate our findings.
1.3 Objectives
VCC has a strong need to improve development efficiency by testing fewer objective
measures that are enough to define excellent vehicle dynamics characteristics in subjective
perception.
The aim of this master’s thesis is to explore subjective-objective correlation links based on
the data measured and collected at VCC. Both linear and nonlinear correlations between
subjective assessments and objective metrics will be investigated by using tools of regression
analysis, variance analysis and MATLAB Neural Networks Toolbox.
The main application of these results should be to tell if a vehicle would give good driving
experience only according to data of certain objective measures within certain ranges. The
outcome is improvement of objective and subjective measuring standards by focusing on the
most significant parameters during tests, optimization of their value ranges and producing
sufficient data to serve in the practical calibration work.
2
Links between Subjective Assessments and Objective Metrics for Steering
2 Method
In this chapter the way to deal with data sets will be firstly described. Then,
the approaches used to analyze subjective-objective correlations will be
introduced.
2.1 Case Study
All data used in this thesis belongs to either subjective or objective part. Each test driver did
his or her own assessments on test vehicles. Therefore a number of subjective data sets of a
given vehicle can be attained depending on how many test drivers are involved. As for
objective metrics only one set of objective data from a specific vehicle is available by using a
testing robot [6]. Normally, the average value of the subjective data set and the objective set
will be used to carry out data analysis. However, this approach may not cover the true results
when considering our small data size. Chen [4] introduced case study method in subjectiveobjective correlation analysis. It will be an appropriate approach to describe and explain
correlation findings for each driver in this case. The way to analyze each case is shown in
Figure 1.
Figure 1. Approach of case study with subjective and objective data.
3
In addition, since every driver has his or her own driving preference, it is not likely to get a
general quantitative conclusion even though they are all expert drivers with large testing
experience. Instead, the method of case analysis will help to get a qualitative conclusion on
subjective-objective correlation by studying the preference tendency of most drivers. As seen
in Figure 1, assuming that Driver X is studied now, his subjective assessments and objective
measures are picked to conduct linear and nonlinear analysis. Thereafter, the possible
findings from this driver should be recorded. Likewise, the same analysis will be conducted
for the rest of drivers. If the same linear or nonlinear correlation is presented under most
cases (individual drivers), a reliable link can be confirmed.
2.2 Linear Regression Analysis
Linear regression is used to model the relationship between a dependent variable and one or
more explanatory variables. In both simple and multiple regression analysis, linear functions
and relevant parameters are used to model the data.
2.2.1 Simple Linear
Simple linear regression fits a straight line to estimate a linear model with one single
explanatory variable (objective measure in our case) , seen as Equation (1), where is true
value (subjective assessment in our case),
is the regression value and
is a random
component, also called error or residual.
(1).
The approach used to find the best fit is to minimize the sum of the squares of the difference
between the data and a line, so-called least squares minimization. The correlation coefficient
[
]) is widely adopted as a measure of the strength of linear dependence between
(
two variables. Two estimators, the constant
and the regression coefficient
are
determined by following minimization problem. The correlation coefficient can be defined as
Equation (2), where , ̅ and are sample size, sample mean and sample standard deviation
respectively.
∑(
̅
)(
̅
)
(2).
An example of how to process the data by using the linear regression can be seen in Figure 2.
A data set with five paired subjective ratings
and objective data
can be fitted with a
regression equation
. In this case, the value of regression coefficient is -0.84
that is close to -1, so the straight line fits the data set well. In other words, the regression
equation can basically represent the raw data.
4
Links between Subjective Assessments and Objective Metrics for Steering
An Example of Linear Regression ( r = - 0.84)
y ~= -1.7x + 13
8.5
Data
Fit
8
(x , y )
3
3
y: Ratings of Parking Efforts
7.5
(x4, y4)
7
(x1, y1)
6.5
6
(x5, y5)
5.5
(x , y )
2
5
2.8
3
3.2
3.4
3.6
3.8
x: Steering Wheel Torque [Nm]
4
2
4.2
Figure 2. Example of linear regression based on the data of a test driver.
2.2.2 Multiple Linear
Multi-linear regression basically follows the same procedure to achieve best fit. The only
difference is that more than one explanatory variable are introduced as seen in Equation (3).
All the estimators, including regression coefficient
are calculated by using
[ ]), seen as
least squares minimization and the coefficient of determination
(
Equation (4), is a measure to study how well the true values are likely to be predicted by the
model. In this expression,
is the total sum of squares, while
is the residual sum of
squares and means the predicted model.
(3).
̅
(4).
For multi-linear regression models, F-statistics can be used to check the significance of
regression equation and each regression coefficient in the equation. The F-value of regression
equation is expressed by Equation (5), which should follow F-distribution (in this thesis,
degree of freedom
is the number of regressors and
is the number of samples). By
checking the F-distribution table with certain confidence in Appendix A the significance
5
can be verified if the calculated F-value is larger than critical value
. The
significance of each regression coefficient is checked in a very similar approach. The
corresponding F-value
of a regression coefficient
is calculated by following Equation
(6), where
and
are the new sum of squares for the regression equation without
coefficient
.
. Likewise, the significance can be verified only if
⁄
⁄
⁄
is larger than
(5).
(6).
2.3 Nonlinear Regression Analysis
Results from linear correlation basically only tell if there is monotonically positive or
negative effect on some subjective assessments from certain objective measures. If an
optimal range needs to be defined for an objective measure that will give the best subjective
perception, then some other methods have to be used to find nonlinear correlations between
them.
2.3.1 Artificial Neural Network
Generally, Fuzzy Logic and Neural Network (NN) are frequently used tools to find out
nonlinear correlations between different parameters. The former one can be adopted for nonmetric subjective feel, such as ‘good/bad’ or ‘heavy/medium/soft’. The latter one can be used
while subjective ratings are in numbered form, which is available in this thesis. Thus, the
Neural Network Toolbox in Matlab is utilized.
NN is a kind of mathematical structure that composes of interconnected artificial neurons to
imitate the way a biological neural system works (e.g., our brain). It has the ability to learn
from data and can be used to explore nonlinear subjective-objective links in this thesis. A
typical multi-layer neural network consists of an input layer, hidden layer and output layer of
neurons. Its structure is graphically demonstrated as seen in Figure 3.
6
Links between Subjective Assessments and Objective Metrics for Steering
Figure 3. Structure of three-layer static Neural Network.
The input metrics
is transmitted through the connections by first multiplying the scalar
weight
to form a product
. Secondly, it will be added with a scalar bias
and
processed by a transfer function . Three types of transfer functions are most commonly
used. They are called hard-limit, linear and tan-sigmoid, shown in Figure 4. Tan-sigmoid is
usually adopted as the transfer function in the hidden layer to find out nonlinear subjectiveobjective correlations. Linear transfer function will be used in the output layer. The
calculated results from the hidden layer will then be added in the output layer, where output
is produced and can be compared with target value of subjective ratings. The main principle
of neural network is that scalar weight and bias can be adjusted in order to obtain desired
results [7].
Figure 4. Three types of most commonly used transfer functions.
7
There are two types of NN, feed-forward (static) and feed-back (dynamic). Signal flows of
both of them start from the input layer and end at the output layer. One of the main
differences between them is if the output depends on the previous output or not. Feedforward NN is mostly applied to model the system where the network is stable and the output
is only determined by input signals. This confirms exactly to the case of this thesis.
In a normal NN, there are several layers with neural nodes, including hidden layer and output
layer, transfer functions of layers and neurons in each hidden layer. In addition to these main
components and constructions, learning algorithm parameters and learning rate have to be
selected before training NN. The parameters that need to be decided are the following:
 Type of NN connectivity
As mentioned before, neural connections can be either feed-back or feed-forward. In this
thesis, the outputs or subjective ratings of drivers are only decided by the corresponding
objective data, and no previous outputs are needed as in the case of feed-back system. Thus,
the feed-forward (static) NN is chosen.
 Number of hidden layers
The number of hidden layers varies from zero to two. Without hidden layers, NN does not
differ a lot from linear regression, while too many hidden layers contribute to over-fit.
Research of Ash [8] also shows that one hidden layer can be suitable for this kind of
problem.
 Learning algorithm
The main learning algorithms available in NN can be divided into three types, supervised,
unsupervised and reinforcement learning. In the supervised learning, corresponding targets
are provided with each input; while in the unsupervised learning, no such output targets exist.
The reinforcement learning concerns with how an agent in an environment should react so as
to maximize rewards feedback from the environment, and this algorithm differs from the
supervised one in a way that no input/output mapping is presented.
In this thesis, since both numerical input and target output are available, the supervised
learning is suitable to be adopted, where the error correction will be used to minimize the
error between output and target by modifying weight and bias. In the field of studying
subjective-objective links, the most often used algorithm is back-propagation compared with
other algorithms such as perception, self-organizing, recurrent, etc [9].
Another important aspect is that the data used for training and testing the NN should be
different. This can be achieved by setting the dividing parameters, which decides the ratio of
vectors for training and testing as well.

8
Learning algorithm parameters (learning rate and momentum factor)
Links between Subjective Assessments and Objective Metrics for Steering
In a NN, the appropriate learning rate and momentum factor should be decided before
training. Too small learning rate leads to extremely long training time, while too large one
makes it hard for the NN to converge. The well-specified momentum constant helps to
reduce the learning time, which decides the proportion of previous weight update to make
use of. Problems of insignificance or instability could occur if the momentum factor is set too
low or too high. Since unfortunately no systematic method can be used to determine them,
both are set relatively low so as to guarantee the convergence of the NN.
 Initial weight value
Initial weight values are decided by experience, and this setting will not make great
fundamental difference in generalization quality between using optimal initial weights and
random initial weights [10]. Thus, the initial weight value is set to random values.
 Transfer function
The transfer function in the hidden layer is set as tan-sigmoid function, which is considered
able to represent the relationships best [11]. The transfer function in the output layer is just to
produce the final output by adding all from former hidden layer together, and since the rating
range is scaled between 0 and 10, the linear function is selected so that the output value can
be transferred into normal range (from 0 to 10), i.e., if the assessment is done by judging
‘good or bad’, the step function can be adopted.
 Number of neurons in each hidden layer
With more neurons in each hidden layer, the NN is capable of training the models with
higher complexity. The fit of training gets better with more complicated NN system.
However, the trained network with better fit (more neurons) usually over-fits the model,
which leads to larger error between predicted and true ratings for the testing data [11]. Three
neurons in the hidden layer are considered to be appropriate for the case studied in this thesis.
Trial Neural Network design with another data sample:
After deciding NN learning process, some example data from another similar research at
KTH is introduced to design a trial NN as follows:





Example data with 16 configurations (16 samples)
 Steering feel (SF) is the assessment
 Five objective parameters
 Data with speed at 80km/h
 Data from 15 expert drivers (with similar driving experience to VCC data)
Feed-forward NN with gradient descent back-propagation (BP) training function
One hidden layer with tan-sigmoid transfer function (tansig)
One output layer with linear transfer function (purelin)
Three neurons in the hidden layer
9






Maximum epochs: 500
Goal error:
Learning rate: 0.01 (default value)
Momentum factor: 0.9 (default value)
Ratio of vectors for training: 75% (12 samples)
Ratio of vectors for testing: 25% (4 samples which reach the minimum statistic number)
The single-input network is used to find the preferred range of an objective parameter where
higher subjective rating can be expected. As seen in Figure 5, the output is steering feel and
the input signal is one of five objective measures. Implementing this training process for
every driver with each objective parameter results in regression coefficients that are collected
and shown in Table 1. When r-value is larger than 0.7 it is marked as red to indicate a model
with successful training.
Figure 5. Single-input Neural Network structure with one-hidden-layer and three neurons.
Table 1. Overview of training regression (r-value) for expert drivers at 80km/h.
Expert driver No.
1
2
3
4
5
6
7
8
9
10 11 12 13 14
Max. lat.
0.30 0.40 0.30 0.20 0.40 0.50 0.40 0.50 0.40 0.20 0.20 0.20 0.30 0.30
acceleration
Steering
0.40 0.40 0.40 0.40 0.50 0.80 0.40 0.60 0.30 0.30 0.30 0.50 0.20 0.10
sensitivity
Steering
0.60 0.50 0.80 0.60 0.80 0.30 0.40 0.50 0.40 0.50 0.80 0.30 0.30 0.30
stiffness
Torque
0.60 0.50 0.70 0.50 0.70 0.40 0.30 0.80 0.30 0.80 0.30 0.10 0.30 0.10
gain
Yaw
0.60 0.10 0.10 0.50 0.50 0.20 0.10 0.40 0.50 0.30 0.30 0.40 0.10 0.20
delay
15
0.10
0.50
0.60
0.60
0.20
Most training results with good fit (r-value ≥ 0.7) concern steering stiffness and torque gain,
which are defined in Figure 6, and they are similarly defined as torque buildup into the
corner in VCC DNA files. If close examination is done, the relationship between these
10
Links between Subjective Assessments and Objective Metrics for Steering
parameters and steering feel rating should be like Figure 7, where certain preferred or dislike
range is visualized.
Figure 6. Definitions of steering stiffness and torque gain.
Neural Network training result for Expert Driver 3 @ 80km/h (16 configruations)
3.2
Real values
Predicted values
3
SF(steering feel) rating
2.8
2.6
2.4
2.2
2
1.8
1.6
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Steering stiffness [Nm/deg]
Figure 7. Relationship between steering feel and steering stiffness for Driver 3.
11
According to the figure above, the preferred steering stiffness is from 0.06 to 0.12Nm/deg,
which provides Driver 3 with better steering feel. Table 2 shows all preferred ranges of
different objective parameters from well-trained models. The main findings concern steering
stiffness and torque gain, both of which are preferred to be low to achieve better steering feel.
Thus, it is possible to find preferred range through NN, especially when the linear
correlations have already been confirmed.
Table 2. Preferred range for better SF rating (expert driver).
Objective measure
Preferred range for higher rating
Max. lat. Acceleration
NA
Steering sensitivity
NA
Steering stiffness
0.06~0.12 Nm/deg
Torque gain
0.05~0.15 Nm/deg
Yaw delay
NA
Apart from expert drivers, the data from other 15 novice drivers are also used to train NN
model. The valid findings are also focusing on steering stiffness and torque gain. The
preferred ranges of both parameters are almost exactly the same as expert drivers.
Another issue to consider is if the sample size could be smaller to save test time and cost. It
depends on how the NN processes data. The NN system chooses the training data and testing
data randomly even though the ratio of vectors for training and testing is defined. This fact
makes the trained vectors and tested vectors never fixed. Moreover, the data used for testing
is always much less than that for training. As a result, sometimes if extreme outliers are
included in vectors for testing, the regression will get quite unstable. The regression plots of
test in Figure 8 show how much the test regression could vary with different selected data in
training and testing.
12
Training: R=0.73831
Data
Fit
Y=T
3
2.5
2
2
2.5
3
Output ~= 0.28*Target + 2.1
Output ~= 0.53*Target + 1.2
Links between Subjective Assessments and Objective Metrics for Steering
Test: R=0.93766
3
Data
Fit
Y=T
2.8
2.6
2.4
2.2
2
2
2.5
Output ~= 0.55*Target + 1.2
Target
3
Target
All: R=0.73893
Data
Fit
Y=T
3
2.5
2
2
2.5
3
Training: R=0.72449
Data
Fit
Y=T
3
2.5
2
2
2.5
3
Output ~= 0.57*Target + 1.1
Target
Output ~= 0.27*Target + 1.9
Output ~= 0.58*Target + 1.1
Target
Test: R=0.096104
3
Data
Fit
Y=T
2.8
2.6
2.4
2.2
2
2
2.5
3
Target
All: R=0.64615
Data
Fit
Y=T
3
2.5
2
2
2.5
3
Target
Figure 8. Different regression-plots under the same data samples.
As seen in Figure 8, for the regression of training data and all data (the blue and red lines),
the fit is relatively stable (r-value of all samples fluctuates by 0.1 units). On the other hand,
the fit of testing data (the green line) varies remarkably (0.94 vs. 0.096). When the r-value is
0.94 in the first case, the randomly selected testing data by the NN can fit with predicted
13
model very fortunately. On the contrary, due to the small testing sample, if extreme data are
selected for testing, the r-value will get worse, i.e., in the second case the r-value is just
0.096.
With 16 samples in all, the fit of training model is stable. However, the error between true
value (target) and predicted model (output) will vary randomly for the testing part (only 4
samples). The point is that 16 samples are just enough to train the NN model, but sometimes
the stability of testing cannot be ensured. If the sample number is reduced slightly, the
reliability of preferred range through NN training will not be guaranteed anymore. In other
words, at least 16 test objects (cars) or different settings of the cars are needed in order to use
NN to find nonlinear correlations. As for how many test drivers are needed in this kind of
study, it will be discussed later on. Furthermore, there is a possibility to add to the data set
gathered data from a later test, meaning that if 16 settings could be varied during one test day
data from another day with different cars and/or settings can be added in order to reach a
better validity of the NN model.
2.3.2 Adaptive Neuro Fuzzy Inference System (ANFIS)
The fuzzy logic system is able to model nonlinear functions as well by mapping inputs into a
logic space first, and then inferring some rules to obtain wanted output. However, an
ordinary fuzzy logic system cannot learn rules without support from Neural Network. That is
why Adaptive Neuro Fuzzy Inference System (ANFIS) is needed.
ANFIS uses a hybrid learning algorithm to identify the membership function parameters of
single output, with first order Sugeno-type Fuzzy Inference System. The Tagaki-Sugeno
fuzzy if-then rules in ANFIS can be configured as follows:
The nth rule: if
is
and
is
and
is , then
where , , and
are linear parameters of function
and
signals.
,
,
and
,
are input
A normal structure of ANFIS is shown in Figure 9, totally with six distinct layers [12] [13]:
For instance, with this illustrated ANFIS, the 5th rule and the fifth function are valid when
is
and is
and
is .
14
Links between Subjective Assessments and Objective Metrics for Steering
Figure 9. ANFIS structure with first-order Sugeno model.
Layer 1 is the input layer, simply giving incoming signals, can be objective measures.
Layer 2 is the fuzzification layer. The output of each node in this layer is decided by the
membership function (MF). For example, given a triangular MF (trimf) shown in Figure 10,
it can be presented as Equation (7).
(7).
{
}
15
Figure 10. Membership function: trimf.
Layer 3 is the rule layer. Each node in this layer computes the weight
of the rules can be decided by orthogonalizing various inputs and MFs.
for the nth rule. All
Layer 4 is the normalization layer. The normalized firing strength of a given rule is the
output of this layer defined as the ratio of the firing strength of the considered node to the
sum of all rule firing strengths. The normalized firing strength for the nth node is represented
as Equation (8).
̅
(8).
Layer 5 is the defuzzification layer. The output of Layer 4 and initial inputs are received and
the weighted consequent value of a given rule is calculated as Equation (9).
̅
̅
(9).
Layer 6 is the layer with one summation node, where adding up all outputs of the previous
layer produces the single ANFIS output.
∑
̅
(10).
The ANFIS training adopts a gradient descent algorithm to optimize the antecedent
parameters and a least squares algorithm to solve for the consequent (linear) parameters.
Similar to the procedure of NN, parameters such as MF type, goal error and maximum
epochs have to be empirically determined before training. Above all, the number of MFs for
each input is extremely important, which decided how many logic divisions should be
generated.
16
Links between Subjective Assessments and Objective Metrics for Steering
Concerning subjective-objective correlation where subjective assessments can be seen as
output of the ANFIS and if a certain preferred range exists there should exist upper and lower
boundary for a given objective measure. That means only those values in the preferred range
will give better perception seen as Figure 11. So the objective measuring data should be
divided into three logic zones (three MFs), two of them giving lower rating score. However,
in the actual case, extreme values outside the boundaries might not be acquired on testing
vehicles. As a result, one or even two logic zones should be removed. Therefore, the number
of MFs in ANFIS training cannot be chosen in order to study subjective-objective links
unless extreme cases are surely detected.
Figure 11. General shape of subjective-objective correlation function.
ANFIS training with subjective rating of a certain level and of its sub-level
In order to test the ANFIS model a collection of subjective data from a sub-level is
introduced with rating range from 100 to 500-points that will replace the objective data in
Figure 11. All assessments in the sub-level are considered to have effect on the subjective
assessment (top-level) as seen in Figure 12. The 100 or 500-point ratings represent extreme
cases. Using these data samples it will be possible to demonstrate how ANFIS works and if
the parameters from the sub-level are enough to represent some property.
17
Figure 12. Explanation of sub-level for subjective assessment.
All four various assessments in the sub-level are included in the fuzzy inference system since
each sub-level assessment should give its contribution. With three MFs in each of them
shown in Figure 13, the FIS part can be represented as seen in Figure 14.
Figure 13. Example of membership function with Gauss MF (three MFs).
Figure 14. Fuzzy Inference System (FIS) part.
18
Links between Subjective Assessments and Objective Metrics for Steering
However, due to more complicated structure of ANFIS as seen in Figure 9 when using multiinput, there will be hundreds of unknown parameters needed to be inferred, including linear
and nonlinear ones. It turns out that no convincing results can be acquired unless a huge
database is provided. Thus, the ANFIS tool could be another option in the future when
sufficient data set is given.
19
20
Links between Subjective Assessments and Objective Metrics for Steering
3 Parameter Selection and Matching
In this chapter all raw data will be looked through and preprocessed. Some
parameters in the raw datasheets are going to be excluded since they are
either lacking data samples or not relevant to this thesis. Explanatory
subjective assessments will be initially mapped to corresponding objective
measures before linear and nonlinear analysis.
Raw objective data from VCC includes four different vehicle classes and subjective data are
also made up of same classes. Each test group includes several drivers, but the drivers do not
remain the same across different tests (e.g., Driver 3 in C-class is not the same one in Dclass). The focus of this thesis is steering characteristics. In addition, some of important data
is lacking for drivers or vehicles. For example the testing data is missing for majority of
vehicles regarding certain objective measures that will not be considered in analysis. So the
work of reorganizing and matching the raw data is necessary.
The classification can be seen in Table 3. Further analysis will be based on each vehicle
class. One important aspect that has to be kept in mind is that in some classes not all test
drivers did their judgment on every subjective assessment and each vehicle model. That is,
the sum of drivers is not consistent with total runs of each vehicle.
Table 3. Overview of test vehicles in each class.
Classification
Vehicle 1 Vehicle 2 Vehicle 3 Vehicle 4 Vehicle 5 Vehicle 6 Vehicle 7
Sum of
vehicles
C-class
C1
C2
C3
C4
C5
C6.
-
6
D-class
D1
D2
D3
D4
D5
-
-
5
E-class
E1
E2
E3
E4
E5
E6
E7
7
SUV
SUV1
SUV2
SUV3
SUV4
SUV5
-
-
5
3.1 Selection of Subjective Assessments
The emphasis in this thesis work should be focused on the vehicle steering characteristics.
Twelve subjective assessment indexes shown in Table 4 will be studied. However, the data
under Level 2 (steering disturbances/error states) is not taken into account because there is no
corresponding objective data available. Thus, these subjective assessments are not considered
while calculating the rating for Level 1 (steering).
21
Level 1
Steering
Table 4. Selected subjective assessments.
Level 2
Level 3
First 50m Test
First 50m test
Efforts
Park/Maneuvering
Returnability
Response
Roll Control
Straight Ahead Controllability
Torque Feedback
Modulation
Response
Roll Control
Cornering Controllability
Torque Feedback
Returnability
Modulation
Assessment
SA-1
SA-2
SA-3
SA-4
SA-5
SA-6
SA-7
SA-8
SA-9
SA-10
SA-11
SA-12
3.2 Objective Measures Used in Analysis
First of all, if there is lack of data in some objective measures, they have to be eliminated in
order to keep the uniformity on objective data. After that the objective measures that have
little effect on subjective assessments of drivers are allowed to be removed, such as (lat. acc.
resp. gain @ max load) and (sine time lag @max load). The reason is that the case with
maximum load was never achieved at the subjective data collection scenarios. With these
aforementioned work done, the standardized objective measures can be utilized in the
following analysis. Totally, 27 categorized parameters are shown in Table 5.
Table 5. Overview of all needed objective measures.
Straight Ahead Controllability
Level 2 Level 3
22
Level 4
Window
SWA At 0.05 g
Response Gain Straight Path
On Center Yaw gain straight
Lateral Acc. Resp. Gain
Overall Steering Sensitivity
Response
Lateral Acc. Resp. Gain
Overall Steering Sensitivity
Gain Linearity
Steering Sensitivity Ratio
Response Time Delay
Yaw 45° Phase Lag Time
Roll Control Straight Path
Roll
Control Total Rollrate Gradient @ 1 Hz
Torque Deadband
SWA at 1.3 Nm
Torque
Feedback Torque Build Up
Torsional Rate
Unit
[]
Measure
OM-1
[⁄ ⁄
] OM-2
[ ⁄
] OM-3
[ ⁄
] OM-4
[
[ ]
OM-5
[
OM-6
]
[ ⁄ ⁄ ]
OM-7
[]
OM-8
⁄
] OM-9
50m test (first
impression)
Cornering Controllability
Links between Subjective Assessments and Objective Metrics for Steering
Friction Feel
Torque at 0 g
Yaw Response Gain
Off Center Yaw Gain
Response Gain Understeer
Linear Range Understeer Gradient
Response Gain Linearity
Yaw Gain Linearity
Rel. yaw [email protected] lat. Acc.
Response
Yaw [email protected] lat acc/max yaw gain
Sine Time Lag
Yaw - SWA phase time lag @ 4m/s2
Sine Time Lag
Ay - SWA phase time lag @ 4m/s2
Sine Time Lag
Ay - Yaw phase time lag @ 4m/s2
Roll Control Cornering
Roll
Control Total Roll Gain
Torque Buildup Into The Corner
Torsional Rate Cornering
Torque Buildup Cornering
Off Center Torque Gradient
On Center Hysteresis
Torque
Feedback Torque Deadband in Degrees
Off Center Hysteresis
Torque hysteresis @ 0.3 g
Effort Level
Torque @ 0.3 g
Low Speed Response Gain
On Center Yaw Gain
Low Speed Torque Buildup
Max. Torsional Rate
Parking Efforts Standstill
Parking Efforts Near Center
Parking Efforts Rolling
Parking Efforts Just Off Center
[
]
[⁄ ⁄
] OM-11
[⁄ ]
OM-12
[ ]
OM-13
[⁄ ⁄
[
OM-10
] OM-14
[
]
OM-15
[
]
OM-16
[
]
OM-17
[⁄ ]
OM-18
⁄
[
] OM-19
⁄ ]
[]
OM-20
OM-21
[
]
OM-22
[
]
OM-23
[⁄ ⁄
] OM-24
[
] OM-25
⁄
[
]
OM-26
[
]
OM-27
3.3 Initial Parameter Matching
Having decided the subjective and objective parameters to be studied, corresponding group
mapping are set before conducting the analysis. Not all of the objective inputs should be
studied regarding a specific subjective assessment, since only those parameters that are
measured under the same test environment should be considered. For example the SA-4,
response about straight-ahead controllability, the corresponding objective data should come
from OM-1~OM-6, which are listed in Table 5.
Besides, subjective ratings on Level 2 are actually calculated by using the data from Level 3
and expressed in the form of the average value of them. In other words, these values in Level
2 do not directly depend on objective measures. Instead, they are merely decided by the
subjective judgment at next level (Level 3). That is why the subjective assessments on Level
2 are not included in Table 6.
Table 6. Corresponding setting for each subjective assessment.
SA
Subjective Level 3
1
First 50m test
Objective Level 4
Low Speed Response Gain [°/s/100°SWA]
Low Speed Torque Buildup [Nm/100°SWA]
OM
24
25
23
24
2
Efforts
3
Returnability
4
Response
5
Roll Control
6
Torque Feedback
7
Modulation
8
Response
9
Roll Control
10
Torque Feedback
11
Returnability
12
Modulation
Parking Efforts Standstill [Nm]
Parking Efforts Rolling [Nm]
Parking Efforts Standstill [Nm]
Parking Efforts Rolling [Nm]
Window [deg]
Response Gain Straight Path [°/s/100°SWA]
Lateral Acc. Resp. Gain [°/s/100°SWA]
Lateral Acc. Resp. Gain [°/s/100°SWA]
Gain Linearity [-]
Response Time Delay [ms]
Roll Control Straight Path [°/s/g]
Torque Deadband [°]
Torque Build Up [Nm/100°SWA]
Friction Feel [Nm]
Window [deg]
Response Gain Straight Path [°/s/100°SWA]
Lateral Acc. Resp. Gain [°/s/100°SWA]
Lateral Acc. Resp. Gain [°/s/100°SWA]
Gain Linearity [-]
Response Time Delay [ms]
Torque Deadband [°]
Torque Build Up [Nm/100°SWA]
Friction Feel [Nm]
Yaw Response Gain [°/s/100°SWA]
Response Gain Understeer [°/g]
Response Gain Linearity [%]
Rel. yaw [email protected] lat. Acc.
Sine Time Lag Yaw-SWA [ms]
Sine Time Lag Ay-SWA [ms]
Sine Time Lag Ay-Yaw [ms]
Roll Control Cornering [deg/g]
Torque Buildup Into The Corner [Nm/100°SWA]
Torque Buildup Cornering [Nm/g]
On Center Hysteresis [°]
Off Center Hysteresis [Nm]
Effort Level [Nm]
Torque Buildup Into The Corner [Nm/100°SWA]
Torque Buildup Cornering [Nm/g]
On Center Hysteresis [°]
Off Center Hysteresis [Nm]
Effort Level [Nm]
Yaw Response Gain [°/s/100°SWA]
Response Gain Understeer [°/g]
Response Gain Linearity [%]
Rel. yaw [email protected] lat. Acc.
Sine Time Lag Yaw-SWA [ms]
Sine Time Lag Ay-SWA [ms]
Sine Time Lag Ay-Yaw [ms]
Torque Buildup Into The Corner [Nm/100°SWA]
Torque Buildup Cornering [Nm/g]
On Center Hysteresis [°]
Off Center Hysteresis [Nm]
Effort Level [Nm]
26
27
26
27
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
19
20
21
22
23
11
12
13
14
15
16
17
19
20
21
22
23
Links between Subjective Assessments and Objective Metrics for Steering
4 Mathematical Modeling
In this chapter the bicycle model as well as some time trajectories of
steering wheel angle, yaw rate, lateral acceleration and steering wheel
torque will be introduced. These models and parameters set the basis of
vehicle dynamics system. It is necessary to clarify that the model-based
analysis is not used to calculate the value of a specific objective parameter.
The goal of the mathematical modeling is to help understand and explain
the data and findings of this thesis work.
4.1 Representation of Simple Vehicle Model
To build and analyze an ideal simple vehicle model, the basic assumption made about the
vehicle and its driving environment is adopted.




The height of the center of gravity over ground is zero.
The only external force affecting the motion of vehicle is the tire force and the side force
is linear functions of the slip angles.
The slip angles are much smaller than 180°.
There is constant speed at stationary driving.
Figure 15. Bicycle model (single track model).
25
The most simplified vehicle model, bicycle model or also called single track model, is a
three-degree-of-freedom model as seen in Figure 15. It only represents the horizontal
motions in the X-Y plane. The matrix form is generally used to express the equations of
motion as seen in Equation (11).
( ̇)
(
)
(11).
(
)
If the driving maneuver is considered to be a simple stationary case, the definition for
steering sensitivity
̇
and understeer
can be expressed as in Equation (14) and (15):
̈
̇
(12).
̇
(13).
̇
(14).
(15).
In stationary driving, the definition of yaw response gain is similar to steering sensitivity, and
with Equation (13), the expression for overall steering sensitivity
is given in Equation
(16). The way to transfer understeer to response gain understeer
is shown as Equation
(17).
̇
[
]
(16).
(17).
These two expressions show that theoretically, overall steering sensitivity and response gain
understeer should correspondingly follow steering sensitivity and understeer.
4.2 Parameters with Time Trajectories
All the objective metrics can be divided into two types, instant values or time trajectories. In
this thesis, time trajectories are used to show and understand the time lag, deadband or
26
Links between Subjective Assessments and Objective Metrics for Steering
hysteresis. van Daal [14] introduces all time trajectories regarding friction and compliance in
steering system. Examples of these characteristics can be seen in Figure 16 to 18.
Figure 16. Time trajectories of steering wheel angle and lateral acceleration.
Figure 17. Time trajectories, lateral acceleration versus steering wheel torque.
27
Figure 18. Time trajectories, steering wheel angle versus steering wheel torque.
In Figure 16, there is a time lag between the two time trajectories. Because vehicles need to
take a short time period to overcome the inertia and compliance of steering system,
compliance of suspension and tire relaxation delay. The vehicles cannot respond immediately
with a steering wheel input, and this is also referred to as response time lag.
In Figure 17, friction feel indicates the needed torque to initiate turning when driving at
straight path, while off-center hysteresis shows the needed effort to correct the steering in
cornering.
In Figure 18, on-center hysteresis indicates when drivers will feel resistance torque when
changing steering wheel direction and passing the center region. Torque deadband shows the
feeling of play in steering.
28
Links between Subjective Assessments and Objective Metrics for Steering
5 Evaluation of Acquired Data
In this chapter the goal is to examine the objective-objective correlations
and the validity of subjective data. The results could be used to eliminate
some unnecessary regressors in the correlation analysis.
5.1 Correlation between Objective Measures
The purpose of correlation coefficient analysis is to find correlations between two objective
measures (Level 4) that are under the same level (Level 3). These correlations can be either
positive or negative. By doing this with all vehicles, some similar or different tendencies
regarding correlation between different classes are supposed to be attained. Such information
can be used to take decisions on whether to disregard some objective measures in the initial
studies.
The example of C-class is shown in Figure 19. The correlation coefficient between any two
different objective measures is calculated. After that, all absolute values larger than 0.7 are
shaded dark in the grid. The statistics for the rest of the vehicle classes are tabulated in
Appendix B.
R-value of Obj vs Obj
Objective measures
1
2
3
4
5
6
7
8
Objective measures
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Figure 19. Correlation coefficient between two objective measures for C-class (| |
shaded).
is
29
Table 7. Overview of correlations between two objective measures (negative correlations are
shaded in gray).
Test
Straight
1-4
2-3
2-4
Correlated 3-4
parameter 9-10
C-class
Corner 50m
11-14 26-27
12-14
16-17
20-22
D-class
Straight Corner 50m
1-2
11-12 25-27
1-4
12-14
2-3
19-20
2-4
19-22
3-4
20-22
8-9
20-23
8-10
22-23
9-10
E-class
Straight Corner 50m
1-2
11-12 25-26
1-3
11-14 25-27
1-4
16-17
2-3
19-22
2-4
19-23
3-4
SUV
Straight Corner 50m
1-5
16-17 26-27
2-4
19-21
2-5
19-23
4-5
22-23
8-10
9-10
By observing Table 7, where all found objective-objective correlations are listed by different
testing type. In the test of driving straight-ahead, for all non-SUV classes, OM-1~OM-4 are
cross-correlated, except for that correlation between OM-1 and 3 is not that strong (from 0.78 to -0.37). Moreover, OM-1 (window) only shows negative correlation with three other
parameters.
In cornering test, OM-11, 12 and 14 tend to correlate with each other while no unequivocal
sign is confirmed except that OM-11 always reveals negative correlation with OM-12, and
still SUV class does not display such properties.
In all non-SUV classes except OM-21 all torque feedback related parameters (OM-19, 20, 22
and 23) somehow show positive correlation with each other, especially between OM-19 and
OM-23 as well as between OM-22 and OM-23, which can also been found in SUV.
Across all four segments correlation between time lag 16 and 17 is nearly confirmed,
although the r-value is 0.66 in the D-class. In addition at 50m test OM-27 is either correlated
with OM-25 (in C and E classes) or with OM-26 (in B and SUV classes).
The conclusions that can be drawn from the analysis in this section are:



30
For all sedans, yaw response gain and lateral acceleration response gain on the straight
path have positive correlation because of the relationship as seen is Equation (13) and
(16) according to the bicycle model.
Window indicates the sensitivity of steering wheel before the turning can be felt. So the
lower value of this parameter will give higher sensitivity (nervous property), which leads
to faster response of both yaw gain and lateral acceleration. That is why the negative
correlation is acquired between window and the rest of the three parameters.
The negative correlation between off-center yaw gain and response gain understeer is
understandable. Since response gain understeer almost follows the change of understeer
as seen in Equation (17) the yaw rate will grow more slowly if the car is more
understeered.
Links between Subjective Assessments and Objective Metrics for Steering

Among three kinds of time lags there should be a relationship as in Equation (18). In
normal driving the slip force on tires is produced as long as a steering maneuver is
transferred to tires, so the yaw motion is present. It will be followed by the centripetal
force (lateral acceleration). In this thesis, only sine time lag Ay-SWA and Ay-Yaw give
positive correlation. If a close examination is taken into these three objective measures
the time lag Yaw-SWA is mostly located around 70ms to 80ms while the lag Ay-Yaw
varies from 20ms to 100ms. This means the main difference of time lag Ay-SWA results
from lag Ay-Yaw, thus they are correlated.
(18).


The most correlations are found in torque feedback in cornering. It is easy to understand
that torque buildup and torque at 0.3g-lateral acceleration are correlated. Besides,
according to Figure 5, larger off-center hysteresis also increases torque gradient and
needed effort in the curve. On the other hand on-center hysteresis is in closer association
with the on-center properties with low lateral acceleration so as not to have clear
correlation with other parameters in torque feedback in cornering.
For all vehicles during parking the needed torque at rolling speed should have
correlation with either low speed torque buildup or the steering torque in the state of
standstill. That means at least one of these two metrics will decide the steering torque at
rolling.
Comparison with other research:
The Table 5 in [5] shows the confirmed correlation between various handling parameters. In
that research, the characteristics that differed in each simulation test are up to four in the
steering system, including friction, damping, stiffness and inertia. Except for these variables,
the rest of the vehicle properties in the simulator remain completely the same. It can be sure
that all differences in test results are due to these varied parameters. If there are certain
relationships between two objective measures, the specific correlations should be fixed. As a
result several correlations are discovered and the correlation coefficients are high (from 0.95
to 1.00) enough to do reduction, even though they come from different tests. That means
these metrics nearly follow the same pattern and it does not matter which one should be
eliminated.
Back to this research, test objects are replaced by real vehicle models that possess totally
distinct parameters, rather than only several varied parameters. In other words, even if all
cars follow the same driving maneuver, the way they behave and their specific correlations
between objective measures will be absolutely different. It turns out that few correlations
with high coefficients (close to 1.0) could be found in this thesis. Those correlated objective
measures with relatively low | |-value (not close to 1.0) cannot guarantee fixed change
tendency between them. Besides, results in this thesis are based on real vehicle tests instead
of simulation platform. So the setting of objective measurement is not close to that in [5].
31
When the correlation coefficient is higher than the critical value 0.7 but not sufficiently close
to 1 it is not so convincing to decide which one should be left in further analysis. So based on
the comparison, these found correlations can only be used to improve the objective metrics
instead of to remove those correlated metrics in regression process.
The improvement of such objective measure should be focusing on the metrics with some
findings (OM-1, 2, 3, 4, 11, 12, 16, 17, 19, 20, 22, 23, 25, 26 and 27). By doing tests with
different configurations on one single fixed vehicle model and if some higher correlation
coefficients are acquired it will be convincing to remove any one of the correlated measures
to save the efforts and time spent on tests.
5.2 Evaluation of Subjective Assessments
Correlation analysis of subjective data cannot be carried out like the objective data. The main
reason is that there are not enough subjective data used to conduct regression analysis for
each driver. For instance, 5 out of 9 drivers in E-class only tested 3 or 4 out of 7 cars in total.
Obviously, the findings of possible internal correlation between subjective assessments based
on inadequate data probably lead us to draw the wrong conclusions. From this perspective,
those subjective data from test drivers who only provided 3 or 4 ratings in each vehicle class
will be abandoned when looking for subjective-objective correlation. These insufficient data
sets are listed and shaded in Table 8.
Table 8. Test vehicle by each driver in every vehicle class.
Vehicle Number (different models in different vehicle classes)
Class Driver No.
Vehicle 1 Vehicle 2 Vehicle 3 Vehicle 4 Vehicle 5 Vehicle 6 Vehicle 7
Driver 1
×
×
×
Driver 2
×
×
×
Driver 3
×
×
×
×
×
×
Driver 4
×
×
×
×
×
×
Driver 5
×
×
×
×
×
×
C
Driver 6*
×
×
×
×
×
×
Driver 7
×
×
×
Driver 8
×
×
×
Driver 9
×
×
×
×
×
×
Driver 10
×
×
×
Driver 1
×
×
×
×
×
Driver 2
×
×
×
×
×
Driver 3
×
×
×
×
×
D
Driver 4
×
×
×
×
×
Driver 5
×
×
×
×
×
Driver 6
×
×
×
×
×
Driver 7
×
×
×
×
×
Driver 1
×
×
×
Driver 2
×
×
×
×
×
×
×
E
Driver 3
×
×
×
Driver 4
×
×
×
×
×
×
×
32
Links between Subjective Assessments and Objective Metrics for Steering
Driver 5
Driver 6
Driver 7
Driver 8
Driver 9*
Driver 1
Driver 2
Driver 3
SUV
Driver 4
Driver 5
Driver 6
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
In this thesis the mean ratings and their 95% confidence intervals for 12 subjective
assessments can be used to roughly examine the validity of the subjective ratings. To be
concrete, the interval with 95% confidence level should be within the subjective rating range,
which is from 1 to 10.
In every vehicle class, there are several ratings from six to ten test drivers on each subjective
assessment. Take the subjective ratings in D-class as an example. There are seven test drivers
who did subjective evaluation for all the five vehicles in this class. As shown in Figure 20 it
seems that the mean ratings for each car scatter around a reference value, roughly 7-point for
the assessments of D1. In this case, the spread of the confidence intervals all fall in the
subjective rating range. Thus, the subjective data can be considered as basically valid for
further analysis. However, there is no obvious rating pattern of the drivers found according to
this figure. It is impossible to profile preference of drivers in subjective rating based on a
small number of tests.
33
7-Drivers
7-Drivers
10
7-Drivers
10
7-Drivers
10
7-Drivers
Mean values and 95% Confidence Interval of Ratings
10
10
8
Vehicle
D1
6
4
1
2
3
4
5
6
7
8
9
10
11
12
8
Vehicle
D2
6
4
1
2
3
4
5
6
7
8
9
10
11
12
8
Vehicle
D3
6
4
1
2
3
4
5
6
7
8
9
10
11
12
8
Vehicle
D4
6
4
1
2
3
4
5
6
7
8
9
10
11
12
8
Vehicle
D5
6
4
1
2
3
4
5
6
7
8
9
10
11
12
Assessment Number
Figure 20. Mean value and 95% confidence interval of ratings for D-class.
To investigate the validity of ratings, the rating spread of drivers on a specific vehicle should
be examined. Figure 21 illustrates ratings of each driver on the vehicles in D-class. Each
shaded rectangle shows rating range of a driver on a specific vehicle, where the top line
indicates the highest rating while the bottom line stands for the lowest rating. For instance,
Driver 1 has a larger rating difference on every vehicle, while Driver 6 has a rather narrower
rating range compared with other drivers. Whereas, this small rating tendency still can be
accepted. One reason could be that these drivers show consistent rating habit across all the
vehicle models. If the rating spread of a driver fluctuates a lot with the given vehicles within
the same segment, the ratings of him or her cannot be considered reliable.
34
Links between Subjective Assessments and Objective Metrics for Steering
Subjective Ratings of Every Vehicle Model
10
8
7.8
8.0
1
2
7.0
7.0
3
4
7.2
6
4
5
7.3
6
7.7
Vehicle
D1
7
10
8
7.7
7.7273
8.2
7.8
8
8.0
7.1
7.0833
7.6
8
8.0
Vehicle
D2
Subjective Ratings
6
4
1
2
3
4
5
6
7
10
8
8.4
8.0
8.3
8.0
8.0
8.0
8.4
Vehicle
D3
6
4
1
2
8.3
8.0
3
4
5
6
7
10
8
7.7
6.8
6
4
1
2
3
6.9
6.9091
7.8
7.0
4
7.3
5
6.9
6
7.9
Vehicle
D4
7
10
8
7.4
4
6.7
6.5
6
1
2
3
4
5
6
7.2
Vehicle
D5
7
Driver Number
Figure 21. Comparison of rating range and mean value on each car in D-class.
5.3 Evaluation of Rating Tendency
The purpose of investigating rating tendency is to get rid of outliers and reduce the
unintended effects caused by individual preference. Seven expert drivers participated in the
driving tests and made subjective evaluation on five vehicles in D-class. Expert drivers with
experience and special competence in subjective evaluation can be expected to consistently
provide better subjective evaluation than novice drivers, since novice drivers need more
mental concentration on the vehicle control during tests rather than the judgment of steering
feel itself [15].
However, expert drivers may also have their own rating tendencies. Some of them are likely
to always rate higher or lower than other drivers, or in other cases some of the drivers
possibly invariably have wider or narrower rating range than the rest. Besides, their rating is
more likely affected by their prejudgment of the technical settings of a test vehicle other than
their actual feel because of their experience.
35
There are five vehicles in D-class so each driver gave five ratings on each subjective
assessment. The maximum, minimum and mean values of subjective evaluation of every
driver are investigated. The way used to find out preferences is basically by comparing these
values between different drivers. The results are graphically illustrated in Figure 22.
SA-2:efforts(parking)
9
9
9
8
8
8
7
7
6
6
5
5
5
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
SA-5:roll control(straight ahead)
SA-6:torque feedback(straight ahead)
10
10
9
9
9
8
8
8
7
Rating
10
Rating
7
7
6
6
6
5
5
5
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
SA-7:modulation(straight ahead)
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
SA-8:response(cornering)
SA-9:roll control(cornering)
10
9
9
9
8
8
8
7
Rating
10
Rating
10
7
7
6
6
6
5
5
5
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
SA-10:torque feedback(cornering)
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
SA-11:returnability(cornering)
SA-12:modulation(cornering)
10
10
9
9
9
8
8
8
Rating
10
7
7
Rating
Rating
7
6
SA-4:response(straight ahead)
Rating
Rating
10
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
Rating
SA-3:returnability(parking)
10
Rating
Rating
SA-1:first 50 test
10
7
6
6
6
5
5
5
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
1 2 3 4 5 6 7 m m1 m4 m14
Driver number and cases omiting some of the drivers
Figure 22. Evaluation of rating based on each subjective assessment.
36
Links between Subjective Assessments and Objective Metrics for Steering
Studying the figure above, each of the 12 subplots presents a subjective assessment with
ratings of seven drivers. The first seven bars shaded in yellow show the rating ranges of
seven drivers. The upper and bottom lines represent the maximum ratings and minimum
ratings respectively. The dot-dash lines represent the mean values of ratings. By examining
their subjective ratings in the figure closely it is easy to find that some of the drivers show
obvious rating tendency. For example, Driver 4 always rated lower in most of the subjective
assessment. Likewise, some other drivers also plausibly show their preferences but in
different patterns. After close observation, four drivers show possible rating tendency and
they are listed in Table 9.
Table 9. Evaluation of rating on each subjective assessment.
H – highest average rating
W – widest rating spread
Notation
L – lowest average rating
N – narrowest rating spread
Driver 1 Driver 2 Driver 3 Driver 4 Driver 5 Driver 6 Driver 7
SA-1
First 50m
SA-2
and
Parking (low speed)
SA-3
SA-4
L
W
H
N
N
SA-5
H
H
H
L
H
L
H
L
H
L
Straight-ahead
SA-6
W
SA-7
Cornering
SA-8
W
SA-9
W
L
SA-10
W
L
SA-11
W
N
L
6 W,4 H
2N
8L
SA-12
Summary
3H
Before trying to profile the rating tendency of drivers it is necessary to clarify the last 4 bars
shaded red in each subplot in Figure 22. The first bar named as ‘m’. The upper line of m
means the mean value of the maximum ratings of all the drivers, while the bottom line of m
is the mean value of the minimum ratings of all the drivers. Also, the mean value of m
marked with a cross is average of the mean ratings of all the drivers. Thus, the bar is mainly
used to show the average level of subjective ratings from 7 drivers. Similarly, the following 3
bars indicate the same meaning. The only difference is that bar m1 excluded the data from
37
Driver 1, bar m4 excluded the ratings from Driver 4 and bar m14 excluded the data from both
Driver 1 and Driver 4.
Driver 1:
As shown in Table 9, Driver 1 has the widest rating spread in 6 out of 12 subjective
assessments. For example, regarding response of cornering controllability rating spread is 3
while most of the rest rated within the range of 1.5. Furthermore, wide rating can also be
found in Driver 1 subjective evaluation such as 6~10 for parking efforts.
Driver 1 also shows a different tendency in the straight-ahead tests. Mean values of
subjective ratings are always higher than other test drivers when considering response, roll
control, torque feedback and modulation in straight-ahead test. After excluding the data from
Driver 1, the bar m1 moves down when comparing with the bar m. In other words, the data
from Driver 1 caused a higher rating level generally in evaluation of straight-ahead
controllability. In addition, the rating spread can be narrowed down in average if neglecting
the data from Driver 1.
Driver 2:
In general, no specific rating tendency can be found from subjective evaluation of Driver 2.
However, it is interesting to find a possible preference when considering returnability. Driver
2 rated returnability of the five vehicles the same in either straight-ahead tests or cornering
tests. It seems that Driver 2 is not sensitive to the difference of subjective feel of returnability
between different vehicles comparing with other test drivers.
Driver 4:
According to Table 9, Driver 4 rated most of the subjective assessments lower than others.
This tendency is clearer in the straight-ahead and cornering tests. If excluding this data, like
the method used to analyze the data of Driver 1 the mean values (bar m4) moves higher in
most of the subjective assessments when comparing with bar m. That is, data of Driver 4
contributes to lowering subjective ratings level of the whole data set.
Driver 7:
There is no obvious preference for the subjective evaluation of Driver 7. Only regarding to
the first three subjective assessments, Driver 7 might tend to give higher grades in the lowspeed tests. Nevertheless, this kind of tendency did not repeat in the following 9 subjective
assessments.
In short, Driver 1 and Driver 4 show clear rating tendencies after analyzing their data. Driver
2 and 7 reveal potential tendency in some specific subjective assessments or test scenarios.
38
Links between Subjective Assessments and Objective Metrics for Steering
For the rest of the drivers, no obvious pattern can be found. To further analyze the subjective
rating, statistical spread analysis can be applied that will be performed in two steps [16].
First, normalize subjective rating due to driver different judgment scale. Secondly, a criterion
of driver exclusion need to be applied to find out if the rating of one driver is particularly far
away from the average rating level with respect to a specific assessment. To carry out such
kind of work, a reference vehicle is the prerequisite that is not the same case in this thesis.
39
40
Links between Subjective Assessments and Objective Metrics for Steering
6 Linear Regression Analysis of Subjective and Objective Data
In this chapter subjective and objective data will be brought together so as
to identify those assessments where subjective-objective linear correlation
can be discovered. These subjective assessments are considered to reveal
direct and clear effects of correlated objective measures.
Ideally, simple linear regression equation is the easiest way to express the simple positive or
negative one-to-one subjective-objective correlation. While under the real test environment
considering the different test vehicles and subjective driving preferences it is not likely to
find that many simple linear correlations in the general way. That is why both simple and
multiple regression are used to uncover as many linear relationships for each subjective
assessment as possible.
6.1 Results from Simple Linear Regression
Simple linear regression is one of the simplest in statistics and attempts to explore the
relationship between two variables using a straight line. R-value of all paired subjectiveobjective parameters has been investigated. The way to find out possible correlations is case
study, which is based on subjective ratings of every driver in each vehicle class. Taking
Driver 3 in class as an example, he totally tested 5 vehicles in this car segment. A correlation
between SA-2 and OM-26 can be found according to his subjective data. As shown in Figure
23, a correlation with the value of | | above 0.7 is revealed. That is, there is a possible
correlation between the parking effort feel and the mean absolute value of steering wheel
torque when turning 180° left and right can be discovered.
41
R-value of Obj vs Sub (Driver 3 in D-class with 5Vs)
Objective mearsures
1
2
3
4
5
Subjective ratings
6
7
8
9
10
11
12
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Figure 23. Result of simple regression of one driver in D-class (correlations with| |
| |
shaded with dark blue, while
are shaded with light blue).
are
However, it is not so convincing to declare a correlation exists, that only depends on one
case. More findings of the same correlation in D-class are required to convince us to believe
that it exists. Considering that small sample size increase chance to reject a correlation, no
less than a half of total drivers show the same correlation can be persuasive to support its
existence. In D-class, there are seven test drivers, out of which six drivers show the
correlation between SA-2 and OM-26. This indicates that this correlation has high likelihood
of existing.
Analyzing the regression coefficients can help us to know if the relation between parking
effort and steering wheel torque is negative or positive. As seen in Table 10, the regression
coefficients are all negative when conducting case study on data of each driver in D-class,
except for Driver 6 for whom the correlation coefficient is not strong enough. Therefore, a
negative relation between SA-2 and OM-26 has been confirmed.
Table 10. Regression coefficients between parking efforts and steering wheel torque.
Driver
1
2
3
4
5
6
7
-3.19
-1.23
-1.69
-1.49
-2.48
/
-0.68
Coefficient ( )
By employing the same method stated above, subjective-objective correlation with | |
has also been found in other vehicle segments. A subjective-objective correlation is
42
Links between Subjective Assessments and Objective Metrics for Steering
Driver
supported only when data from no less than half of drivers demonstrate this correlation. The
regression coefficients are listed from Table 11 to Table 14, where the negative coefficients
are shaded.
Table 11. Regression coefficients ( ) if correlations (| |
SA
7
10
OM
1
2
4
21
23
21.15
1
0.08
5.77
2
1.41
3
4
1.11
-0.14
-2.89
0.15
5
6
-1.61
0.17
2.88
0.34
2.04
7
0.83
-0.09
-1.44
-0.96
8
-0.10
-2.15
0.10
9
1.49
-0.28
-7.14
10.58
10
) if correlations (| |
10
22
23
Table 13. Regression coefficients (
SA
2
4
OM
26
2
4
-2.32
0.36
7.50
1
-1.57
0.09
1.77
2
3
-1.03
0.15
2.58
4
-0.64
-0.04
-0.65
5
-1.54
0.33
6.73
6
0.08
1.29
7
0.17
8
9
) if correlations (| |
7
4
19
-0.89
Driver
Driver
Table 12. Regression coefficients (
SA
2
OM
26
-3.19
1
-1.23
2
-1.69
3
-1.49
4
-2.48
5
6
-0.68
7
0.33
1.443
1.12
1.672
1.16
0.96
0.84
1.284
0.14
3.09
-0.57
3.56
0.65
3.41
2.22
0.10
0.18
0.42
) found in C-class.
2
26
27
-2.05
-2.04
-1.35
-1.43
-0.51
-0.54
-0.43
-0.47
-0.75
-0.52
) found in D-class.
12
23
0.62
0.59
1.04
0.75
0.80
) found in E-class.
10
12
22
23
22
6.67
10.00
0.74
0.89
0.68
-13.33
0.77
1.61
0.74
0.59
-2.10
0.90
1.75
1.10
0.81
0.34
-0.36
1.25
43
Driver
Table 14. Regression coefficients ( ) if correlations (| |
SA
2
3
OM
26
27
26
27
-1.07
-0.91
1
-2.21
-1.76
-0.89
-0.71
2
-1.52
-1.21
-1.09
-0.86
3
-0.81
-0.63
4
-1.52
-1.21
5
-1.00
-2.28
-1.87
6
) found in SUV.
12
15
-0.11
-0.08
-0.04
Subjective-objective correlations found from simple linear regression analysis are
summarized in Table 15. Only five subjective assessments can be found to correlate with
objective measures. Two of the correlations can be seen in every vehicle segment, which are
quite convincing. The results are analyzed and discussed as follows.
Table 15. Subjective-objective Correlations (the negatives are shaded).
SA
2
3
4
7
10
12
26
27
1
2 4 21 23
C
26
22 23
23
D
OM
26
2 4
4
19 22 23 22
E
26
27
26
27
15
SUV
Interpretation of results:



44
SA-2 vs. OM-26 and 27
 OM-26 and 27 are the mean absolute value of steering wheel torque when turning
180° left and right. The only difference between them is the measurement condition
that one is measured standstill while another is measured at the speed of 7 km/h.
They correlated with each other, so only OM-26 actually has correlation with SA-2.
 Negative correlation between SA-2 and OM-26 indicates that drivers prefer a lighter
steering wheel torque during parking. This preference is revealed in all vehicle
classes.
SA-3 vs. OM-26 and 27
 The explanation of negative correlation between .OM-26 or 27 and SA3 is similar to
the previous interpretation. However, the negative effects of these parking efforts
are only found under SUV.
SA-4 vs. OM-2 and 4
 OM-2 and 4 are correlated with each other.
 The positive correlation between yaw/lateral acceleration response gain and
response of straight-ahead controllability implies that a larger on-center steering
sensitivity when driving at 0.2g and 120km/h leads to a higher subjective rating of
response.
Links between Subjective Assessments and Objective Metrics for Steering



SA-7 vs. OM-1, 2 and 4
 The OM-1, 2, 4 actually are correlated with each other. The OM- 1(window) has a
negative relationship with the other two parameters.
 In C-class, higher ratings of steering modulation or precision are achieved when
steering sensitivity is tuned smaller or window is set larger. However, a different
result is found in E-class that a bigger lateral acc. resp. gain will instead make
drivers give higher subjective ratings in steering modulation.
SA-10 vs. OM-19, 21, 22 and 23
 SA-10 is the torque feedback of cornering controllability. In D-class, OM-22 and 23
are correlated. Thus, only the OM-23 is needed to take into consideration when
conduction regression analysis in D-class. In E-class, the correlations found are
based on inadequate data; hence the results are not reliable. OM-23 is named after
effort level, meaning that the steering torque needed at 0.3 g and 75 km/h of steady
state.
 The correlation between torque feedback and effort level (OM-23) has been found in
every vehicle class. The positive correlation implies that the drivers expect a high
torque feedback when cornering at a high lateral acceleration.
 On-center hysteresis (OM-21) has been found to be an important parameter in
defining torque feedback at low-speed cornering test in C-class. A larger torque
deadband gave a higher subjective rating of torque feedback.
SA-12 vs. OM-15, 22 and 23
 It is found that modulation in cornering had different relationships with objective
metrics in three vehicle classes. In SUV segment, modulation negatively correlated
with sine time lag Yaw-SWA. A larger time lag most likely caused drivers to over
compensate steering wheel angle during cornering, which led to bad modulation. In
D and E-class, modulation has a positive correlation with effort level and off-center
hysteresis respectively.
6.2 Results from Multiple Linear Regression
The process of correlating subjective ratings and multiple objective inputs is to match most
important objective measures by ordinary least square regression.
6.2.1 Regressor Elimination Process
In the previous chapters, the needed objective metrics are listed corresponding to each
specific assessment. However, a great problem is highlighted regarding to some assessments
that if all available objective measures are included to build a regression equation, the
number of regressors will be larger even than that of subjective data set, which is the same as
the number of test vehicles in our data set. This fact means that it is impossible to check the
statistic significance of the regression equation and the regressors. As seen in Equation (19),
some elimination work has to be carried out until the regressors are less than the number of
vehicles
in a test class. Besides, some of the metrics represent multicollinearity which
45
might make it hard to define whether the contribution to vehicle response of each regressor is
true or not.
(19).
[




]
is subjective rating of a specific test group for various vehicles.
There are sets of objective metrics for various vehicles.
is the constant in the regression equation.
is the coefficient matrix for regressors
.
The elimination process and principle is mentioned by Chen [4] and ridge plots are used to
identify suitable regressor sets. A new matrix
from the Equation (1) in 6.2.1 of [4] is
introduced giving an estimate of contribution of each regressor in the solution. Significant
regressors can be confirmed by being farther away from
than insignificant ones. Also
a sudden change of slope near around
indicates the multicollinearity, which means
certain regressors have strong relationships to one or more other inputs.
Take the case of Driver 3 in D-class test as example when studying subjective assessment.
Thanks to too many measures as independent inputs in this research, the first version of ridge
plot might be too complicated to make elimination as seen in Figure 24. Those regressors
(objective metrics) with lowest
are eliminated at the beginning so that the number of
regressors is smaller than data set. In most test groups, there are five or six car models
included which means only four or five objective measures are allowed in order to make
clear plot as seen in Figure 25 to decide regressors with multicollinearity. As shown in Figure
24, , and is taken out of the analysis result in Figure 25.
46
Links between Subjective Assessments and Objective Metrics for Steering
Regression coefficient  i
0.8
0
0.6
1
0.4
3
0.2
5
0
7
2
4
6
-0.2
-0.4
-0.6
-0.8
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
k
Figure 24. Ridge plot used in the elimination process (too many regressors as inputs).
1
0
1
0.9
3
Regression coefficient  i
0.8
6
7
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
k
Figure 25. Ridge plot used in the elimination process (regressors less then data set).
After reaching a sufficiently clear ridge plot, the rule of elimination is as follows:

Remove regressors that have slope lines, but lying close to zero, which indicate
insignificance.
47


Remove regressors that have lines exhibiting instability nearly when
effects of multicollinearity.
Leave a subset with at least two regressors for least squares regression.
, meaning the
6.2.2 Multiple Linear Regression
With suitable objective regressors decided the method of least squares is used to get linear
regression equation for each subjective assessment and each test group. In Equation (19),
is fixed for each test class but depends on the decision about regressors before. Rewrite this
equation with the part of residual as Equation (20).
(20).
[
]
Certainly, regression equation can be generated on each assessment and car segment. Several
statistics have to be calculated to determine the quality and validity of the equations.


The coefficient of determination R² reveals how well this regression model is likely to
predict the future outcomes, in this research if the R²-value is below 0.7 it is assumed to
have uncertain correlation.
The F-statistics interprets the statistical significance of regression equation and each
regressor coefficient to itself. Considering the given small dataset (the number of test
vehicles across all classes) 75% confidence level is adopted for F-test. Refer to the Fdistribution table (Appendix A), if calculated F-statistics value is larger than the critical
value with
the statistical significance is confirmed.
With both requests above fulfilled and if still more than three regressors are left the ones with
least statistical significance are allowed to be disregarded until at most three objective
measures remain.
6.2.3 Subjective Ratings vs. Objective Measures Organized by Assessment Number
The results of found subjective-objective correlation are shown in Table 16, and the detailed
regression equations for each assessment and each vehicle class are listed in Appendix C.
Table 16. Overview of assessments where correlation is found for each class.
Subjective assessment number
C-class
2
3
4
6
7
8
10
11
12
Driver 3
×
Driver 4
×
×
×
×
×
Driver 5
×
×
×
Driver 9
×
×
×
D-class
48
Links between Subjective Assessments and Objective Metrics for Steering
Driver 1
Driver 2
Driver 3
Driver 4
Driver 5
Driver 6
Driver 7
E-class
Driver 4
Driver 5
Driver 8
Driver 9
SUV
Driver 1
Driver 2
Driver 3
Driver 4
Driver 5
Driver 6
Frequency
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
1
1
3
×
×
4
9
6
×
×
9
×
8
×
9
All the assessments where at least one correlation is found include SA-2, 3, 4, 6, 7, 8, 10, 11
and 12. On the contrary, nothing is revealed regarding to SA-1, 5 and 9. Considering the total
number of drivers attending each test group with corresponding car segment those
assessments with most frequent correlation are shown in Table 17. Recall that, in the tests
with C and E-class vehicles not all drivers test every available vehicle in that class.
Especially for those drivers with less than four test configurations (test cars), their subjective
rating data is not valid for multiple regression analysis since such small sample is not able to
lead to a valid regression equation.
SA
6
7
8
10
11
12
Table 17. Best correlating assessment across all test classes.
Level 3
(Level 2)
Frequency
Torque Feedback
(Straight Ahead Stability)
4
Class B, C, E, SUV
Modulation
(Straight Ahead Stability)
9
Class B, C, E, SUV
Response
(Cornering Stability)
6
Class B, C, SUV
Torque Feedback
(Cornering Stability)
9
Class B, C, SUV
Returnability
(Cornering Stability)
8
Class B, E, SUV
Modulation
(Cornering Stability)
7
Class B, C, SUV
6.2.4 Best Correlating Assessments
Table 18 to 23 show the regression equations found for these best assessments. F-statistics
for each regressor is excluded to make the table concentrated on the regression coefficient
49
since the significance is already confirmed as seen in Appendix C and each row presents the
regressor coefficient for a given driver, vehicle class and assessment.
Table 18. All found correlation for SA-6 (negative regression coefficient is shaded in gray).
OM
SA-6
Driver
8
9
10
R^2 F-stat
6
0.07
6.58
0.96 39.80
C-class
2
-0.40 7.34 0.96 23.47
D-class
5
-0.03 -0.95 0.99 49.11
E-class
1
1.03 -0.04 2.36 1.00 55551.41
SUV
Table 19. All found correlation for SA-7 (negative regression coefficient is shaded in gray).
OM
SA-7
Driver
1
2
3
4
5
6
8
9
10
R^2 F-stat
3
-0.03
5.80 0.99 179.66
-4.32
-2.14
5.27 0.92 7.75
C-class 4
5
-3.01
-0.74
0.99 102.25
5
1.32
-10.7
0.06
1.00 2366.59
-2.61 -3.05 0.08
1.00 43323.71
D-class 6
7
4.64
-0.57
0.91 9.60
8
0.18
-0.59
0.96 36.21
E-class
9
0.09
-4.01
0.04
0.99 133.18
4
-0.09
0.19
0.98 42.35
SUV
Table 20. All found correlation for SA-8 (negative regression coefficient is shaded in gray).
OM
SA-8
Driver
11
12
13
14
15
16
17
R^2 F-stat
-0.20
0.04 0.83 7.46
C-class 9
1
0.16 -0.17
1.00 2717.99
2
0.75
0.02
0.95 20.21
D-class
4
1.50
0.05
0.95 20.21
6
0.05 0.02
0.97 33.13
5
-0.36
0.05
0.97 29.62
SUV
Table 21. All found correlation for SA-10 (negative regression coefficient is shaded in gray).
OM
SA-10
Driver
19
20
21
22
23
R^2 F-stat
4
1.00 6.93 0.89 7.83
-0.57
-1.82 4.35 0.95 11.70
C-class 5
9
-0.18
0.06
1.44 0.97 21.56
1
-1.27
3.90
0.87 6.49
-0.18
0.68
0.96 22.59
D-class 3
5
-0.06
0.18 1.00 1.00 2675.32
2
0.75
-1.60
0.99 40.58
5
-0.11
-0.32
1.00 3325.66
SUV
6
0.89
-2.37
1.00 495.10
50
Links between Subjective Assessments and Objective Metrics for Steering
Table 22. All found correlation for SA-11 (negative regression coefficient is shaded in gray).
OM
SA-11
Driver
19
20
21
22
23
R^2 F-stat
1
-0.65
-0.36 2.64
0.99 36.80
3
0.23
0.46 0.96 25.68
4
-0.16
0.62 0.98 63.99
D-class
6
-0.51 0.24
0.71 1.00 274.38
6
-0.25
-0.10
1.12 1.00 444.54
7
-0.36
1.35
0.96 22.59
4
-0.40 -0.17 0.89
0.93 13.04
E-class
8
-0.27 1.55
0.82 6.80
6
0.78
-1.91
0.99 33.91
SUV
Table 23. All found correlation for SA-12 (negative regression coefficient is shaded in gray).
OM
SA-12
Driver
11 12 13 14 15 16 17 20 21 22 23 R^2 F-stat
4
0.09
-0.04
3.98 0.99 40.37
C-class
5
-0.14
0.01 -0.07
0.98 42.12
1
-4.55
-0.06
1.42
1.00 137.63
3
-0.03 0.02 0.04
1.00 190.00
4
-0.08
0.69
0.99 101.39
D-class
5
-0.12
0.92 0.99 106.21
7
-0.04 0.04
0.78
1.00 132.75
4
-0.33 1.38
1.00 7914.13
SUV
6
6.67
-0.19
0.98 48.97
As mentioned in the previous section one of the reasons to do metrics elimination is to leave
suitable ones as regressors. However, even if the number of regressors in each equation is
limited the appearing connected metrics in the equation differ a lot depending on the given
driver especially for SA-7 and 12 where there are more objective metrics in the initial
parameter pool. As a result, despite of some relationships regarding to these two assessments
the correlated objective measures have very large change distribution and there is no
tendency about the sign of their coefficients.
On the other hand, for those assessments with relatively few initial measures, SA-6, 8, 10 and
11, the key correlated measures are easy to find. For example, in SA-11, OM-19, 21, 22 and
23 stand out quite much, and the sign of coefficients shows good agreement. In other words,
it demonstrates that a correlated objective measure has unequivocally either positive or
negative effect regardless of the given driver as long as there is no excess of initial
regressors. Even across different best assessments part of metrics has a fixed tendency to a
positive or negative sign. Close examination on each correlated measures is done later on.
Another conclusion based on these two groups of assessments is that some true and
significant correlation should be found if some of the initial parameters used for SA-7 and 12
51
can be cut. It is the same for the whole subjective assessment data set, it is improper to use all
objective measures as regressors unless there is larger data size (the number of test vehicles).
6.2.5 Interpretation of Results
Through the results of previous section the hypothesis could be verified that each driver has
his or her own unique manner or preference compared with other drivers. Given the
confidence interval for various vehicle-classes and the idea that drivers grade the steering
performance according to different response or metrics is reasonable.
The result of this verified hypothesis is that analyzing the average ratings of all test drivers
might lead to unreliable information; some unreal correlations might be discovered while
some existing ones might be ignored mistakenly. As shown in Table 24, for those found
correlations between objective metrics and average subjective assessments the corresponding
measure inputs differ a lot and show no consistency with individual driver study in the
previous section. As an example of SA-11, OM-20 and 22 are included in the equation but
the sign of OM-20 does not follow the tendency of majority of drivers. Due to the possibility
of causing wrong conclusion the method of individual case study is more appropriate than
adopting average ratings.
Table 24. All found correlations between objective metrics and average subjective assessments.
OM
2
6
8
Car
4 D-class 0.03 0.04
C-class
6
E-class
-0.27
7 C-class
8 D-class
C-class
10
D-class
11 SUV
D-class
12
SUV
SA
9
10
0.04 4.27
0.03
0.05 4.45
12
13
14
15
16
19
20
22
23
R^2
1.00
0.87
0.82
0.94
0.10 -0.04
0.04
0.99
-0.26
2.13 0.94
-0.42
1.37 0.33 1.00
0.18 -0.53
0.99
-0.05
0.63 0.99
4.29
-0.07
1.00
F-stat
486.25
10.32
8.89
25.42
25.85
23.62
837.61
45.28
117.16
283.72
6.2.6 Influence of Correlated Objective Metrics in Best Assessments
Back to the correlations in these best assessments, some conclusions on the effect of
objective metrics are drawn.

52
SA-6 torque feedback in straight-ahead controllability
Shown in Table 18, SA-6 mostly shows negative correlation with torque buildup (OM-9)
and positive with torque at 0g-lateral acceleration (OM-10).
 At the state with 0g or before the moment of starting turning, zero or too small
steering torque (OM-10) means no feedback from the vehicle and usually does not
give a good subjective perception.
Links between Subjective Assessments and Objective Metrics for Steering



On the other hand, the increase of steering wheel torque (OM-9) should not depend
on lateral acceleration. Otherwise, an excessively high torque, which might be out
of normal range, is needed with a sudden increased lateral acceleration.
SA-8 response in cornering controllability
SA-8 in Table 20 reveals a few correlations with OM-11, 12, 13 and 16. However, none
of them is as universal as that for SA-6. Thus, the subjective feeling of the response
during cornering probably depends on individual driving preference. Only D-class cars
show somewhat uniform links on OM-12 and 16.
 Response gain understeer (OM-12) is looked on as needed steer angle with
increasing lateral acceleration. For a better response feeling drivers of D-class do
not mind giving more excessive steer angle that means a better way to achieve
wanted turning status precisely.
 The other correlated parameter sine time lag Ay-SWA (OM-16) also interestingly
shows positive relation. This could be because the drivers are influenced by some
other properties of the car, such as roll and yaw motion. If the response depends too
much on the preference of a driver he or she may also rate it lower because the car
feels nervous, even though this is mainly an issue when driving straight or at small
lateral acceleration.
SA-10 torque feedback in cornering controllability
As seen in Table 21, sedans (mainly C and D-class) and SUVs exhibit different multiple
regression equations on SA-10.
 For all non-SUV cars, SA-10 shows negative correlation with OM-19, while
positive ones with OM-22 and 23.
 Torque buildup into the corner (OM-19) should be quite small. The explanation is
the same as in straight-ahead controllability (SA-6).
 The positive correlation between off-center hysteresis (OM-22) and effort level at
0.3g-lateral acceleration (OM-23), which has been clarified in the section 5.1,
encourages both of the two to have positive effect on this torque feedback feeling.
In the steady cornering larger steering torque or correction torque (off-center
hysteresis indicates the effort to correct steering) make sure drivers can get
enough torque feedback from the steering wheel.
 For SUVs, SA-10 has negative link with both OM-20 and 22.
 It is interesting that for SUVs they have more emphasis on low torque buildup in
cornering (OM-20) instead of torque buildup into cornering (OM-21). However,
all cars with higher torque buildup will not cause a likable torque feedback.
 The preference of off-center hysteresis (OM-22) is opposite to that for sedans. As
in this segment, the range for this parameter is concentrating on [1.5, 3.5] as seen
in Figure 26 (the SUV subplot), while for the non-SUV class most metrics locate
in [1.0, 2.0]. Thus the explanation for this conflict of signs is that there might be
an optimal vale (around 1.5 to 2.0), so when having smaller than this interval it is
good to have higher off-center hysteresis but when having larger than this interval
[1.5, 2.0] there is negative effect on subjective feeling. That is why for these
53
SUVs with growing off-center hysteresis the feeling of torque feedback gets
worse.
Figure 26. The range of off-center hysteresis (the upper one from D-class and the lower one
from SUV).


For E-class vehicles, no found correlation is based on two reasons. Too small data
size for the multiple regression analysis and that the ratings on this assessment
concentrate on some points makes difficult to produce regression results with high
confidence.
SA-11 returnability in cornering controllability
The initial regressor input for this assessment seen as Table 22 is same as the last one. It
is interesting to compare the significance of each metrics. Since not that many findings
come from the SUVs the comparison for sedans shows that the significant objective
measures (OM-19, 22 and 23) for returnability have the same negative or positive
property as for torque feedback (SA-10).
 Only one added measure here is on-center hysteresis (OM-21), which is the only
parameter deciding when the assistance force will disappear during turning the
steering wheel back. A zero value on this measure makes sure the drivers feel no
resistance torque unless the steering wheel is at the straight path position. That
consequently leads to a feeling of easy returning.
6.3 Result Analysis
Considering the small sample size of data available a case study is used to find out possible
correlations according to findings from test data. Both simple and multiple linear regression
analysis are employed. All of the links between subjective assessments and objective metrics,
which have high possibility of existing, can be used to provide insights for tuning strategy on
vehicle steering. Recall that, the main analyzing approach is individual case study and the
conclusion is based on the findings that are confirmed from expert drivers.



54
Not that many correlations found among E-class cars, since the number of drivers who
went through all vehicle tests is insufficient to become a statistical sample.
The most common and uniform results come from torque feedback or returnability in
straight-ahead and cornering tests (SA-6 and 10), where appropriate initial explanatory
variables are set.
For those subjective assessments with only one explanatory variable (i.e., roll control),
more corresponding objective metrics are needed in order to explore regression
Links between Subjective Assessments and Objective Metrics for Steering






correlation.
Parking torques (OM-26 or 27), known as mean value of steering torque, are preferred to
be small for parking efforts perception (SA-2).
For response assessments in both straight-ahead and cornering controllability (SA-4 and
8), the found correlations are not common for all classes of cars. More data samples are
to be introduced if the positive correlations with response gain at straight path (OM-2
and 4), understeer (OM-12) and time lag (OM-16) should to be confirmed.
The torques for initial steering wheel turning (OM-10) and for holding steady in
cornering (OM-23) are preferred to be larger to give better torque feedback (SA-6 and
10).
For all vehicles, lower torque buildup at straight path and cornering (OM-9 and 19/20) is
favorable as a larger value means the steering wheel torque could increase too rapidly
with lateral acceleration.
On-center hysteresis (OM-21) mainly has negative effect on returnability (SA-11)
because in the process of returning steering wheel, the less on-center hysteresis
guarantees that the assistance force from steering wheel will disappear until the steering
wheel angle is quite close to zero-degree position. On the contrary, a larger on-center
hysteresis makes an unanticipated resistance force leading to worse returnability feeling.
Off-center hysteresis (OM-22) has positive correlation on torque feedback and
returnability (SA-10 and SA-11) for non-SUVs (C, D and E-class). Meantime, it shows
negative link for SUVs. The reason is not there is different preference, but the objective
value for SUVs is higher than optimal range, so if higher grade wants to be achieved,
this objective value should be lower (closer to preferred interval), and vice versa for the
non-SUVs.
55
56
Links between Subjective Assessments and Objective Metrics for Steering
7 Nonlinear Regression Analysis of Subjective and Objective Data
In this chapter selected data from VCC will be used in Neural Network
training to define preferred ranges of some objective measures that can give
better subjective evaluation for steering.
7.1 Initial Results from Neural Network Model
As illustrated previously Neural Network can be used to explore nonlinear subjectiveobjective correlation and optimize the value range of objective measures in product tuning.
With the tentative trial of the data from a similar research at KTH it is already sure that at
least 16 testing examples are necessary to achieve NN training.
Based on present VCC data set in each vehicle class it is not possible to reach enough sample
size unless different testing groups are merged. Hence, data from different vehicle classes
was combined in order to train NN successfully. After some initial results from NN models
are acquired they can be used to check the conclusion from our previous study and other
research.
7.1.1 Data Used in NN Training
To have 16 testing objects some groups have to be combined. However, the problem that the
expectation from various classes varies a lot has to be considered. Thus, D-class, E-class and
some C-class vehicles (compact SUVs are excluded) will be collected together since at least
they share some sedan properties. Totally, this combination will give 15 samples that are
barely enough to fulfill the requirement of building a NN model.
Through all test drivers only three of them experienced all tests with these 15 vehicles. That
means only their subjective data are valid to make NN training. By using case study method
each of three drivers will have a result table that is the basis for further analysis.
7.1.2 NN Training
The basic settings for NN training have already been shown in section 2.3.1 and parameters
of NN can follow the previous design as well. Though different data were used the previous
NN model still can be considered as the most reasonable design to explore nonlinear results.
It consists of 27 objective measures, 12 subjective assessments and 3 drivers so 972 (
) networks can be produced for a single input system. As seen in Table 25 and 26 the
relationship between single subjective assessment and objective measure for Driver I is
shown. Table 25 shows the regression coefficient (r-value) between target output and
predicted rating for a given pair of subjective and objective parameter. That indicates if the
learned function can relate the input and output well enough and the cells with r-value larger
than 0.7 are marked.
57
Subjective assessments
Subjective assessments
Table 25. NN results, r-value for Driver I.
Objective measures
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 0.1 0.4 0.3 0.2 0.3 0.6 0.4 0.8 0.7 0.6 0.4 0.4 0.5 0.4 0.6 0.4 0.3 0.5 0.6 0.7 0.4 0.7 0.7 0.6 0.5 0.8 0.6
2 0.0 0.4 0.5 0.1 0.4 0.5 0.5 0.5 0.6 0.5 0.4 0.5 0.6 0.3 0.5 0.5 0.4 0.6 0.4 0.7 0.6 0.6 0.2 0.5 0.5 0.8 0.6
3 0.6 0.9 0.6 0.7 0.3 0.6 0.4 0.7 0.6 0.7 0.4 0.3 0.8 0.5 0.5 0.5 0.5 0.1 0.7 0.6 0.6 0.8 0.4 0.7 0.8 0.3 0.7
4 0.7 0.5 0.3 0.6 0.3 0.1 0.2 0.5 0.5 0.4 0.2 0.3 0.2 0.5 0.2 0.5 0.7 0.2 0.5 0.5 0.5 0.4 0.7 0.1 0.1 0.4 0.6
5 0.3 0.1 0.4 0.1 0.5 0.3 0.5 0.5 0.3 0.4 0.2 0.3 0.5 0.4 0.4 0.5 0.8 0.3 0.2 0.1 0.5 0.5 0.2 0.2 0.3 0.8 0.1
6 0.4 0.2 0.4 0.4 0.5 0.4 0.3 0.4 0.3 0.4 0.6 0.3 0.5 0.3 0.7 0.4 0.4 0.3 0.2 0.7 0.3 0.2 0.1 0.2 0.1 0.4 0.2
7 0.2 0.2 0.5 0.2 0.7 0.3 0.3 0.3 0.2 0.5 0.8 0.3 0.4 0.2 0.6 0.5 0.5 0.5 0.3 0.6 0.2 0.3 0.2 0.3 0.2 0.2 0.2
8 0.3 0.5 0.4 0.3 0.5 0.5 0.5 0.4 0.2 0.4 0.3 0.4 0.2 0.4 0.4 0.3 0.3 0.5 0.3 0.4 0.3 0.4 0.5 0.3 0.5 0.6 0.0
9 0.3 0.1 0.3 0.1 0.3 0.2 0.6 0.5 0.1 0.4 0.3 0.3 0.5 0.4 0.5 0.5 0.6 0.4 0.1 0.1 0.5 0.4 0.2 0.1 0.7 0.8 0.2
10 0.5 0.7 0.6 0.5 0.2 0.7 0.3 0.7 0.5 0.4 0.7 0.2 0.8 0.5 0.6 0.5 0.4 0.4 0.7 0.5 0.3 0.7 0.2 0.7 0.5 0.4 0.7
11 0.8 0.8 0.7 0.9 0.4 0.2 0.3 0.3 0.6 0.9 0.4 0.4 0.8 0.4 0.4 0.4 0.3 0.1 0.5 0.2 0.6 0.3 0.5 0.2 0.4 0.3 0.2
12 0.3 0.4 0.3 0.4 0.6 0.8 0.5 0.2 0.4 0.2 0.5 0.5 0.4 0.4 0.6 0.5 0.2 0.4 0.2 0.8 0.4 0.2 0.2 0.4 0.2 0.3 0.1
1
2
3
4
5
6
7
8
9
10
11
12
Table 26. NN results, mean error for Driver I.
Objective measures
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1.0 0.9 0.9 1.0 0.9 0.7 0.8 0.5 0.7 0.7 0.9 0.9 0.7 0.9 0.8 0.9 0.9 0.8 0.8 0.7 0.8 0.7 0.7 0.7 0.8 0.6 0.7
1.0 0.7 0.8 0.9 0.8 0.7 0.7 0.8 0.7 0.7 0.9 0.8 0.6 0.9 0.7 0.7 0.9 0.7 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.4 0.6
0.8 0.5 0.9 0.7 0.9 0.8 1.0 0.6 0.7 0.7 0.9 0.9 0.7 0.8 0.8 0.9 1.0 0.9 0.7 0.8 0.8 0.5 0.8 0.7 0.5 0.9 0.7
0.3 0.4 0.5 0.3 0.4 0.5 0.5 0.4 0.5 0.5 0.5 0.4 0.5 0.4 0.5 0.4 0.3 0.5 0.4 0.4 0.4 0.4 0.3 0.5 0.5 0.4 0.4
0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.4 0.6 0.5 0.6 0.6 0.5 0.6 0.6 0.5 0.3 0.6 0.7 0.7 0.5 0.6 0.6 0.6 0.4 0.4 0.6
0.5 0.5 0.5 0.4 0.4 0.4 0.5 0.5 0.5 0.4 0.4 0.5 0.4 0.5 0.4 0.4 0.5 0.4 0.5 0.3 0.4 0.5 0.5 0.5 0.5 0.4 0.5
0.6 0.6 0.5 0.6 0.4 0.6 0.6 0.6 0.6 0.6 0.3 0.5 0.5 0.6 0.4 0.6 0.6 0.5 0.6 0.5 0.7 0.6 0.6 0.5 0.6 0.6 0.6
0.5 0.4 0.5 0.5 0.4 0.4 0.4 0.5 0.4 0.5 0.5 0.5 0.5 0.4 0.5 0.5 0.5 0.4 0.5 0.5 0.4 0.5 0.4 0.5 0.5 0.4 0.5
0.5 0.5 0.6 0.6 0.5 0.5 0.5 0.4 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.3 0.4 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.3 0.5
0.2 0.2 0.2 0.3 0.3 0.2 0.2 0.2 0.2 0.3 0.2 0.3 0.2 0.3 0.2 0.2 0.3 0.3 0.2 0.2 0.3 0.2 0.3 0.2 0.3 0.3 0.2
0.4 0.4 0.5 0.3 0.6 0.6 0.6 0.6 0.5 0.4 0.6 0.7 0.4 0.6 0.5 0.5 0.7 0.7 0.5 0.6 0.5 0.5 0.6 0.6 0.7 0.9 0.7
0.3 0.4 0.3 0.3 0.3 0.4 0.3 0.4 0.3 0.4 0.3 0.3 0.4 0.3 0.3 0.3 0.4 0.3 0.4 0.3 0.3 0.4 0.4 0.3 0.4 0.4 0.4
The regression coefficient is not the only criterion to check the quality of learning. Even if
the r-value is 1 the mean error (ME) between the predicted output and target value might be
contradictorily large. The reason is that the regression line with perfect learning should not
only have small r-value but also go through the origin with 45°-slope as seen in Figure 27.
Therefore, it is also important to check the ME seen in Table 26 where the cells with mean
error smaller than 0.5 are shaded as well.
58
Links between Subjective Assessments and Objective Metrics for Steering
Figure 27. Regression between target rating and output rating learnt by NN.
7.1.3 Correlation Results
Preferred objective measures can be defined according to the learning results from NN as
seen in Table 25 and 26 if there are links with both good regression coefficient and ME (i.e.,
large r-value and small ME). Taking Driver I as an example, the r-value of OM-5 and SA-7
is 0.7 and the ME is 0.4. Then this link can be considered as valid. By following the same
method illustrated in Figure 7 the preferred range for OM-5 that gives better score on SA-7 is
larger than 1.4. All the preferred values for objective measures based on Driver I are listed in
Table 27. The total results from all of the three drivers are collected in Appendix D.
Table 27. NN results with good fitness and small error, preferred ranges for Driver I.
Objective measures
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
1
2
<3.2
Subjective assessments
3
4
45~
75
5
6
<3.8
5~6
7
28~
32
>1.4
8
9
<3.8
10
11
12
>95
3.5~ 25~ 1~ 1.5~
4.5 30 1.25 1.85
>28.
5
<2.2
<1.8
/
>92
>2
>90
3~6
59
Regardless of various drivers or subjective assessments the preferred range for each objective
measure can be defined if summarizing the findings in Appendix D. Since only the subjective
rating of three drivers is applied in this analysis the found results must be quite limited and
without very high certainty. The preferred ranges of some objective measures are shown in
Table 28. With results from King et al. [11] more analysis and comparison will be done in
the next section.
Table 28. Overview of objective measure range causing high rating.
No. Objective measures
Preferred range Reference from other research
2 Response Gain Straight Path [°/s/100°SWA]
25~30
Yaw velocity gain: 20~25
6 Response Time Delay [ms]
>95
8 Torque Deadband [°]
<2.2
11 Yaw Response Gain [°/s/100°SWA]
28~32
20 Torque Buildup Cornering [Nm/g]
4~6
22 Off Center Hysteresis [Nm]
1.5~2.2
23 Effort Level [Nm]
3.6~4.5
Steering torque at 0.3g: <6.5
26 Parking Efforts Standstill [Nm]
<3.3
27 Parking Efforts Rolling [Nm]
>1.5
7.2 Result Analysis
The value of each objective measure falling in a certain range will give drivers better
subjective perception. The ranges of objective measures can be roughly defined by
comparing nonlinear correlation results found from the data of three drivers as seen in Table
28. It is necessary to check the rationality of the results found in this thesis work. Therefore
further examination is done by comparing the results with the findings from previous work.
Though the results are not 100% reliable depending on small sample size they may still
provide some insights for further study.
 Response gain straight path
When the response gain at straight path is larger than 25°/s/100°SWA the subjective rating
will increase. Compared with the similar objective metrics in another research [11] where the
preferred yaw gain at 0.7Hz is within the range of 20~25 the result of this metric seems to
map well when taking the difference of testing settings into account.
 Torque deadband
The preferred range of torque deadband is smaller than 2.2°. The objective metrics larger
2.2° leads to imperfect subjective perception. On the other hand to specify the lower
boundary of preferred torque deadband more data with extremely small value (lower than
1.0°) must be collected.
 Yaw response gain
During the cornering state drivers preferred the yaw gain within the range
28~32°/s/100°SWA regardless of the assessments being judged.
60
Links between Subjective Assessments and Objective Metrics for Steering
 Torque buildup cornering
The preferred range of torque buildup in cornering is defined by the NN models, which is
4~6Nm/g. However, in the previous linear correlation study there is no clear relationship for
this metric.
 Off-center hysteresis
As discussed in the section 6.2.6, a potential optimal range of 1.5~2.0Nm is identified for offcenter hysteresis. In this NN modeling, the favorable range is about 1.5~2.2Nm, which ties
quite well with the former hypothesis.
 Effort level
The optimal range for effort level at cornering is starting from 3.6Nm. This result fits with the
positive correlation in earlier analysis. Based on current data the upper boundary of this
range cannot be specified since there is no metrics larger than 4.5Nm. In the research of King
et al. [11] it is defined that the objective measure steering torque at 0.3g should not exceed
6.5Nm, which is somehow the upper boundary of this effort level.
 Parking efforts standstill
The negative correlation between parking effort at standstill state and subjective rating is
revealed quite clearly in linear regression chapter. Here, through the nonlinear NN the upper
boundary is defined as 3.3Nm. That means the steering torque larger than 3.3 Nm will lead to
worse parking perception.
 Others (response time delay and parking efforts rolling)
The preferred ranges of response time delay and parking efforts rolling found in this thesis
seem confused when compared with normal driving experience. Response time delay found
above 95ms is preferred that is normally expected to be small. The possible reason is that
practical values of this parameter for our test objects are sufficiently low leading to more or
less too fast response that is not pleased either. There should be an optimal range of this
parameter (e.g. 95< response time delay<120ms) that cannot be defined based on our
available data set. The same consideration can be applied to parking efforts rolling.
61
62
Links between Subjective Assessments and Objective Metrics for Steering
8 Recommended Procedure of Obtaining Ideal Data for Analysis
In this chapter several methods to perform tests and obtain subjective data
with the least uncertainty and the highest uniformity to achieve good NN
training and get reliable results will be introduced. Among these methods
the one considered as most appropriate for VCC to carry out will be
proposed.
8.1 Track Test with Real Vehicle
The data size must be large enough to carry out linear or nonlinear correlation analysis. If
assessed items are standardized and maintained in the future tests according to the trail NN
design the appropriate number of sample in subjective assessment tests should be at least 16
to draw a clear and valid conclusion.
8.1.1 Test Track
In order to make the subjective tests consistent with measured objective data the test track
should be able to cover all wanted test environments including 50m test, straight-ahead test
and cornering test.
8.1.2 Various Vehicles
Testing many different vehicles is not adopted as main method since the key problem is that
it is not easy to collect sufficient vehicles for a research study. Considering the least sample
number it will be costly to gather 16 test vehicles with various steering characteristics. In
references [16] [17] where fewer than ten real cars are chosen as test objects, the purpose of
them is to do simple linear regression, and the focus is more than on steering. Other factors
than only steering system that should be taken into account are the differences of power train,
chassis and car body. Test people cannot control these different parameters or attributes. As a
result it could be a problem to confirm if the difference in assessment is due to some steering
properties or other attributes with little interests (i.e., power, chassis, size and mass). The
interferences cannot be excluded although these tests can be conducted simultaneously
saving both time and cost.
8.1.3 Same Vehicle with Various Steering Characteristics
In the previous research [4] [8] [18], various configurations or steering settings are used to
vary the characteristics. The changes they made include different tires, dampers, yaw-inertia,
bump steer and stiffness. If adequate statistical samples are wanted at least four factors have
to be varied between two levels so that 16 configurations are acquired with full factorial test
design as seen in Table 29. However, in this case the difference between configurations could
be too small to give significant difference in subjective perception. Then, if still wanting to
achieve 16 configurations (L16 matrix) instead of using fractional factorial test design [4]
more factors can be varied for Resolution V (
) or Resolution IV (
) as Table 30 or
63
31. If this method is used the biggest challenges are time consumption and adjustment
accuracy.
Table 29. Two-level full factorial test design for orthogonal L16 matrix.
Configuration Factor 1 Factor 2 Factor 3 Factor 4
No. 1
+
+
+
+
No. 2
+
+
+
No. 3
+
+
+
No. 4
+
+
No. 5
+
+
+
No. 6
+
+
No. 7
+
+
No. 8
+
No. 9
+
+
+
No. 10
+
+
No. 11
+
+
No. 12
+
No. 13
+
+
No. 14
+
No. 15
+
No. 16
Table 30. Two-level fractional factorial test design for orthogonal L16 matrix (resolution V).
Configuration Factor 1 Factor 2 Factor 3 Factor 4 Factor 5
No. 1
+
No. 2
+
No. 3
+
No. 4
+
+
+
No. 5
+
No. 6
+
+
+
No. 7
+
+
+
No. 8
+
+
+
No. 9
+
No. 10
+
+
+
No. 11
+
+
+
No. 12
+
+
+
No. 13
+
+
+
No. 14
+
+
+
No. 15
+
+
+
No. 16
+
+
+
+
+
Table 31. Two-level fractional factorial test design for orthogonal L16 matrix (resolution IV).
Configuration Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6 Factor 7 Factor 8
No. 1
No. 2
+
+
+
+
No. 3
+
+
+
+
No. 4
+
+
+
+
-
64
Links between Subjective Assessments and Objective Metrics for Steering
No. 5
No. 6
No. 7
No. 8
No. 9
No. 10
No. 11
No. 12
No. 13
No. 14
No. 15
No. 16
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
8.2 Simulator Experiments
Real track test is abandoned in [15] due to the small tractor samples and simulator
experiment is chosen. The way of varying characteristics for simulator is quite similar with
real vehicles. The only difference is that one factor can be adjusted at more than two levels
and the accuracy of variation is guaranteed. Besides in simulator experiments, the test
maneuver can be included in simulation program before the test such as lane change, slalom,
parking, etc. and also run all after each other in one simulation run.
By comparing these possible test objects, the conclusion of their features is shown in Table
32.
Table 32. Overview of test objects.
Test object
Monetary cost
Time cost
Influence
Analysis
Different vehicles
Rent of vehicles
Test simultaneously
Difference of components
Linear correlation
Different configurations
Rent of a vehicle and tire sets
Test configurations and adjust settings
Adjustment accuracy
Linear and/or nonlinear correlation
Simulator
Rent of simulator
Test configurations one by one
Simulation feel
Linear and/or nonlinear correlation
8.3 Suitable Test Objects
In the current database there are already five vehicle models in D-class so the best option is
to combine different vehicles and different configurations. To be specific, all these five cars
are varied in some factors so that under each vehicle there are several distinct settings. Tire
and mass adjustments are interesting since that is the easiest to adjust. Given the variation as
seen in Table 33 with three adjustments and fractional factorial design totally there will be 16
or 20 different test objects. This is at least sufficient for linear and/or nonlinear regression
analysis. Another important aspect is that once these test objects are determined collection of
objective data should be conducted as well.
65
Table 33. Test objects with two-level factorial test design for orthogonal L4 matrix (resolution
III).
Vehicle parameter First level [-]
Second level [+]
Front tires
Normal tires (tire pressure)
Special tires (tire pressure)
Rear tires
Normal tires (tire pressure)
Special tires (tire pressure)
Mass
Normal load
Extra load (ballast/passenger)
4 or 5 vehicles
Configuration 1
Configuration 2
Configuration 3
Configuration 4
Front tires
+
+
Rear tires
+
+
Mass
+
+
8.4 Test Subjects
Studies from Agebro, Schoeggle et al. [19] [20] have revealed the huge difference of driving
skill and rating ranges between expert and novice drivers. However, the final purpose is to
give best driving perception to general customers. It is good to include some general drivers
appropriately so as to check if these two groups have significantly driving preference.
Both expert and novice groups should include at least six or seven drivers. That is the
minimum statistical number. Expert drivers in this thesis are vehicle engineers employed by
VCC and novice driver can be picked from normal customers, students or professors. Each
driver should drive each car with another one who notes down the subjective feeling and with
best efforts to follow the maneuvers. The questionnaire must be completed soon after tests
and without allowing drivers to exchange opinions. The plus side of always having a codriver is that the feelings that the driver feels can be noted and captured right away. This is
especially important when using less skilled drivers.
8.5 Pre-test with a Reference Vehicle
In the previous section regarding individual assessment evaluation the importance of a
reference vehicle (configuration) and rating range of each assessment is needed even for
expert drivers if analysis should be done based on mean rating value. So a vehicle model
with similar features to one of the test objects is required as a reference vehicle.
For each driver after driving the reference vehicle, he or she evaluates the experimental one
and compare with perception from reference rating value (which could be 7-point). Ideally,
across all tests single-blind trails should be done to drivers but this might take too much time.
Since several vehicles with various settings are selected at least one of the test objects could
be blind. Meaning that no changes are done to this test object and it is thus the same as
someone else of the test objects. If this is possible the repeatability of subjective assessment
can be analyzed by studying difference in ratings from these two tests.
66
Links between Subjective Assessments and Objective Metrics for Steering
8.6 Normalization Work
8.6.1 Rating Mean Normalization
As long as the reference is really adopted by all test drivers and their rating is repeatable, i.e.,
the difference in ratings from two single-blind tests on the reference vehicle is small. The
mean ratings for each assessment and every driver should be very close to each other.
Otherwise, some work to normalize the rating mean is inevitable.
8.6.2 Spread Normalization
Even if the reference vehicle gives drivers the base line to start their assessment, the spread
of their ratings or the sensitivity of their perception cannot and should not be controlled. The
normalized subjective evaluation should place all ratings in the same range. As long as the
reference perception is followed by test drivers sufficiently well the normalized evaluation
can lead to reliable mean ratings so that the method of case study is abandoned to save time
and improve validity.
Back to the flow chart as seen in Figure 1 and for a specific assessment item SA-Y (m drivers
make evaluation on n configurations,
[
],
[
]), first the average value of
minimum and maximum rating given by all m drivers is acquired by Equation (21) and (22).
Then, the ratio of normal rating range to individual range is defined for each driver as
Equation (23).
∑
[
]
(21).
∑
[
]
(22).
[
[
[
]
]
]
[
[
]
]
(23).
(24).
(25).
With normalization work done the subjective data of drivers should be similar to each other
as seen in Figure 28. If there is still someone with larger range or mean values than the others
that data can be excluded as an outlier. The rest of the subjective ratings are considered as
truthful and the average values are allowed to represent the comprehensive data.
67
Figure 28. Effect of rating mean and spread normalization.
68
Links between Subjective Assessments and Objective Metrics for Steering
9 Summary Results
In this chapter the findings regarding subjective and objective data
evaluation, linear links and preferred ranges will be summarized and
presented.
9.1 Correlation of Objective Measures and Evaluation of Rating
The aforementioned test objects in this study are different vehicles not one vehicle with
various settings. Thus, it is understandable that found correlations between pairs of objective
measures do not have extremely high correlation coefficients as shown in Table 7. In other
words, these cannot support any elimination work of objective parameters. Nevertheless,
these correlations can still reveal some confirmed relationships between two different
objective measures.
The work of subjective data evaluation shows that the data used in previous analyses is
basically valid. Some tendency based on specific subjective assessments or drivers can be
found according to the exemplified subjective data from D-class. However, the possible
rating tendency of drivers will not affect the results in this thesis where the method of case
study is applied.
9.2 Confirmed Subjective-objective Links for Steering
All confirmed results about simple/multiple linear correlation and preferred range according
to Neural Network are collected in Table 34. Most of the found linear correlated objective
measures are under three categories torque feedback, response and 50m test. That means
these objective measures are key to adjust the corresponding subjective assessments.
Furthermore, nearly all confirmed links have pretty unequivocal pattern either purely positive
or negative correlation. Some of them, for example on-center hysteresis can be used to infer
the optimal value of objective measure.
As for nonlinear analysis, based on current data set, the findings are basically agreed with
results from linear part. Still, most found preferred ranges are about torque feedback,
response and 50m test, regardless of drivers and subjective assessments. Besides, compared
with results from King [11] the confirmed ranges of certain objective measures in this thesis
are reasonable and even more detailed. So, the information about these range values is
supposed to be useful when tuning the steering characteristics of a normal sedan vehicle.
69
Table 34. Confirmed linear and nonlinear links between subjective assessments and objective
measures for steering.
Level 3
Response
OM Objective measures (Level 4)
1
Window [deg]
2
Response Gain Straight Path [°/s/100°SWA]
3
Lateral Acc. Resp. Gain [°/s/100°SWA]
4
Lateral Acc. Resp. Gain [°/s/100°SWA]
5
Gain Linearity [-]
6
Roll Control 7
8
Torque
9
Feedback
10
SA Subjective assessments (Level 3)
Linear
correlation
25~30
Response Time Delay [ms]
Roll Control Straight Path [°/g/s]
Torque Deadband [°]
>95
<2.2
Torque Build Up [Nm/100°SWA]
6
Torque Feedback (Straight-ahead) Negative
Friction Feel [Nm]
6
Torque Feedback (Straight-ahead) Positive
11
Yaw Response Gain [°/s/100°SWA]
8
Response (Cornering)
Negative
12
Response Gain Understeer [°/g]
8
Response (Cornering)
Positive
13
Response Gain Linearity [%]
8
Response (Cornering)
Positive
14
Rel. yaw [email protected] lat. Acc.
15
Sine Time Lag Yaw-SWA [ms]
16
Sine Time Lag Ay-SWA [ms]
8
Response (Cornering)
Positive
17
Sine Time Lag Ay-Yaw [ms]
Roll Control 18
Roll Control Cornering [°/g]
Response
Torque
Feedback
- (50m test)
70
Preferred
range
28~32
10 Torque Feedback (Cornering)
Negative
11 Returnability (Cornering)
Negative
Torque Buildup Cornering [Nm/g]
10 Torque Feedback (Cornering)
Negative
On Center Hysteresis [°]
11 Returnability (Cornering)
Negative
10 Torque Feedback (Cornering)
Positive/Negative
11 Returnability (Cornering)
Positive
10 Torque Feedback (Cornering)
Positive
11 Returnability (Cornering)
Positive
Parking Efforts Standstill [Nm]
2
Efforts (Parking)
Negative
<3.3
Parking Efforts Rolling [Nm]
2
Efforts (Parking)
Negative
>1.5
19
Torque Buildup Into The Corner [Nm/100°SWA]
20
21
22
Off Center Hysteresis [Nm]
23
Effort Level [Nm]
24
Low Speed Response Gain [°/s/100°SWA]
25
Low Speed Torque Buildup [Nm/100°SWA]
26
27
4~6
1.5~2.2
3.6~4.5
Links between Subjective Assessments and Objective Metrics for Steering
10 Conclusions and Recommendations
This chapter will briefly conclude findings by using linear and nonlinear
correlation analysis. Based on the ideal testing procedure, some
recommendations to future work will be made.
10.1 Conclusions
Simple and multiple linear analysis are useful to detect if certain direct positive or negative
correlation exists while Neural Network is an important tool to specify preferred range of
objective measures even to find some nonlinear links that are not easy to be described by
linear correlation.
Both of the two analyzing approaches lead to the findings concerning almost the same
objective measures.




The preferred range for yaw response gain (OM-2 and 11) is located in a normal scope.
That means for the purpose of good driving response neither too nervous nor too lazy
steering wheel will be praised.
Steering torque at low or high lateral acceleration, and off-center hysteresis (OM-10, 22
and 23) at high speed are preferred to be reasonably large. This fact gives reason to
define the detailed ranges of these three measures.
Torque buildup at the start of cornering and during cornering (OM-19 and 20) should not
exceed a certain upper boundary. Otherwise, it will cause an uncomfortable torque
feedback feeling.
The steering torque at parking condition (OM-26 and 27) definitely has a limit on the
maximum value. The steering torque below this critical value will surely guarantee a
better perception of parking effort. However the lower boundary is difficult to define
because practical steering systems cannot reach infinitely small parking torque.
In addition to agreed findings in linear and nonlinear parts on-center hysteresis and response
time delay should be investigated more deeply as long as more data set is available. Anyway,
the approach of linear or nonlinear analysis in this thesis has shown a promising future to
assess steering properties via objective metrics. An evaluation tool that uses only objective
metrics as input to produce subjective rating for steering can be built based on the confirmed
subjective-objective links. Thus, real vehicle tests are not necessary if this tool can be used
during simulation. Furthermore, this tool can also be integrated into steering robot test, which
can enable the robot to collect not only objective metrics but also subjective evaluation.
10.2 Recommendations to Future Work
In conclusion the future tests must guarantee that the amount of data samples is sufficient to
finish a complete NN training and achieve more convincing conclusion. The detailed
suggestions for future work can cover several aspects.
71


MUST:
 Recollect both subjective and objective data due to new test objects
 16 to 20 test objects (4 or 5 vehicles in the same segment each with 4 various
settings)
 At least 6 or 7 expert drivers (more involved will result in less uncertainty)
 Unify the rating methods of drivers with best efforts after each test expedition
(rating range and/or rating reference)
 Uniform data sheets with fixed content and format
 No cell regarding objective and subjective items in data sheets is supposed to be
blank
OPTIONAL:
 Novice drivers (at least 6 or 7)
 Single-blind test for test drivers (drivers have no idea what adjustments have been
made on each vehicle)
 Pre-test to give drivers rating reference (with this one totally 17 or 21 test objects)
 The reference vehicle duplicated in test objects to check the repeatability of rating
 Focus on those assessments or measures already with found correlations
(OM-1, 2, 3, 4, 11, 12, 16, 17, 19, 20, 22, 23, 25, 26 and 27; SA-2, 4, 6, 7, 8, 10, 11
and 12.)
 Have a Lotus Elise or extreme vehicle in a vehicle class to gain extreme ratings and
wider spread of data
The MUST steps make sure test data can be used for normal NN training and the
OPTIONAL items can increase the confidence of final conclusion on preferred objective
range. With this work and the method of NN able to produce wanted results further studies
on subjective-objective correlation links are possible.
There should also be a possibility to add several new vehicles to already existing data sets
given that the process stay the same and the track and environment stay roughly the same.
Thus an extensive model and database could be built up across all various vehicle segments
in order to model vehicles with ‘perfect driving perception’ over the years.
As long as the subjective-objective links are defined more accurately and comprehensively
an evaluation tool can be built and used during front end simulations. During software
simulation process the adjustment of steering configurations will not be limited by the safety
consideration for test drivers. Therefore, wider spread of objective and subjective data could
be achieved. In turn, this can help optimize the links. As a result, it is more likely to develop
a vehicle with optimal steering design.
72
Links between Subjective Assessments and Objective Metrics for Steering
Reference
[1] S. Davies, "A Holistic Approach to Quality," Manufacturing Engineer, vol. 86, no. 1,
pp. 8-13, Feb.-March 2007.
[2] Subjective Rating Scale for Vehicle Handling, Vehicle Dynamics Standards Committee,
2007.
[3] G. Helmut List and S. Peter Schöggl, "Method for Analyzing the Driving Behavior of
Motor Vehicles". United Stats Patent 6079258, 27 June 2000.
[4] Chen, David, "Subjective and Objective Vehicle Handling Behaviour," PhD Thesis, The
University of Leeds, Leeds, 1997.
[5] Rothhämel, M., IJkema, J., Drugge, L., "A Method to Find Correlations between
Steering Feel and Vehicle Handling Properties Using a Moving Base Driving
Simulator," Vehicle System Dynamics, vol. 49, no. 12, pp. 1837-1854, August 2011..
[6] Anthony Best Dynamics, "Steering Robot Systems: In-vehicle robot test systems," 2011.
[Online].
Available:
http://www.abd.uk.com/upload/files/2011-11-17_15-1153_SP6020%20-%20Steering%20Robots%20outline%20-%20Issue%206.pdf.
[7] Beale, M. H., Hagan, M. T., Demuth, H. B., "User’s Guide (Neural Network
Toolbox™)," [Online]. Available: http://www.mathworks.com. [Accessed 2011].
[8] H. A. S. Ash, "Correlation of Subjective and Objective Handling of Vehicle Behaviour,"
PhD Thesis, The University of Leeds, Leeds, 2002.
[9] Rumelhart, D. E., Hinton, G. E., Williams, R. J., "Learning Internal Representations by
Error Propagation," in Parallel Distributed Processing, MIT Press, ISBN:0-262-68503X, 1986, pp. 318-336.
[10] Yam, Y., Chow, T., "Determing Initial Weights of Feedforward Neural Networks based
on Least Squares Method," Nerual Processing Letters, vol. 2, no. 2, pp. 13-17, 1995.
[11] King, R. P., Crolla,D. A., Ash, H. A. S., Whitehead, J., "Identfication of SubjectiveObjective Vehicle Handling Links Using Neural Networks for the Foresight Vehicle," in
SAE 2002 World Congress, Detroit, 2002.
[12] Sivanandam, S. N., Sumathi, S., Deepa, S. N., Introduction to Fuzzy Logic Using
73
Matlab, Springer; 1 edition, 2006.
[13] Mariani, F., Grimaldi, C., Sgatti, S., De Cesare, M. et al., "Artificial Intelligence
Methodologies for Oxygen Virtual Sensing at Diesel Engine Intake," SAE Technical
Paper 2012-01-1153, 2012.
[14] B.A.M van Daal, "Friction and Compliance Identification in a Vehicle's Steering
System," Extern traineeship report, Eindhoven, 2007.
[15] M. Rothhämel, "Capturing Steering Feel - A step towards implementation of active
steering in heavy vehicles," Licentiate Thesis, US-AB, Trita-AVE 2010:57, ISSN 16517660, Stockholm, 2010.
[16] Data, S., Frigerio, F., "Objective Evaluation of Handling Quality," in IMechE J.
Automobile Engineering, Orbassano, Italy, 2002.
[17] de Paula Eduardo, G., "Manual Steering Objective Reference Data Definition based on
Subjective Evaluation Correlation," SAE Technical Paper 2011-36-0031, 2011,
doi:10.4271/2011-36-0031.
[18] Monsma, S., Defilippi, W., "Artificial Neural Networks for the Assessment of Driver
Judgement," Vehicle System Dynamics, IAVSD 2011, Manchester, 2011.
[19] Agebro, M., "Driver Preferences of Steering Characteristics," Licentiate Thesis, KTH,
Stockholm, Trita-AVE 2007:66, ISSN 1651-7660, 2007..
[20] Schoeggl, P., Ramschak, E., "Vehicle Drivability Assessment Using Neural Networks
for Development, Calibration and Quality Tests," SAE International, 2000.
74
Links between Subjective Assessments and Objective Metrics for Steering
Appendix A
F-Distribution Table for
confidence level (
and
when checking the regression significance).
and
is corresponding to
75
76
Links between Subjective Assessments and Objective Metrics for Steering
Appendix B
Correlation coefficient between two objective measures for C-class.
R-value of Obj vs Obj
Objective measures
1
2
3
4
5
6
7
8
Objective measures
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Correlation coefficient between two objective measures for D-class.
R-value of Obj vs Obj
Objective measures
1
2
3
4
5
6
7
8
Objective measures
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
77
Correlation coefficient between two objective measures for E-class.
R-value of Obj vs Obj
Objective measures
1
2
3
4
5
6
7
8
Objective measures
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Correlation coefficient between two objective measures for SUV.
R-value of Obj vs Obj
Objective measures
1
78
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
2
3
4
5
6
7
8
Objective measures
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
(T-stat)
Q12
(T-stat)
Driver 9
Q4
(T-stat)
Q8
(T-stat)
Q10
(T-stat)
Q4
(T-stat)
Q6
(T-stat)
Q7
(T-stat)
Q10
(T-stat)
Q12
(T-stat)
Driver 5
Q7
(T-stat)
Q10
Driver 3
Q7
(T-stat)
Driver 4
1
-4.09
-3.43
-10.7
-2.86
-7.70
-2.14
-4.32
-6.67
-0.08
-0.74
2.68
10
8.74
5.84
4.37
5.27
6.58
0.07
18.88
9
-12.3
8
5.80
7
-0.03
6
-12.5
-7.29
5
-3.01
-11.9
7.05
4
22.77
3
0.50
2
Multiple linear regression for C-class.
Appendix C
0.04
2.98
2.57
-3.08
0.01
17
-10.7
16
-0.14
15
-0.20
-5.65
14
Objective measure
-0.04
13
3.07
12
0.09
11
18
0.06
3.40
21
-4.13
-2.95
-0.07
20
-0.18
-5.75
-0.57
19
-4.64
-1.82
2.57
4.55
1.44
5.05
4.35
5.94
3.98
6.93
3.10
23
1.00
22
24
25
26
27
0.97
0.83
0.73
0.98
0.95
0.99
0.99
0.89
0.92
0.96
0.97
R^2
0.99
21.56
7.46
4.09
42.12
11.70
102.25
40.37
7.83
7.75
39.80
18.46
F-stat
179.66
Links between Subjective Assessments and Objective Metrics for Steering
79
80
Driver 1
Q8
(T-stat)
Q10
(T-stat)
Q11
(T-stat)
Q12
(T-stat)
Driver 2
Q6
(T-stat)
Q8
(T-stat)
Driver 3
Q10
(T-stat)
Q11
(T-stat)
Q12
(T-stat)
Driver 4
Q8
(T-stat)
Q11
(T-stat)
Q12
(T-stat)
Driver 5
Q7
(T-stat)
Q10
(T-stat)
Q12
(T-stat)
Driver 6
Q7
(T-stat)
Q8
(T-stat)
Q11
(T-stat)
Driver 7
Q7
(T-stat)
Q11
(T-stat)
Q12
(T-stat)
-10.7
-41.5
3
21.14
2
1.32
1
-4.10
8
-0.57
278.0
0.08
7
3.57
-192
27.48
0.06
6
4.64
-3.05
-163
5
-2.61
4
Multiple linear regression for D-class.
10
7.34
5.21
9
-0.40
-5.84
2.64
-0.36
-8.03
-0.65
-6.06
6.17
39.24
1.00
14.24
-5.45
0.18
0.69
-0.08
-8.86
10.14
-0.06
0.62
-8.52
3.03
-0.16
0.46
4.85
6.45
-4.13
23
0.23
0.68
-0.18
8.47
1.42
9.42
3.60
22
3.90
21
-2.79
20
-1.27
19
0.04
7.81
-6.57
2.72
6.50
-0.04
0.02
0.05
6.45
9.86
0.78
1.35
-4.13
9.21
-17.2
-0.36
0.24
-0.51
12.19
0.71
11.60
18
0.92
4.94
17
-4.13
0.05
16.57
0.04
3.88
7.84
1.50
0.02
-6.85
4.94
-0.03
0.02
16
3.88
15
Objective measure
0.75
-3.14
-58.3
24.24
-0.06
-0.17
0.16
-7.92
14
13
-4.55
12
-0.12
11
24
25
26
27
1.00
0.96
0.91
1.00
0.97
1.00
0.99
1.00
1.00
0.99
0.98
0.95
1.00
0.96
0.96
0.95
0.96
1.00
0.99
0.87
R^2
1.00
132.75
22.59
9.60
274.38
33.13
43323.7
106.21
2675.32
2366.59
101.39
63.99
20.21
190.00
25.68
22.59
20.21
23.47
137.63
36.80
6.49
F-stat
2717.99
Q7
(T-stat)
Q11
(T-stat)
Q12
(T-stat)
Driver 9
Q7
(T-stat)
Driver 4
Q11
(T-stat)
Driver 5
Q6
(T-stat)
Driver 8
1
6
7
8
-4.01
-8.33
0.09
3.82
-3.38
5
-0.59
4
5.59
3
0.18
2
Multiple linear regression for E-class.
-0.95
-5.60
-0.03
-9.38
2.34
0.04
10
9
11
12
18
19
3.20
5.34
1.55
-5.92
-4.27
0.89
22
-2.84
-0.17
-0.40
-0.27
21
20
3.88
17
1.10
16
2.12
15
0.13
14
Objective measure
13
23
24
25
26
27
0.99
0.89
0.82
0.96
0.99
R^2
0.93
133.18
12.65
6.80
36.21
49.11
F-stat
13.04
Links between Subjective Assessments and Objective Metrics for Steering
81
82
Driver 1
Q6
(T-stat)
Driver 2
Q10
(T-stat)
Driver 3
Q2
(T-stat)
Q3
(T-stat)
Driver 4
Q7
(T-stat)
Q12
(T-stat)
Driver 5
Q4
(T-stat)
Q8
(T-stat)
Q10
(T-stat)
Driver 6
Q10
(T-stat)
Q11
(T-stat)
Q12
(T-stat)
-7.19
-5.91
3
2.68
2
0.57
1
4
5
-42.9
320
7.43
-0.04
1.03
0.19
9
8
-9.03
7
-0.09
6
Multiple linear regression for SUV.
166
2.36
10
-0.19
-8.38
2.93
6.52
15
6.67
0.05
-7.43
52.2
14
Objective measure
13
-0.36
1.38
-95.6
12
-0.33
11
16
17
18
19
-0.32
-47.1
-2.37
-30.3
-1.91
-7.78
-31.4
0.89
23.1
0.78
6.42
-8.07
-0.11
-1.60
22
7.67
21
0.75
20
23
24
25
-3.58
-7.09
-4.07
-4.11
4.41
3.84
3.02
27
2.86
26
0.98
0.99
1.00
1.00
0.97
0.98
1.00
0.98
0.96
0.99
0.99
R^2
1.00
48.97
33.91
495.10
3325.66
29.62
31.35
7914.13
42.35
23.98
122.78
40.58
F-stat
55551.4
9
8
7
6
5
4
3
2
1
9
8
7
6
5
4
3
2
1
9
12
11
>22
>23
8
>26
2
>23
2
25~30
2
>25
<4.8
1
1
3.5~4.5
1
7
6
5
4
3
2
1
10
Driver 3
12
11
10
Driver 2
12
11
10
Driver 1
Appendix D
Subjective assessments
Subjective assessments
Subjective assessments
3
3
1~1.25
3
4
4
1.5~1.85
4
5
5
>1.4
5
>95
>90
>95
>100
-
6
6
-
>90
6
7*
7*
7*
<2.5
<2.4
<2.5
8
<2
<2
8
<2.2
8
>25
9
>25
9
9
10
>1.3
10
<1.8
10
11
11
>28.5
28~32
11
2.2~3
12
12
12
13
13
>90
>92
13
-
14
-
60~72
-
14
14
15
15
15
Objective measures
-
16
16
16
17
17
45~75
17
18*
18*
18*
>10
11~13
>13
19
10~14
>15
>16
19
19
3~6
3~6
20
4~6
20
3~6
5~6
20
21
21
21
1.5~2.2
1.5~2.2
1.5~2.2
22
>1.5
22
22
3.6~4.5
23
>3.6
>3.6
>3.7
23
23
>24
-
-
24
24
24
<13
-
-
-
25
25
25
26
<3.4
26
<3.8
<3.8
<3.2
26
Links between Subjective Assessments and Objective Metrics for Steering
83
-
-
>1.5
>1.5
27
27
>2
27
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement