Links between Subjective Assessments and Objective Metrics for Steering Xuxin He and Zhicheng Su Master Thesis in Vehicle Engineering Vehicle Dynamics Aeronautical and Vehicle Engineering Royal Institute of Technology TRITA-AVE 2012:36 ISSN 1651-7660 Postal address Visiting address Telephone Internet Royal Institute of Technology Teknikringen 8 +46 8 790 6000 www.ave.kth.se Vehicle Dynamics Stockholm Telefax SE-100 44 Stockholm +46 8 790 6500 TRITA-AVE 2012:36 ISSN 1651-7660 Links between Subjective Assessments and Objective Metrics for Steering Xuxin He and Zhicheng Su Master's Thesis in the Master's program Vehicle Engineering ©Xuxin He and Zhicheng Su, 2012 Abstract The characteristics of vehicle steering perception are decisive factors concerning vehicle safety and overall pleasure behind the wheel. It is a challenge for vehicle manufacturers to achieve these features and qualities, because usually vehicle tuning almost only relies on subjective evaluation of test drivers, which is costly and time consuming. In order to optimize suspension design and develop a tool that can be used to evaluate steering with objective metrics instead of subjective assessment, links between them must be confirmed. In this master thesis, both objective and subjective testing data of over 20 vehicles across four different segments are introduced in linear and nonlinear analysis. Linear regression analysis is applied to investigate simply positive or negative correlation between a pair of subjective-objective parameters. However, even if certain linear correlations are obtained, it is still hard to define the optimal value for objective metrics. Considering that the general shape of a correlation function can reveal which objective range give higher subjective rating, it is possible to define these preferred ranges with Neural Network (NN). The best data available is adopted from three drivers who tested 15 sedans, and some interesting results are found. The initial results demonstrate that NN is a powerful tool to uncover and graphically illustrate the links between objective metrics and subjective assessments, i.e., the specific range leading to better steering feel. Given a larger sample size, more reliable and optimal links can be defined by following the same method. These confirmed links would enable vehicle dynamics engineers to more effectively develop new vehicles with nearly perfect steering feel. Keywords: Steering feel, objective measure/metric, subjective assessment, linear regression, nonlinear correlation, Neural Network, preferred range I Acknowledgment The thesis work was performed during 2012-01 to 2012-06 at Vehicle Dynamics and Calibration (96520) of Volvo Car Corporation (VCC). We are grateful for financial and technical support from VCC. It is with great gratitude that we acknowledge the continuous guidance and advices of our supervisor Mikael Nybacka, Assistant Professor at Vehicle Dynamics of KTH Royal Institute of Technology. We also would like to thank Carl Sandberg, the supervisor at VCC, for his support and assistance. We are indebted to many people, Lars Drugge at KTH and Stefan Karlsson, Egbert Bakker as well as Kenneth Ekström at VCC, for their valuable feedback and suggestions on this thesis. Finally, our appreciation also goes to all the colleagues and friends at KTH for a wonderful time and interesting discussions about thesis and research. Xuxin He and Zhicheng Su Stockholm, June 2012 III Contents Abstract ..................................................................................................................................... I Acknowledgment ................................................................................................................... III List of Symbols .................................................................................................................... VII List of Abbreviations ......................................................................................................... VIII 1 Introduction .......................................................................................................................... 1 1.1 Background ..................................................................................................................... 1 1.2 Problem Formulation....................................................................................................... 1 1.3 Objectives ........................................................................................................................ 2 2 Method .................................................................................................................................. 3 2.1 Case Study ....................................................................................................................... 3 2.2 Linear Regression Analysis ............................................................................................. 4 2.2.1 Simple Linear ........................................................................................................... 4 2.2.2 Multiple Linear ......................................................................................................... 5 2.3 Nonlinear Regression Analysis ....................................................................................... 6 2.3.1 Artificial Neural Network ......................................................................................... 6 2.3.2 Adaptive Neuro Fuzzy Inference System (ANFIS) ................................................ 14 3 Parameter Selection and Matching .................................................................................. 21 3.1 Selection of Subjective Assessments ............................................................................ 21 3.2 Objective Measures Used in Analysis ........................................................................... 22 3.3 Initial Parameter Matching ............................................................................................ 23 4 Mathematical Modeling..................................................................................................... 25 4.1 Representation of Simple Vehicle Model ..................................................................... 25 4.2 Parameters with Time Trajectories ............................................................................... 26 5 Evaluation of Acquired Data ............................................................................................ 29 5.1 Correlation between Objective Measures ..................................................................... 29 5.2 Evaluation of Subjective Assessments .......................................................................... 32 5.3 Evaluation of Rating Tendency ..................................................................................... 35 6 Linear Regression Analysis of Subjective and Objective Data ...................................... 41 6.1 Results from Simple Linear Regression ........................................................................ 41 6.2 Results from Multiple Linear Regression ..................................................................... 45 6.2.1 Regressor Elimination Process ............................................................................... 45 6.2.2 Multiple Linear Regression .................................................................................... 48 6.2.3 Subjective Ratings vs. Objective Measures Organized by Assessment Number ... 48 6.2.4 Best Correlating Assessments ................................................................................ 49 6.2.5 Interpretation of Results ......................................................................................... 52 6.2.6 Influence of Correlated Objective Metrics in Best Assessments ........................... 52 V 6.3 Result Analysis .............................................................................................................. 54 7 Nonlinear Regression Analysis of Subjective and Objective Data ................................ 57 7.1 Initial Results from Neural Network Model ................................................................. 57 7.1.1 Data Used in NN Training ...................................................................................... 57 7.1.2 NN Training............................................................................................................ 57 7.1.3 Correlation Results ................................................................................................. 59 7.2 Result Analysis .............................................................................................................. 60 8 Recommended Procedure of Obtaining Ideal Data for Analysis .................................. 63 8.1 Track Test with Real Vehicle ........................................................................................ 63 8.1.1 Test Track ............................................................................................................... 63 8.1.2 Various Vehicles .................................................................................................... 63 8.1.3 Same Vehicle with Various Steering Characteristics ............................................. 63 8.2 Simulator Experiments .................................................................................................. 65 8.3 Suitable Test Objects ..................................................................................................... 65 8.4 Test Subjects ................................................................................................................. 66 8.5 Pre-test with a Reference Vehicle ................................................................................. 66 8.6 Normalization Work...................................................................................................... 67 8.6.1 Rating Mean Normalization ................................................................................... 67 8.6.2 Spread Normalization ............................................................................................. 67 9 Summary Results ............................................................................................................... 69 9.1 Correlation of Objective Measures and Evaluation of Rating ...................................... 69 9.2 Confirmed Subjective-objective Links for Steering...................................................... 69 10 Conclusions and Recommendations ............................................................................... 71 10.1 Conclusions ................................................................................................................. 71 10.2 Recommendations to Future Work ............................................................................. 71 Reference ............................................................................................................................... 73 Appendix A ............................................................................................................................ 75 Appendix B ............................................................................................................................ 77 Appendix C ............................................................................................................................ 79 Appendix D ............................................................................................................................ 83 List of Symbols Lateral acceleration Bias Cornering stiffness of front tires Cornering stiffness of rear tires Frequency Gravitational acceleration Lateral slip force on front tires Lateral slip force on rear tires Steering ratio between steering wheel and front wheel angle Inertia around the z-axis Front axle to center of gravity distance Rear axle to center of gravity distance Vehicle mass Steering wheel torque Curve radius Time delay Lateral velocity Longitudinal velocity Weight Front slip angle Rear slip angle Regression coefficient VII Steering angle Steering wheel angle Sum Roll angle ̇ Roll rate Yaw angle ̇ Yaw rate List of Abbreviations ANFIS Adaptive Neuro Fuzzy Inference System AVES Alliance Vehicle Evaluation Standard BP Back-propagation FIS Fuzzy Inference System KTH Royal Institute of Technology, Stockholm ME Mean error MF Membership function MIRA Motor Industry Research Association NN Neural Network OM Objective measure SA Subjective assessment SS Sum of squares VCC Volvo Car Corporation Links between Subjective Assessments and Objective Metrics for Steering 1 Introduction This chapter will give an overview of the background for this thesis. Based on the state-of-the-art of research in subjective-objective links as well as requirements from Volvo Car Corporation (VCC), specific objectives are generated. 1.1 Background Dynamics quality of a vehicle is a crucial aspect for safety and overall experience of driving pleasure. Nowadays, vehicle dynamics characteristics could vary a lot even with similar components from same suppliers. It is the appropriate calibration that makes a new product, an integration of these components, receive good evaluation about the dynamic properties from customers. Vehicle manufacturers usually use two approaches, subjective evaluation using test drivers and objective metrics, to evaluate the dynamics features of a vehicle. Though some of objective measures can be applied to describe dynamics behavior of a vehicle, they cannot indicate if these technical settings lead to good interaction between a driver and the vehicle. Thus, the subjective perception of driving or riding a vehicle is an effective measure, in addition to objective metrics, to evaluate the performance of a vehicle. Subjective evaluation of a vehicle includes various assessments to judge it as a whole. Instead of monotonous objective data, the concrete driving experience of vehicle users is revealed by drivability, handling, steering, ride comfort and some other specific perceptions. So far many companies in vehicle industry use subjective evaluation in the tuning process of vehicle development and some of them develop their own evaluation standards. AVES (Alliance Vehicle Evaluation Standard) [1] defined by Renault-Nissan Alliance have been adopted as a major item to evaluate the vehicle quality. Approximately 350 criteria concerning static and dynamic characteristics are listed in this evaluation system. The most common subjective rating scale from SAE Recommended Practice J1441 [2] has been widely applied since 1985, as seen in the figure 1 in [2]. 1.2 Problem Formulation One of the most challenging tasks for vehicle engineers is, under cost and time constraints, to satisfy every requirement of the subjective perception of customers regarding vehicle dynamics characteristics. Subjective evaluation has its own limitations when the vehicle tests need to be carried out in complicated or dangerous driving situations. Moreover, the reliability and repeatability are also somehow weaknesses for pure subjective evaluation. However, vehicle tuning still mainly relies on purely subjective evaluation of test drivers, owing to a limited knowledge of links between subjective assessments and objective metrics. This is not cost-effective during development. Therefore, there is a strong need in vehicle industry to improve the efficiency of vehicle calibration. 1 AVL List GmbH has worked with their developed tool AVL-DRIVE [3] to benchmark vehicle longitudinal drivability. The most significant information captured by this system includes vehicle velocity, longitudinal acceleration, engine speed, pedal position, vibrations and others giving over 500 objective criteria. Through analysis of driving test and calibration both research associations and vehicle industries, such as MIRA (Motor Industry Research Association) [4] and Scania CV AB [5], have already explored subjective-objective correlations in vehicle steering. Links found from such work cannot only save efforts on testing or calibration, but also help to lead to wanted subjective perception of steering feel for general customers. In this thesis the related work is used to validate our findings. 1.3 Objectives VCC has a strong need to improve development efficiency by testing fewer objective measures that are enough to define excellent vehicle dynamics characteristics in subjective perception. The aim of this master’s thesis is to explore subjective-objective correlation links based on the data measured and collected at VCC. Both linear and nonlinear correlations between subjective assessments and objective metrics will be investigated by using tools of regression analysis, variance analysis and MATLAB Neural Networks Toolbox. The main application of these results should be to tell if a vehicle would give good driving experience only according to data of certain objective measures within certain ranges. The outcome is improvement of objective and subjective measuring standards by focusing on the most significant parameters during tests, optimization of their value ranges and producing sufficient data to serve in the practical calibration work. 2 Links between Subjective Assessments and Objective Metrics for Steering 2 Method In this chapter the way to deal with data sets will be firstly described. Then, the approaches used to analyze subjective-objective correlations will be introduced. 2.1 Case Study All data used in this thesis belongs to either subjective or objective part. Each test driver did his or her own assessments on test vehicles. Therefore a number of subjective data sets of a given vehicle can be attained depending on how many test drivers are involved. As for objective metrics only one set of objective data from a specific vehicle is available by using a testing robot [6]. Normally, the average value of the subjective data set and the objective set will be used to carry out data analysis. However, this approach may not cover the true results when considering our small data size. Chen [4] introduced case study method in subjectiveobjective correlation analysis. It will be an appropriate approach to describe and explain correlation findings for each driver in this case. The way to analyze each case is shown in Figure 1. Figure 1. Approach of case study with subjective and objective data. 3 In addition, since every driver has his or her own driving preference, it is not likely to get a general quantitative conclusion even though they are all expert drivers with large testing experience. Instead, the method of case analysis will help to get a qualitative conclusion on subjective-objective correlation by studying the preference tendency of most drivers. As seen in Figure 1, assuming that Driver X is studied now, his subjective assessments and objective measures are picked to conduct linear and nonlinear analysis. Thereafter, the possible findings from this driver should be recorded. Likewise, the same analysis will be conducted for the rest of drivers. If the same linear or nonlinear correlation is presented under most cases (individual drivers), a reliable link can be confirmed. 2.2 Linear Regression Analysis Linear regression is used to model the relationship between a dependent variable and one or more explanatory variables. In both simple and multiple regression analysis, linear functions and relevant parameters are used to model the data. 2.2.1 Simple Linear Simple linear regression fits a straight line to estimate a linear model with one single explanatory variable (objective measure in our case) , seen as Equation (1), where is true value (subjective assessment in our case), is the regression value and is a random component, also called error or residual. (1). The approach used to find the best fit is to minimize the sum of the squares of the difference between the data and a line, so-called least squares minimization. The correlation coefficient [ ]) is widely adopted as a measure of the strength of linear dependence between ( two variables. Two estimators, the constant and the regression coefficient are determined by following minimization problem. The correlation coefficient can be defined as Equation (2), where , ̅ and are sample size, sample mean and sample standard deviation respectively. ∑( ̅ )( ̅ ) (2). An example of how to process the data by using the linear regression can be seen in Figure 2. A data set with five paired subjective ratings and objective data can be fitted with a regression equation . In this case, the value of regression coefficient is -0.84 that is close to -1, so the straight line fits the data set well. In other words, the regression equation can basically represent the raw data. 4 Links between Subjective Assessments and Objective Metrics for Steering An Example of Linear Regression ( r = - 0.84) y ~= -1.7x + 13 8.5 Data Fit 8 (x , y ) 3 3 y: Ratings of Parking Efforts 7.5 (x4, y4) 7 (x1, y1) 6.5 6 (x5, y5) 5.5 (x , y ) 2 5 2.8 3 3.2 3.4 3.6 3.8 x: Steering Wheel Torque [Nm] 4 2 4.2 Figure 2. Example of linear regression based on the data of a test driver. 2.2.2 Multiple Linear Multi-linear regression basically follows the same procedure to achieve best fit. The only difference is that more than one explanatory variable are introduced as seen in Equation (3). All the estimators, including regression coefficient are calculated by using [ ]), seen as least squares minimization and the coefficient of determination ( Equation (4), is a measure to study how well the true values are likely to be predicted by the model. In this expression, is the total sum of squares, while is the residual sum of squares and means the predicted model. (3). ̅ (4). For multi-linear regression models, F-statistics can be used to check the significance of regression equation and each regression coefficient in the equation. The F-value of regression equation is expressed by Equation (5), which should follow F-distribution (in this thesis, degree of freedom is the number of regressors and is the number of samples). By checking the F-distribution table with certain confidence in Appendix A the significance 5 can be verified if the calculated F-value is larger than critical value . The significance of each regression coefficient is checked in a very similar approach. The corresponding F-value of a regression coefficient is calculated by following Equation (6), where and are the new sum of squares for the regression equation without coefficient . . Likewise, the significance can be verified only if ⁄ ⁄ ⁄ is larger than (5). (6). 2.3 Nonlinear Regression Analysis Results from linear correlation basically only tell if there is monotonically positive or negative effect on some subjective assessments from certain objective measures. If an optimal range needs to be defined for an objective measure that will give the best subjective perception, then some other methods have to be used to find nonlinear correlations between them. 2.3.1 Artificial Neural Network Generally, Fuzzy Logic and Neural Network (NN) are frequently used tools to find out nonlinear correlations between different parameters. The former one can be adopted for nonmetric subjective feel, such as ‘good/bad’ or ‘heavy/medium/soft’. The latter one can be used while subjective ratings are in numbered form, which is available in this thesis. Thus, the Neural Network Toolbox in Matlab is utilized. NN is a kind of mathematical structure that composes of interconnected artificial neurons to imitate the way a biological neural system works (e.g., our brain). It has the ability to learn from data and can be used to explore nonlinear subjective-objective links in this thesis. A typical multi-layer neural network consists of an input layer, hidden layer and output layer of neurons. Its structure is graphically demonstrated as seen in Figure 3. 6 Links between Subjective Assessments and Objective Metrics for Steering Figure 3. Structure of three-layer static Neural Network. The input metrics is transmitted through the connections by first multiplying the scalar weight to form a product . Secondly, it will be added with a scalar bias and processed by a transfer function . Three types of transfer functions are most commonly used. They are called hard-limit, linear and tan-sigmoid, shown in Figure 4. Tan-sigmoid is usually adopted as the transfer function in the hidden layer to find out nonlinear subjectiveobjective correlations. Linear transfer function will be used in the output layer. The calculated results from the hidden layer will then be added in the output layer, where output is produced and can be compared with target value of subjective ratings. The main principle of neural network is that scalar weight and bias can be adjusted in order to obtain desired results [7]. Figure 4. Three types of most commonly used transfer functions. 7 There are two types of NN, feed-forward (static) and feed-back (dynamic). Signal flows of both of them start from the input layer and end at the output layer. One of the main differences between them is if the output depends on the previous output or not. Feedforward NN is mostly applied to model the system where the network is stable and the output is only determined by input signals. This confirms exactly to the case of this thesis. In a normal NN, there are several layers with neural nodes, including hidden layer and output layer, transfer functions of layers and neurons in each hidden layer. In addition to these main components and constructions, learning algorithm parameters and learning rate have to be selected before training NN. The parameters that need to be decided are the following: Type of NN connectivity As mentioned before, neural connections can be either feed-back or feed-forward. In this thesis, the outputs or subjective ratings of drivers are only decided by the corresponding objective data, and no previous outputs are needed as in the case of feed-back system. Thus, the feed-forward (static) NN is chosen. Number of hidden layers The number of hidden layers varies from zero to two. Without hidden layers, NN does not differ a lot from linear regression, while too many hidden layers contribute to over-fit. Research of Ash [8] also shows that one hidden layer can be suitable for this kind of problem. Learning algorithm The main learning algorithms available in NN can be divided into three types, supervised, unsupervised and reinforcement learning. In the supervised learning, corresponding targets are provided with each input; while in the unsupervised learning, no such output targets exist. The reinforcement learning concerns with how an agent in an environment should react so as to maximize rewards feedback from the environment, and this algorithm differs from the supervised one in a way that no input/output mapping is presented. In this thesis, since both numerical input and target output are available, the supervised learning is suitable to be adopted, where the error correction will be used to minimize the error between output and target by modifying weight and bias. In the field of studying subjective-objective links, the most often used algorithm is back-propagation compared with other algorithms such as perception, self-organizing, recurrent, etc [9]. Another important aspect is that the data used for training and testing the NN should be different. This can be achieved by setting the dividing parameters, which decides the ratio of vectors for training and testing as well. 8 Learning algorithm parameters (learning rate and momentum factor) Links between Subjective Assessments and Objective Metrics for Steering In a NN, the appropriate learning rate and momentum factor should be decided before training. Too small learning rate leads to extremely long training time, while too large one makes it hard for the NN to converge. The well-specified momentum constant helps to reduce the learning time, which decides the proportion of previous weight update to make use of. Problems of insignificance or instability could occur if the momentum factor is set too low or too high. Since unfortunately no systematic method can be used to determine them, both are set relatively low so as to guarantee the convergence of the NN. Initial weight value Initial weight values are decided by experience, and this setting will not make great fundamental difference in generalization quality between using optimal initial weights and random initial weights [10]. Thus, the initial weight value is set to random values. Transfer function The transfer function in the hidden layer is set as tan-sigmoid function, which is considered able to represent the relationships best [11]. The transfer function in the output layer is just to produce the final output by adding all from former hidden layer together, and since the rating range is scaled between 0 and 10, the linear function is selected so that the output value can be transferred into normal range (from 0 to 10), i.e., if the assessment is done by judging ‘good or bad’, the step function can be adopted. Number of neurons in each hidden layer With more neurons in each hidden layer, the NN is capable of training the models with higher complexity. The fit of training gets better with more complicated NN system. However, the trained network with better fit (more neurons) usually over-fits the model, which leads to larger error between predicted and true ratings for the testing data [11]. Three neurons in the hidden layer are considered to be appropriate for the case studied in this thesis. Trial Neural Network design with another data sample: After deciding NN learning process, some example data from another similar research at KTH is introduced to design a trial NN as follows: Example data with 16 configurations (16 samples) Steering feel (SF) is the assessment Five objective parameters Data with speed at 80km/h Data from 15 expert drivers (with similar driving experience to VCC data) Feed-forward NN with gradient descent back-propagation (BP) training function One hidden layer with tan-sigmoid transfer function (tansig) One output layer with linear transfer function (purelin) Three neurons in the hidden layer 9 Maximum epochs: 500 Goal error: Learning rate: 0.01 (default value) Momentum factor: 0.9 (default value) Ratio of vectors for training: 75% (12 samples) Ratio of vectors for testing: 25% (4 samples which reach the minimum statistic number) The single-input network is used to find the preferred range of an objective parameter where higher subjective rating can be expected. As seen in Figure 5, the output is steering feel and the input signal is one of five objective measures. Implementing this training process for every driver with each objective parameter results in regression coefficients that are collected and shown in Table 1. When r-value is larger than 0.7 it is marked as red to indicate a model with successful training. Figure 5. Single-input Neural Network structure with one-hidden-layer and three neurons. Table 1. Overview of training regression (r-value) for expert drivers at 80km/h. Expert driver No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Max. lat. 0.30 0.40 0.30 0.20 0.40 0.50 0.40 0.50 0.40 0.20 0.20 0.20 0.30 0.30 acceleration Steering 0.40 0.40 0.40 0.40 0.50 0.80 0.40 0.60 0.30 0.30 0.30 0.50 0.20 0.10 sensitivity Steering 0.60 0.50 0.80 0.60 0.80 0.30 0.40 0.50 0.40 0.50 0.80 0.30 0.30 0.30 stiffness Torque 0.60 0.50 0.70 0.50 0.70 0.40 0.30 0.80 0.30 0.80 0.30 0.10 0.30 0.10 gain Yaw 0.60 0.10 0.10 0.50 0.50 0.20 0.10 0.40 0.50 0.30 0.30 0.40 0.10 0.20 delay 15 0.10 0.50 0.60 0.60 0.20 Most training results with good fit (r-value ≥ 0.7) concern steering stiffness and torque gain, which are defined in Figure 6, and they are similarly defined as torque buildup into the corner in VCC DNA files. If close examination is done, the relationship between these 10 Links between Subjective Assessments and Objective Metrics for Steering parameters and steering feel rating should be like Figure 7, where certain preferred or dislike range is visualized. Figure 6. Definitions of steering stiffness and torque gain. Neural Network training result for Expert Driver 3 @ 80km/h (16 configruations) 3.2 Real values Predicted values 3 SF(steering feel) rating 2.8 2.6 2.4 2.2 2 1.8 1.6 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Steering stiffness [Nm/deg] Figure 7. Relationship between steering feel and steering stiffness for Driver 3. 11 According to the figure above, the preferred steering stiffness is from 0.06 to 0.12Nm/deg, which provides Driver 3 with better steering feel. Table 2 shows all preferred ranges of different objective parameters from well-trained models. The main findings concern steering stiffness and torque gain, both of which are preferred to be low to achieve better steering feel. Thus, it is possible to find preferred range through NN, especially when the linear correlations have already been confirmed. Table 2. Preferred range for better SF rating (expert driver). Objective measure Preferred range for higher rating Max. lat. Acceleration NA Steering sensitivity NA Steering stiffness 0.06~0.12 Nm/deg Torque gain 0.05~0.15 Nm/deg Yaw delay NA Apart from expert drivers, the data from other 15 novice drivers are also used to train NN model. The valid findings are also focusing on steering stiffness and torque gain. The preferred ranges of both parameters are almost exactly the same as expert drivers. Another issue to consider is if the sample size could be smaller to save test time and cost. It depends on how the NN processes data. The NN system chooses the training data and testing data randomly even though the ratio of vectors for training and testing is defined. This fact makes the trained vectors and tested vectors never fixed. Moreover, the data used for testing is always much less than that for training. As a result, sometimes if extreme outliers are included in vectors for testing, the regression will get quite unstable. The regression plots of test in Figure 8 show how much the test regression could vary with different selected data in training and testing. 12 Training: R=0.73831 Data Fit Y=T 3 2.5 2 2 2.5 3 Output ~= 0.28*Target + 2.1 Output ~= 0.53*Target + 1.2 Links between Subjective Assessments and Objective Metrics for Steering Test: R=0.93766 3 Data Fit Y=T 2.8 2.6 2.4 2.2 2 2 2.5 Output ~= 0.55*Target + 1.2 Target 3 Target All: R=0.73893 Data Fit Y=T 3 2.5 2 2 2.5 3 Training: R=0.72449 Data Fit Y=T 3 2.5 2 2 2.5 3 Output ~= 0.57*Target + 1.1 Target Output ~= 0.27*Target + 1.9 Output ~= 0.58*Target + 1.1 Target Test: R=0.096104 3 Data Fit Y=T 2.8 2.6 2.4 2.2 2 2 2.5 3 Target All: R=0.64615 Data Fit Y=T 3 2.5 2 2 2.5 3 Target Figure 8. Different regression-plots under the same data samples. As seen in Figure 8, for the regression of training data and all data (the blue and red lines), the fit is relatively stable (r-value of all samples fluctuates by 0.1 units). On the other hand, the fit of testing data (the green line) varies remarkably (0.94 vs. 0.096). When the r-value is 0.94 in the first case, the randomly selected testing data by the NN can fit with predicted 13 model very fortunately. On the contrary, due to the small testing sample, if extreme data are selected for testing, the r-value will get worse, i.e., in the second case the r-value is just 0.096. With 16 samples in all, the fit of training model is stable. However, the error between true value (target) and predicted model (output) will vary randomly for the testing part (only 4 samples). The point is that 16 samples are just enough to train the NN model, but sometimes the stability of testing cannot be ensured. If the sample number is reduced slightly, the reliability of preferred range through NN training will not be guaranteed anymore. In other words, at least 16 test objects (cars) or different settings of the cars are needed in order to use NN to find nonlinear correlations. As for how many test drivers are needed in this kind of study, it will be discussed later on. Furthermore, there is a possibility to add to the data set gathered data from a later test, meaning that if 16 settings could be varied during one test day data from another day with different cars and/or settings can be added in order to reach a better validity of the NN model. 2.3.2 Adaptive Neuro Fuzzy Inference System (ANFIS) The fuzzy logic system is able to model nonlinear functions as well by mapping inputs into a logic space first, and then inferring some rules to obtain wanted output. However, an ordinary fuzzy logic system cannot learn rules without support from Neural Network. That is why Adaptive Neuro Fuzzy Inference System (ANFIS) is needed. ANFIS uses a hybrid learning algorithm to identify the membership function parameters of single output, with first order Sugeno-type Fuzzy Inference System. The Tagaki-Sugeno fuzzy if-then rules in ANFIS can be configured as follows: The nth rule: if is and is and is , then where , , and are linear parameters of function and signals. , , and , are input A normal structure of ANFIS is shown in Figure 9, totally with six distinct layers [12] [13]: For instance, with this illustrated ANFIS, the 5th rule and the fifth function are valid when is and is and is . 14 Links between Subjective Assessments and Objective Metrics for Steering Figure 9. ANFIS structure with first-order Sugeno model. Layer 1 is the input layer, simply giving incoming signals, can be objective measures. Layer 2 is the fuzzification layer. The output of each node in this layer is decided by the membership function (MF). For example, given a triangular MF (trimf) shown in Figure 10, it can be presented as Equation (7). (7). { } 15 Figure 10. Membership function: trimf. Layer 3 is the rule layer. Each node in this layer computes the weight of the rules can be decided by orthogonalizing various inputs and MFs. for the nth rule. All Layer 4 is the normalization layer. The normalized firing strength of a given rule is the output of this layer defined as the ratio of the firing strength of the considered node to the sum of all rule firing strengths. The normalized firing strength for the nth node is represented as Equation (8). ̅ (8). Layer 5 is the defuzzification layer. The output of Layer 4 and initial inputs are received and the weighted consequent value of a given rule is calculated as Equation (9). ̅ ̅ (9). Layer 6 is the layer with one summation node, where adding up all outputs of the previous layer produces the single ANFIS output. ∑ ̅ (10). The ANFIS training adopts a gradient descent algorithm to optimize the antecedent parameters and a least squares algorithm to solve for the consequent (linear) parameters. Similar to the procedure of NN, parameters such as MF type, goal error and maximum epochs have to be empirically determined before training. Above all, the number of MFs for each input is extremely important, which decided how many logic divisions should be generated. 16 Links between Subjective Assessments and Objective Metrics for Steering Concerning subjective-objective correlation where subjective assessments can be seen as output of the ANFIS and if a certain preferred range exists there should exist upper and lower boundary for a given objective measure. That means only those values in the preferred range will give better perception seen as Figure 11. So the objective measuring data should be divided into three logic zones (three MFs), two of them giving lower rating score. However, in the actual case, extreme values outside the boundaries might not be acquired on testing vehicles. As a result, one or even two logic zones should be removed. Therefore, the number of MFs in ANFIS training cannot be chosen in order to study subjective-objective links unless extreme cases are surely detected. Figure 11. General shape of subjective-objective correlation function. ANFIS training with subjective rating of a certain level and of its sub-level In order to test the ANFIS model a collection of subjective data from a sub-level is introduced with rating range from 100 to 500-points that will replace the objective data in Figure 11. All assessments in the sub-level are considered to have effect on the subjective assessment (top-level) as seen in Figure 12. The 100 or 500-point ratings represent extreme cases. Using these data samples it will be possible to demonstrate how ANFIS works and if the parameters from the sub-level are enough to represent some property. 17 Figure 12. Explanation of sub-level for subjective assessment. All four various assessments in the sub-level are included in the fuzzy inference system since each sub-level assessment should give its contribution. With three MFs in each of them shown in Figure 13, the FIS part can be represented as seen in Figure 14. Figure 13. Example of membership function with Gauss MF (three MFs). Figure 14. Fuzzy Inference System (FIS) part. 18 Links between Subjective Assessments and Objective Metrics for Steering However, due to more complicated structure of ANFIS as seen in Figure 9 when using multiinput, there will be hundreds of unknown parameters needed to be inferred, including linear and nonlinear ones. It turns out that no convincing results can be acquired unless a huge database is provided. Thus, the ANFIS tool could be another option in the future when sufficient data set is given. 19 20 Links between Subjective Assessments and Objective Metrics for Steering 3 Parameter Selection and Matching In this chapter all raw data will be looked through and preprocessed. Some parameters in the raw datasheets are going to be excluded since they are either lacking data samples or not relevant to this thesis. Explanatory subjective assessments will be initially mapped to corresponding objective measures before linear and nonlinear analysis. Raw objective data from VCC includes four different vehicle classes and subjective data are also made up of same classes. Each test group includes several drivers, but the drivers do not remain the same across different tests (e.g., Driver 3 in C-class is not the same one in Dclass). The focus of this thesis is steering characteristics. In addition, some of important data is lacking for drivers or vehicles. For example the testing data is missing for majority of vehicles regarding certain objective measures that will not be considered in analysis. So the work of reorganizing and matching the raw data is necessary. The classification can be seen in Table 3. Further analysis will be based on each vehicle class. One important aspect that has to be kept in mind is that in some classes not all test drivers did their judgment on every subjective assessment and each vehicle model. That is, the sum of drivers is not consistent with total runs of each vehicle. Table 3. Overview of test vehicles in each class. Classification Vehicle 1 Vehicle 2 Vehicle 3 Vehicle 4 Vehicle 5 Vehicle 6 Vehicle 7 Sum of vehicles C-class C1 C2 C3 C4 C5 C6. - 6 D-class D1 D2 D3 D4 D5 - - 5 E-class E1 E2 E3 E4 E5 E6 E7 7 SUV SUV1 SUV2 SUV3 SUV4 SUV5 - - 5 3.1 Selection of Subjective Assessments The emphasis in this thesis work should be focused on the vehicle steering characteristics. Twelve subjective assessment indexes shown in Table 4 will be studied. However, the data under Level 2 (steering disturbances/error states) is not taken into account because there is no corresponding objective data available. Thus, these subjective assessments are not considered while calculating the rating for Level 1 (steering). 21 Level 1 Steering Table 4. Selected subjective assessments. Level 2 Level 3 First 50m Test First 50m test Efforts Park/Maneuvering Returnability Response Roll Control Straight Ahead Controllability Torque Feedback Modulation Response Roll Control Cornering Controllability Torque Feedback Returnability Modulation Assessment SA-1 SA-2 SA-3 SA-4 SA-5 SA-6 SA-7 SA-8 SA-9 SA-10 SA-11 SA-12 3.2 Objective Measures Used in Analysis First of all, if there is lack of data in some objective measures, they have to be eliminated in order to keep the uniformity on objective data. After that the objective measures that have little effect on subjective assessments of drivers are allowed to be removed, such as (lat. acc. resp. gain @ max load) and (sine time lag @max load). The reason is that the case with maximum load was never achieved at the subjective data collection scenarios. With these aforementioned work done, the standardized objective measures can be utilized in the following analysis. Totally, 27 categorized parameters are shown in Table 5. Table 5. Overview of all needed objective measures. Straight Ahead Controllability Level 2 Level 3 22 Level 4 Window SWA At 0.05 g Response Gain Straight Path On Center Yaw gain straight Lateral Acc. Resp. Gain Overall Steering Sensitivity Response Lateral Acc. Resp. Gain Overall Steering Sensitivity Gain Linearity Steering Sensitivity Ratio Response Time Delay Yaw 45° Phase Lag Time Roll Control Straight Path Roll Control Total Rollrate Gradient @ 1 Hz Torque Deadband SWA at 1.3 Nm Torque Feedback Torque Build Up Torsional Rate Unit [] Measure OM-1 [⁄ ⁄ ] OM-2 [ ⁄ ] OM-3 [ ⁄ ] OM-4 [ [ ] OM-5 [ OM-6 ] [ ⁄ ⁄ ] OM-7 [] OM-8 ⁄ ] OM-9 50m test (first impression) Cornering Controllability Links between Subjective Assessments and Objective Metrics for Steering Friction Feel Torque at 0 g Yaw Response Gain Off Center Yaw Gain Response Gain Understeer Linear Range Understeer Gradient Response Gain Linearity Yaw Gain Linearity Rel. yaw [email protected] lat. Acc. Response Yaw [email protected] lat acc/max yaw gain Sine Time Lag Yaw - SWA phase time lag @ 4m/s2 Sine Time Lag Ay - SWA phase time lag @ 4m/s2 Sine Time Lag Ay - Yaw phase time lag @ 4m/s2 Roll Control Cornering Roll Control Total Roll Gain Torque Buildup Into The Corner Torsional Rate Cornering Torque Buildup Cornering Off Center Torque Gradient On Center Hysteresis Torque Feedback Torque Deadband in Degrees Off Center Hysteresis Torque hysteresis @ 0.3 g Effort Level Torque @ 0.3 g Low Speed Response Gain On Center Yaw Gain Low Speed Torque Buildup Max. Torsional Rate Parking Efforts Standstill Parking Efforts Near Center Parking Efforts Rolling Parking Efforts Just Off Center [ ] [⁄ ⁄ ] OM-11 [⁄ ] OM-12 [ ] OM-13 [⁄ ⁄ [ OM-10 ] OM-14 [ ] OM-15 [ ] OM-16 [ ] OM-17 [⁄ ] OM-18 ⁄ [ ] OM-19 ⁄ ] [] OM-20 OM-21 [ ] OM-22 [ ] OM-23 [⁄ ⁄ ] OM-24 [ ] OM-25 ⁄ [ ] OM-26 [ ] OM-27 3.3 Initial Parameter Matching Having decided the subjective and objective parameters to be studied, corresponding group mapping are set before conducting the analysis. Not all of the objective inputs should be studied regarding a specific subjective assessment, since only those parameters that are measured under the same test environment should be considered. For example the SA-4, response about straight-ahead controllability, the corresponding objective data should come from OM-1~OM-6, which are listed in Table 5. Besides, subjective ratings on Level 2 are actually calculated by using the data from Level 3 and expressed in the form of the average value of them. In other words, these values in Level 2 do not directly depend on objective measures. Instead, they are merely decided by the subjective judgment at next level (Level 3). That is why the subjective assessments on Level 2 are not included in Table 6. Table 6. Corresponding setting for each subjective assessment. SA Subjective Level 3 1 First 50m test Objective Level 4 Low Speed Response Gain [°/s/100°SWA] Low Speed Torque Buildup [Nm/100°SWA] OM 24 25 23 24 2 Efforts 3 Returnability 4 Response 5 Roll Control 6 Torque Feedback 7 Modulation 8 Response 9 Roll Control 10 Torque Feedback 11 Returnability 12 Modulation Parking Efforts Standstill [Nm] Parking Efforts Rolling [Nm] Parking Efforts Standstill [Nm] Parking Efforts Rolling [Nm] Window [deg] Response Gain Straight Path [°/s/100°SWA] Lateral Acc. Resp. Gain [°/s/100°SWA] Lateral Acc. Resp. Gain [°/s/100°SWA] Gain Linearity [-] Response Time Delay [ms] Roll Control Straight Path [°/s/g] Torque Deadband [°] Torque Build Up [Nm/100°SWA] Friction Feel [Nm] Window [deg] Response Gain Straight Path [°/s/100°SWA] Lateral Acc. Resp. Gain [°/s/100°SWA] Lateral Acc. Resp. Gain [°/s/100°SWA] Gain Linearity [-] Response Time Delay [ms] Torque Deadband [°] Torque Build Up [Nm/100°SWA] Friction Feel [Nm] Yaw Response Gain [°/s/100°SWA] Response Gain Understeer [°/g] Response Gain Linearity [%] Rel. yaw [email protected] lat. Acc. Sine Time Lag Yaw-SWA [ms] Sine Time Lag Ay-SWA [ms] Sine Time Lag Ay-Yaw [ms] Roll Control Cornering [deg/g] Torque Buildup Into The Corner [Nm/100°SWA] Torque Buildup Cornering [Nm/g] On Center Hysteresis [°] Off Center Hysteresis [Nm] Effort Level [Nm] Torque Buildup Into The Corner [Nm/100°SWA] Torque Buildup Cornering [Nm/g] On Center Hysteresis [°] Off Center Hysteresis [Nm] Effort Level [Nm] Yaw Response Gain [°/s/100°SWA] Response Gain Understeer [°/g] Response Gain Linearity [%] Rel. yaw [email protected] lat. Acc. Sine Time Lag Yaw-SWA [ms] Sine Time Lag Ay-SWA [ms] Sine Time Lag Ay-Yaw [ms] Torque Buildup Into The Corner [Nm/100°SWA] Torque Buildup Cornering [Nm/g] On Center Hysteresis [°] Off Center Hysteresis [Nm] Effort Level [Nm] 26 27 26 27 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 19 20 21 22 23 11 12 13 14 15 16 17 19 20 21 22 23 Links between Subjective Assessments and Objective Metrics for Steering 4 Mathematical Modeling In this chapter the bicycle model as well as some time trajectories of steering wheel angle, yaw rate, lateral acceleration and steering wheel torque will be introduced. These models and parameters set the basis of vehicle dynamics system. It is necessary to clarify that the model-based analysis is not used to calculate the value of a specific objective parameter. The goal of the mathematical modeling is to help understand and explain the data and findings of this thesis work. 4.1 Representation of Simple Vehicle Model To build and analyze an ideal simple vehicle model, the basic assumption made about the vehicle and its driving environment is adopted. The height of the center of gravity over ground is zero. The only external force affecting the motion of vehicle is the tire force and the side force is linear functions of the slip angles. The slip angles are much smaller than 180°. There is constant speed at stationary driving. Figure 15. Bicycle model (single track model). 25 The most simplified vehicle model, bicycle model or also called single track model, is a three-degree-of-freedom model as seen in Figure 15. It only represents the horizontal motions in the X-Y plane. The matrix form is generally used to express the equations of motion as seen in Equation (11). ( ̇) ( ) (11). ( ) If the driving maneuver is considered to be a simple stationary case, the definition for steering sensitivity ̇ and understeer can be expressed as in Equation (14) and (15): ̈ ̇ (12). ̇ (13). ̇ (14). (15). In stationary driving, the definition of yaw response gain is similar to steering sensitivity, and with Equation (13), the expression for overall steering sensitivity is given in Equation (16). The way to transfer understeer to response gain understeer is shown as Equation (17). ̇ [ ] (16). (17). These two expressions show that theoretically, overall steering sensitivity and response gain understeer should correspondingly follow steering sensitivity and understeer. 4.2 Parameters with Time Trajectories All the objective metrics can be divided into two types, instant values or time trajectories. In this thesis, time trajectories are used to show and understand the time lag, deadband or 26 Links between Subjective Assessments and Objective Metrics for Steering hysteresis. van Daal [14] introduces all time trajectories regarding friction and compliance in steering system. Examples of these characteristics can be seen in Figure 16 to 18. Figure 16. Time trajectories of steering wheel angle and lateral acceleration. Figure 17. Time trajectories, lateral acceleration versus steering wheel torque. 27 Figure 18. Time trajectories, steering wheel angle versus steering wheel torque. In Figure 16, there is a time lag between the two time trajectories. Because vehicles need to take a short time period to overcome the inertia and compliance of steering system, compliance of suspension and tire relaxation delay. The vehicles cannot respond immediately with a steering wheel input, and this is also referred to as response time lag. In Figure 17, friction feel indicates the needed torque to initiate turning when driving at straight path, while off-center hysteresis shows the needed effort to correct the steering in cornering. In Figure 18, on-center hysteresis indicates when drivers will feel resistance torque when changing steering wheel direction and passing the center region. Torque deadband shows the feeling of play in steering. 28 Links between Subjective Assessments and Objective Metrics for Steering 5 Evaluation of Acquired Data In this chapter the goal is to examine the objective-objective correlations and the validity of subjective data. The results could be used to eliminate some unnecessary regressors in the correlation analysis. 5.1 Correlation between Objective Measures The purpose of correlation coefficient analysis is to find correlations between two objective measures (Level 4) that are under the same level (Level 3). These correlations can be either positive or negative. By doing this with all vehicles, some similar or different tendencies regarding correlation between different classes are supposed to be attained. Such information can be used to take decisions on whether to disregard some objective measures in the initial studies. The example of C-class is shown in Figure 19. The correlation coefficient between any two different objective measures is calculated. After that, all absolute values larger than 0.7 are shaded dark in the grid. The statistics for the rest of the vehicle classes are tabulated in Appendix B. R-value of Obj vs Obj Objective measures 1 2 3 4 5 6 7 8 Objective measures 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Figure 19. Correlation coefficient between two objective measures for C-class (| | shaded). is 29 Table 7. Overview of correlations between two objective measures (negative correlations are shaded in gray). Test Straight 1-4 2-3 2-4 Correlated 3-4 parameter 9-10 C-class Corner 50m 11-14 26-27 12-14 16-17 20-22 D-class Straight Corner 50m 1-2 11-12 25-27 1-4 12-14 2-3 19-20 2-4 19-22 3-4 20-22 8-9 20-23 8-10 22-23 9-10 E-class Straight Corner 50m 1-2 11-12 25-26 1-3 11-14 25-27 1-4 16-17 2-3 19-22 2-4 19-23 3-4 SUV Straight Corner 50m 1-5 16-17 26-27 2-4 19-21 2-5 19-23 4-5 22-23 8-10 9-10 By observing Table 7, where all found objective-objective correlations are listed by different testing type. In the test of driving straight-ahead, for all non-SUV classes, OM-1~OM-4 are cross-correlated, except for that correlation between OM-1 and 3 is not that strong (from 0.78 to -0.37). Moreover, OM-1 (window) only shows negative correlation with three other parameters. In cornering test, OM-11, 12 and 14 tend to correlate with each other while no unequivocal sign is confirmed except that OM-11 always reveals negative correlation with OM-12, and still SUV class does not display such properties. In all non-SUV classes except OM-21 all torque feedback related parameters (OM-19, 20, 22 and 23) somehow show positive correlation with each other, especially between OM-19 and OM-23 as well as between OM-22 and OM-23, which can also been found in SUV. Across all four segments correlation between time lag 16 and 17 is nearly confirmed, although the r-value is 0.66 in the D-class. In addition at 50m test OM-27 is either correlated with OM-25 (in C and E classes) or with OM-26 (in B and SUV classes). The conclusions that can be drawn from the analysis in this section are: 30 For all sedans, yaw response gain and lateral acceleration response gain on the straight path have positive correlation because of the relationship as seen is Equation (13) and (16) according to the bicycle model. Window indicates the sensitivity of steering wheel before the turning can be felt. So the lower value of this parameter will give higher sensitivity (nervous property), which leads to faster response of both yaw gain and lateral acceleration. That is why the negative correlation is acquired between window and the rest of the three parameters. The negative correlation between off-center yaw gain and response gain understeer is understandable. Since response gain understeer almost follows the change of understeer as seen in Equation (17) the yaw rate will grow more slowly if the car is more understeered. Links between Subjective Assessments and Objective Metrics for Steering Among three kinds of time lags there should be a relationship as in Equation (18). In normal driving the slip force on tires is produced as long as a steering maneuver is transferred to tires, so the yaw motion is present. It will be followed by the centripetal force (lateral acceleration). In this thesis, only sine time lag Ay-SWA and Ay-Yaw give positive correlation. If a close examination is taken into these three objective measures the time lag Yaw-SWA is mostly located around 70ms to 80ms while the lag Ay-Yaw varies from 20ms to 100ms. This means the main difference of time lag Ay-SWA results from lag Ay-Yaw, thus they are correlated. (18). The most correlations are found in torque feedback in cornering. It is easy to understand that torque buildup and torque at 0.3g-lateral acceleration are correlated. Besides, according to Figure 5, larger off-center hysteresis also increases torque gradient and needed effort in the curve. On the other hand on-center hysteresis is in closer association with the on-center properties with low lateral acceleration so as not to have clear correlation with other parameters in torque feedback in cornering. For all vehicles during parking the needed torque at rolling speed should have correlation with either low speed torque buildup or the steering torque in the state of standstill. That means at least one of these two metrics will decide the steering torque at rolling. Comparison with other research: The Table 5 in [5] shows the confirmed correlation between various handling parameters. In that research, the characteristics that differed in each simulation test are up to four in the steering system, including friction, damping, stiffness and inertia. Except for these variables, the rest of the vehicle properties in the simulator remain completely the same. It can be sure that all differences in test results are due to these varied parameters. If there are certain relationships between two objective measures, the specific correlations should be fixed. As a result several correlations are discovered and the correlation coefficients are high (from 0.95 to 1.00) enough to do reduction, even though they come from different tests. That means these metrics nearly follow the same pattern and it does not matter which one should be eliminated. Back to this research, test objects are replaced by real vehicle models that possess totally distinct parameters, rather than only several varied parameters. In other words, even if all cars follow the same driving maneuver, the way they behave and their specific correlations between objective measures will be absolutely different. It turns out that few correlations with high coefficients (close to 1.0) could be found in this thesis. Those correlated objective measures with relatively low | |-value (not close to 1.0) cannot guarantee fixed change tendency between them. Besides, results in this thesis are based on real vehicle tests instead of simulation platform. So the setting of objective measurement is not close to that in [5]. 31 When the correlation coefficient is higher than the critical value 0.7 but not sufficiently close to 1 it is not so convincing to decide which one should be left in further analysis. So based on the comparison, these found correlations can only be used to improve the objective metrics instead of to remove those correlated metrics in regression process. The improvement of such objective measure should be focusing on the metrics with some findings (OM-1, 2, 3, 4, 11, 12, 16, 17, 19, 20, 22, 23, 25, 26 and 27). By doing tests with different configurations on one single fixed vehicle model and if some higher correlation coefficients are acquired it will be convincing to remove any one of the correlated measures to save the efforts and time spent on tests. 5.2 Evaluation of Subjective Assessments Correlation analysis of subjective data cannot be carried out like the objective data. The main reason is that there are not enough subjective data used to conduct regression analysis for each driver. For instance, 5 out of 9 drivers in E-class only tested 3 or 4 out of 7 cars in total. Obviously, the findings of possible internal correlation between subjective assessments based on inadequate data probably lead us to draw the wrong conclusions. From this perspective, those subjective data from test drivers who only provided 3 or 4 ratings in each vehicle class will be abandoned when looking for subjective-objective correlation. These insufficient data sets are listed and shaded in Table 8. Table 8. Test vehicle by each driver in every vehicle class. Vehicle Number (different models in different vehicle classes) Class Driver No. Vehicle 1 Vehicle 2 Vehicle 3 Vehicle 4 Vehicle 5 Vehicle 6 Vehicle 7 Driver 1 × × × Driver 2 × × × Driver 3 × × × × × × Driver 4 × × × × × × Driver 5 × × × × × × C Driver 6* × × × × × × Driver 7 × × × Driver 8 × × × Driver 9 × × × × × × Driver 10 × × × Driver 1 × × × × × Driver 2 × × × × × Driver 3 × × × × × D Driver 4 × × × × × Driver 5 × × × × × Driver 6 × × × × × Driver 7 × × × × × Driver 1 × × × Driver 2 × × × × × × × E Driver 3 × × × Driver 4 × × × × × × × 32 Links between Subjective Assessments and Objective Metrics for Steering Driver 5 Driver 6 Driver 7 Driver 8 Driver 9* Driver 1 Driver 2 Driver 3 SUV Driver 4 Driver 5 Driver 6 × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × In this thesis the mean ratings and their 95% confidence intervals for 12 subjective assessments can be used to roughly examine the validity of the subjective ratings. To be concrete, the interval with 95% confidence level should be within the subjective rating range, which is from 1 to 10. In every vehicle class, there are several ratings from six to ten test drivers on each subjective assessment. Take the subjective ratings in D-class as an example. There are seven test drivers who did subjective evaluation for all the five vehicles in this class. As shown in Figure 20 it seems that the mean ratings for each car scatter around a reference value, roughly 7-point for the assessments of D1. In this case, the spread of the confidence intervals all fall in the subjective rating range. Thus, the subjective data can be considered as basically valid for further analysis. However, there is no obvious rating pattern of the drivers found according to this figure. It is impossible to profile preference of drivers in subjective rating based on a small number of tests. 33 7-Drivers 7-Drivers 10 7-Drivers 10 7-Drivers 10 7-Drivers Mean values and 95% Confidence Interval of Ratings 10 10 8 Vehicle D1 6 4 1 2 3 4 5 6 7 8 9 10 11 12 8 Vehicle D2 6 4 1 2 3 4 5 6 7 8 9 10 11 12 8 Vehicle D3 6 4 1 2 3 4 5 6 7 8 9 10 11 12 8 Vehicle D4 6 4 1 2 3 4 5 6 7 8 9 10 11 12 8 Vehicle D5 6 4 1 2 3 4 5 6 7 8 9 10 11 12 Assessment Number Figure 20. Mean value and 95% confidence interval of ratings for D-class. To investigate the validity of ratings, the rating spread of drivers on a specific vehicle should be examined. Figure 21 illustrates ratings of each driver on the vehicles in D-class. Each shaded rectangle shows rating range of a driver on a specific vehicle, where the top line indicates the highest rating while the bottom line stands for the lowest rating. For instance, Driver 1 has a larger rating difference on every vehicle, while Driver 6 has a rather narrower rating range compared with other drivers. Whereas, this small rating tendency still can be accepted. One reason could be that these drivers show consistent rating habit across all the vehicle models. If the rating spread of a driver fluctuates a lot with the given vehicles within the same segment, the ratings of him or her cannot be considered reliable. 34 Links between Subjective Assessments and Objective Metrics for Steering Subjective Ratings of Every Vehicle Model 10 8 7.8 8.0 1 2 7.0 7.0 3 4 7.2 6 4 5 7.3 6 7.7 Vehicle D1 7 10 8 7.7 7.7273 8.2 7.8 8 8.0 7.1 7.0833 7.6 8 8.0 Vehicle D2 Subjective Ratings 6 4 1 2 3 4 5 6 7 10 8 8.4 8.0 8.3 8.0 8.0 8.0 8.4 Vehicle D3 6 4 1 2 8.3 8.0 3 4 5 6 7 10 8 7.7 6.8 6 4 1 2 3 6.9 6.9091 7.8 7.0 4 7.3 5 6.9 6 7.9 Vehicle D4 7 10 8 7.4 4 6.7 6.5 6 1 2 3 4 5 6 7.2 Vehicle D5 7 Driver Number Figure 21. Comparison of rating range and mean value on each car in D-class. 5.3 Evaluation of Rating Tendency The purpose of investigating rating tendency is to get rid of outliers and reduce the unintended effects caused by individual preference. Seven expert drivers participated in the driving tests and made subjective evaluation on five vehicles in D-class. Expert drivers with experience and special competence in subjective evaluation can be expected to consistently provide better subjective evaluation than novice drivers, since novice drivers need more mental concentration on the vehicle control during tests rather than the judgment of steering feel itself [15]. However, expert drivers may also have their own rating tendencies. Some of them are likely to always rate higher or lower than other drivers, or in other cases some of the drivers possibly invariably have wider or narrower rating range than the rest. Besides, their rating is more likely affected by their prejudgment of the technical settings of a test vehicle other than their actual feel because of their experience. 35 There are five vehicles in D-class so each driver gave five ratings on each subjective assessment. The maximum, minimum and mean values of subjective evaluation of every driver are investigated. The way used to find out preferences is basically by comparing these values between different drivers. The results are graphically illustrated in Figure 22. SA-2:efforts(parking) 9 9 9 8 8 8 7 7 6 6 5 5 5 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers SA-5:roll control(straight ahead) SA-6:torque feedback(straight ahead) 10 10 9 9 9 8 8 8 7 Rating 10 Rating 7 7 6 6 6 5 5 5 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers SA-7:modulation(straight ahead) 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers SA-8:response(cornering) SA-9:roll control(cornering) 10 9 9 9 8 8 8 7 Rating 10 Rating 10 7 7 6 6 6 5 5 5 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers SA-10:torque feedback(cornering) 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers SA-11:returnability(cornering) SA-12:modulation(cornering) 10 10 9 9 9 8 8 8 Rating 10 7 7 Rating Rating 7 6 SA-4:response(straight ahead) Rating Rating 10 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers Rating SA-3:returnability(parking) 10 Rating Rating SA-1:first 50 test 10 7 6 6 6 5 5 5 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers 1 2 3 4 5 6 7 m m1 m4 m14 Driver number and cases omiting some of the drivers Figure 22. Evaluation of rating based on each subjective assessment. 36 Links between Subjective Assessments and Objective Metrics for Steering Studying the figure above, each of the 12 subplots presents a subjective assessment with ratings of seven drivers. The first seven bars shaded in yellow show the rating ranges of seven drivers. The upper and bottom lines represent the maximum ratings and minimum ratings respectively. The dot-dash lines represent the mean values of ratings. By examining their subjective ratings in the figure closely it is easy to find that some of the drivers show obvious rating tendency. For example, Driver 4 always rated lower in most of the subjective assessment. Likewise, some other drivers also plausibly show their preferences but in different patterns. After close observation, four drivers show possible rating tendency and they are listed in Table 9. Table 9. Evaluation of rating on each subjective assessment. H – highest average rating W – widest rating spread Notation L – lowest average rating N – narrowest rating spread Driver 1 Driver 2 Driver 3 Driver 4 Driver 5 Driver 6 Driver 7 SA-1 First 50m SA-2 and Parking (low speed) SA-3 SA-4 L W H N N SA-5 H H H L H L H L H L Straight-ahead SA-6 W SA-7 Cornering SA-8 W SA-9 W L SA-10 W L SA-11 W N L 6 W,4 H 2N 8L SA-12 Summary 3H Before trying to profile the rating tendency of drivers it is necessary to clarify the last 4 bars shaded red in each subplot in Figure 22. The first bar named as ‘m’. The upper line of m means the mean value of the maximum ratings of all the drivers, while the bottom line of m is the mean value of the minimum ratings of all the drivers. Also, the mean value of m marked with a cross is average of the mean ratings of all the drivers. Thus, the bar is mainly used to show the average level of subjective ratings from 7 drivers. Similarly, the following 3 bars indicate the same meaning. The only difference is that bar m1 excluded the data from 37 Driver 1, bar m4 excluded the ratings from Driver 4 and bar m14 excluded the data from both Driver 1 and Driver 4. Driver 1: As shown in Table 9, Driver 1 has the widest rating spread in 6 out of 12 subjective assessments. For example, regarding response of cornering controllability rating spread is 3 while most of the rest rated within the range of 1.5. Furthermore, wide rating can also be found in Driver 1 subjective evaluation such as 6~10 for parking efforts. Driver 1 also shows a different tendency in the straight-ahead tests. Mean values of subjective ratings are always higher than other test drivers when considering response, roll control, torque feedback and modulation in straight-ahead test. After excluding the data from Driver 1, the bar m1 moves down when comparing with the bar m. In other words, the data from Driver 1 caused a higher rating level generally in evaluation of straight-ahead controllability. In addition, the rating spread can be narrowed down in average if neglecting the data from Driver 1. Driver 2: In general, no specific rating tendency can be found from subjective evaluation of Driver 2. However, it is interesting to find a possible preference when considering returnability. Driver 2 rated returnability of the five vehicles the same in either straight-ahead tests or cornering tests. It seems that Driver 2 is not sensitive to the difference of subjective feel of returnability between different vehicles comparing with other test drivers. Driver 4: According to Table 9, Driver 4 rated most of the subjective assessments lower than others. This tendency is clearer in the straight-ahead and cornering tests. If excluding this data, like the method used to analyze the data of Driver 1 the mean values (bar m4) moves higher in most of the subjective assessments when comparing with bar m. That is, data of Driver 4 contributes to lowering subjective ratings level of the whole data set. Driver 7: There is no obvious preference for the subjective evaluation of Driver 7. Only regarding to the first three subjective assessments, Driver 7 might tend to give higher grades in the lowspeed tests. Nevertheless, this kind of tendency did not repeat in the following 9 subjective assessments. In short, Driver 1 and Driver 4 show clear rating tendencies after analyzing their data. Driver 2 and 7 reveal potential tendency in some specific subjective assessments or test scenarios. 38 Links between Subjective Assessments and Objective Metrics for Steering For the rest of the drivers, no obvious pattern can be found. To further analyze the subjective rating, statistical spread analysis can be applied that will be performed in two steps [16]. First, normalize subjective rating due to driver different judgment scale. Secondly, a criterion of driver exclusion need to be applied to find out if the rating of one driver is particularly far away from the average rating level with respect to a specific assessment. To carry out such kind of work, a reference vehicle is the prerequisite that is not the same case in this thesis. 39 40 Links between Subjective Assessments and Objective Metrics for Steering 6 Linear Regression Analysis of Subjective and Objective Data In this chapter subjective and objective data will be brought together so as to identify those assessments where subjective-objective linear correlation can be discovered. These subjective assessments are considered to reveal direct and clear effects of correlated objective measures. Ideally, simple linear regression equation is the easiest way to express the simple positive or negative one-to-one subjective-objective correlation. While under the real test environment considering the different test vehicles and subjective driving preferences it is not likely to find that many simple linear correlations in the general way. That is why both simple and multiple regression are used to uncover as many linear relationships for each subjective assessment as possible. 6.1 Results from Simple Linear Regression Simple linear regression is one of the simplest in statistics and attempts to explore the relationship between two variables using a straight line. R-value of all paired subjectiveobjective parameters has been investigated. The way to find out possible correlations is case study, which is based on subjective ratings of every driver in each vehicle class. Taking Driver 3 in class as an example, he totally tested 5 vehicles in this car segment. A correlation between SA-2 and OM-26 can be found according to his subjective data. As shown in Figure 23, a correlation with the value of | | above 0.7 is revealed. That is, there is a possible correlation between the parking effort feel and the mean absolute value of steering wheel torque when turning 180° left and right can be discovered. 41 R-value of Obj vs Sub (Driver 3 in D-class with 5Vs) Objective mearsures 1 2 3 4 5 Subjective ratings 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Figure 23. Result of simple regression of one driver in D-class (correlations with| | | | shaded with dark blue, while are shaded with light blue). are However, it is not so convincing to declare a correlation exists, that only depends on one case. More findings of the same correlation in D-class are required to convince us to believe that it exists. Considering that small sample size increase chance to reject a correlation, no less than a half of total drivers show the same correlation can be persuasive to support its existence. In D-class, there are seven test drivers, out of which six drivers show the correlation between SA-2 and OM-26. This indicates that this correlation has high likelihood of existing. Analyzing the regression coefficients can help us to know if the relation between parking effort and steering wheel torque is negative or positive. As seen in Table 10, the regression coefficients are all negative when conducting case study on data of each driver in D-class, except for Driver 6 for whom the correlation coefficient is not strong enough. Therefore, a negative relation between SA-2 and OM-26 has been confirmed. Table 10. Regression coefficients between parking efforts and steering wheel torque. Driver 1 2 3 4 5 6 7 -3.19 -1.23 -1.69 -1.49 -2.48 / -0.68 Coefficient ( ) By employing the same method stated above, subjective-objective correlation with | | has also been found in other vehicle segments. A subjective-objective correlation is 42 Links between Subjective Assessments and Objective Metrics for Steering Driver supported only when data from no less than half of drivers demonstrate this correlation. The regression coefficients are listed from Table 11 to Table 14, where the negative coefficients are shaded. Table 11. Regression coefficients ( ) if correlations (| | SA 7 10 OM 1 2 4 21 23 21.15 1 0.08 5.77 2 1.41 3 4 1.11 -0.14 -2.89 0.15 5 6 -1.61 0.17 2.88 0.34 2.04 7 0.83 -0.09 -1.44 -0.96 8 -0.10 -2.15 0.10 9 1.49 -0.28 -7.14 10.58 10 ) if correlations (| | 10 22 23 Table 13. Regression coefficients ( SA 2 4 OM 26 2 4 -2.32 0.36 7.50 1 -1.57 0.09 1.77 2 3 -1.03 0.15 2.58 4 -0.64 -0.04 -0.65 5 -1.54 0.33 6.73 6 0.08 1.29 7 0.17 8 9 ) if correlations (| | 7 4 19 -0.89 Driver Driver Table 12. Regression coefficients ( SA 2 OM 26 -3.19 1 -1.23 2 -1.69 3 -1.49 4 -2.48 5 6 -0.68 7 0.33 1.443 1.12 1.672 1.16 0.96 0.84 1.284 0.14 3.09 -0.57 3.56 0.65 3.41 2.22 0.10 0.18 0.42 ) found in C-class. 2 26 27 -2.05 -2.04 -1.35 -1.43 -0.51 -0.54 -0.43 -0.47 -0.75 -0.52 ) found in D-class. 12 23 0.62 0.59 1.04 0.75 0.80 ) found in E-class. 10 12 22 23 22 6.67 10.00 0.74 0.89 0.68 -13.33 0.77 1.61 0.74 0.59 -2.10 0.90 1.75 1.10 0.81 0.34 -0.36 1.25 43 Driver Table 14. Regression coefficients ( ) if correlations (| | SA 2 3 OM 26 27 26 27 -1.07 -0.91 1 -2.21 -1.76 -0.89 -0.71 2 -1.52 -1.21 -1.09 -0.86 3 -0.81 -0.63 4 -1.52 -1.21 5 -1.00 -2.28 -1.87 6 ) found in SUV. 12 15 -0.11 -0.08 -0.04 Subjective-objective correlations found from simple linear regression analysis are summarized in Table 15. Only five subjective assessments can be found to correlate with objective measures. Two of the correlations can be seen in every vehicle segment, which are quite convincing. The results are analyzed and discussed as follows. Table 15. Subjective-objective Correlations (the negatives are shaded). SA 2 3 4 7 10 12 26 27 1 2 4 21 23 C 26 22 23 23 D OM 26 2 4 4 19 22 23 22 E 26 27 26 27 15 SUV Interpretation of results: 44 SA-2 vs. OM-26 and 27 OM-26 and 27 are the mean absolute value of steering wheel torque when turning 180° left and right. The only difference between them is the measurement condition that one is measured standstill while another is measured at the speed of 7 km/h. They correlated with each other, so only OM-26 actually has correlation with SA-2. Negative correlation between SA-2 and OM-26 indicates that drivers prefer a lighter steering wheel torque during parking. This preference is revealed in all vehicle classes. SA-3 vs. OM-26 and 27 The explanation of negative correlation between .OM-26 or 27 and SA3 is similar to the previous interpretation. However, the negative effects of these parking efforts are only found under SUV. SA-4 vs. OM-2 and 4 OM-2 and 4 are correlated with each other. The positive correlation between yaw/lateral acceleration response gain and response of straight-ahead controllability implies that a larger on-center steering sensitivity when driving at 0.2g and 120km/h leads to a higher subjective rating of response. Links between Subjective Assessments and Objective Metrics for Steering SA-7 vs. OM-1, 2 and 4 The OM-1, 2, 4 actually are correlated with each other. The OM- 1(window) has a negative relationship with the other two parameters. In C-class, higher ratings of steering modulation or precision are achieved when steering sensitivity is tuned smaller or window is set larger. However, a different result is found in E-class that a bigger lateral acc. resp. gain will instead make drivers give higher subjective ratings in steering modulation. SA-10 vs. OM-19, 21, 22 and 23 SA-10 is the torque feedback of cornering controllability. In D-class, OM-22 and 23 are correlated. Thus, only the OM-23 is needed to take into consideration when conduction regression analysis in D-class. In E-class, the correlations found are based on inadequate data; hence the results are not reliable. OM-23 is named after effort level, meaning that the steering torque needed at 0.3 g and 75 km/h of steady state. The correlation between torque feedback and effort level (OM-23) has been found in every vehicle class. The positive correlation implies that the drivers expect a high torque feedback when cornering at a high lateral acceleration. On-center hysteresis (OM-21) has been found to be an important parameter in defining torque feedback at low-speed cornering test in C-class. A larger torque deadband gave a higher subjective rating of torque feedback. SA-12 vs. OM-15, 22 and 23 It is found that modulation in cornering had different relationships with objective metrics in three vehicle classes. In SUV segment, modulation negatively correlated with sine time lag Yaw-SWA. A larger time lag most likely caused drivers to over compensate steering wheel angle during cornering, which led to bad modulation. In D and E-class, modulation has a positive correlation with effort level and off-center hysteresis respectively. 6.2 Results from Multiple Linear Regression The process of correlating subjective ratings and multiple objective inputs is to match most important objective measures by ordinary least square regression. 6.2.1 Regressor Elimination Process In the previous chapters, the needed objective metrics are listed corresponding to each specific assessment. However, a great problem is highlighted regarding to some assessments that if all available objective measures are included to build a regression equation, the number of regressors will be larger even than that of subjective data set, which is the same as the number of test vehicles in our data set. This fact means that it is impossible to check the statistic significance of the regression equation and the regressors. As seen in Equation (19), some elimination work has to be carried out until the regressors are less than the number of vehicles in a test class. Besides, some of the metrics represent multicollinearity which 45 might make it hard to define whether the contribution to vehicle response of each regressor is true or not. (19). [ ] is subjective rating of a specific test group for various vehicles. There are sets of objective metrics for various vehicles. is the constant in the regression equation. is the coefficient matrix for regressors . The elimination process and principle is mentioned by Chen [4] and ridge plots are used to identify suitable regressor sets. A new matrix from the Equation (1) in 6.2.1 of [4] is introduced giving an estimate of contribution of each regressor in the solution. Significant regressors can be confirmed by being farther away from than insignificant ones. Also a sudden change of slope near around indicates the multicollinearity, which means certain regressors have strong relationships to one or more other inputs. Take the case of Driver 3 in D-class test as example when studying subjective assessment. Thanks to too many measures as independent inputs in this research, the first version of ridge plot might be too complicated to make elimination as seen in Figure 24. Those regressors (objective metrics) with lowest are eliminated at the beginning so that the number of regressors is smaller than data set. In most test groups, there are five or six car models included which means only four or five objective measures are allowed in order to make clear plot as seen in Figure 25 to decide regressors with multicollinearity. As shown in Figure 24, , and is taken out of the analysis result in Figure 25. 46 Links between Subjective Assessments and Objective Metrics for Steering Regression coefficient i 0.8 0 0.6 1 0.4 3 0.2 5 0 7 2 4 6 -0.2 -0.4 -0.6 -0.8 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 k Figure 24. Ridge plot used in the elimination process (too many regressors as inputs). 1 0 1 0.9 3 Regression coefficient i 0.8 6 7 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 k Figure 25. Ridge plot used in the elimination process (regressors less then data set). After reaching a sufficiently clear ridge plot, the rule of elimination is as follows: Remove regressors that have slope lines, but lying close to zero, which indicate insignificance. 47 Remove regressors that have lines exhibiting instability nearly when effects of multicollinearity. Leave a subset with at least two regressors for least squares regression. , meaning the 6.2.2 Multiple Linear Regression With suitable objective regressors decided the method of least squares is used to get linear regression equation for each subjective assessment and each test group. In Equation (19), is fixed for each test class but depends on the decision about regressors before. Rewrite this equation with the part of residual as Equation (20). (20). [ ] Certainly, regression equation can be generated on each assessment and car segment. Several statistics have to be calculated to determine the quality and validity of the equations. The coefficient of determination R² reveals how well this regression model is likely to predict the future outcomes, in this research if the R²-value is below 0.7 it is assumed to have uncertain correlation. The F-statistics interprets the statistical significance of regression equation and each regressor coefficient to itself. Considering the given small dataset (the number of test vehicles across all classes) 75% confidence level is adopted for F-test. Refer to the Fdistribution table (Appendix A), if calculated F-statistics value is larger than the critical value with the statistical significance is confirmed. With both requests above fulfilled and if still more than three regressors are left the ones with least statistical significance are allowed to be disregarded until at most three objective measures remain. 6.2.3 Subjective Ratings vs. Objective Measures Organized by Assessment Number The results of found subjective-objective correlation are shown in Table 16, and the detailed regression equations for each assessment and each vehicle class are listed in Appendix C. Table 16. Overview of assessments where correlation is found for each class. Subjective assessment number C-class 2 3 4 6 7 8 10 11 12 Driver 3 × Driver 4 × × × × × Driver 5 × × × Driver 9 × × × D-class 48 Links between Subjective Assessments and Objective Metrics for Steering Driver 1 Driver 2 Driver 3 Driver 4 Driver 5 Driver 6 Driver 7 E-class Driver 4 Driver 5 Driver 8 Driver 9 SUV Driver 1 Driver 2 Driver 3 Driver 4 Driver 5 Driver 6 Frequency × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × 1 1 3 × × 4 9 6 × × 9 × 8 × 9 All the assessments where at least one correlation is found include SA-2, 3, 4, 6, 7, 8, 10, 11 and 12. On the contrary, nothing is revealed regarding to SA-1, 5 and 9. Considering the total number of drivers attending each test group with corresponding car segment those assessments with most frequent correlation are shown in Table 17. Recall that, in the tests with C and E-class vehicles not all drivers test every available vehicle in that class. Especially for those drivers with less than four test configurations (test cars), their subjective rating data is not valid for multiple regression analysis since such small sample is not able to lead to a valid regression equation. SA 6 7 8 10 11 12 Table 17. Best correlating assessment across all test classes. Level 3 (Level 2) Frequency Torque Feedback (Straight Ahead Stability) 4 Class B, C, E, SUV Modulation (Straight Ahead Stability) 9 Class B, C, E, SUV Response (Cornering Stability) 6 Class B, C, SUV Torque Feedback (Cornering Stability) 9 Class B, C, SUV Returnability (Cornering Stability) 8 Class B, E, SUV Modulation (Cornering Stability) 7 Class B, C, SUV 6.2.4 Best Correlating Assessments Table 18 to 23 show the regression equations found for these best assessments. F-statistics for each regressor is excluded to make the table concentrated on the regression coefficient 49 since the significance is already confirmed as seen in Appendix C and each row presents the regressor coefficient for a given driver, vehicle class and assessment. Table 18. All found correlation for SA-6 (negative regression coefficient is shaded in gray). OM SA-6 Driver 8 9 10 R^2 F-stat 6 0.07 6.58 0.96 39.80 C-class 2 -0.40 7.34 0.96 23.47 D-class 5 -0.03 -0.95 0.99 49.11 E-class 1 1.03 -0.04 2.36 1.00 55551.41 SUV Table 19. All found correlation for SA-7 (negative regression coefficient is shaded in gray). OM SA-7 Driver 1 2 3 4 5 6 8 9 10 R^2 F-stat 3 -0.03 5.80 0.99 179.66 -4.32 -2.14 5.27 0.92 7.75 C-class 4 5 -3.01 -0.74 0.99 102.25 5 1.32 -10.7 0.06 1.00 2366.59 -2.61 -3.05 0.08 1.00 43323.71 D-class 6 7 4.64 -0.57 0.91 9.60 8 0.18 -0.59 0.96 36.21 E-class 9 0.09 -4.01 0.04 0.99 133.18 4 -0.09 0.19 0.98 42.35 SUV Table 20. All found correlation for SA-8 (negative regression coefficient is shaded in gray). OM SA-8 Driver 11 12 13 14 15 16 17 R^2 F-stat -0.20 0.04 0.83 7.46 C-class 9 1 0.16 -0.17 1.00 2717.99 2 0.75 0.02 0.95 20.21 D-class 4 1.50 0.05 0.95 20.21 6 0.05 0.02 0.97 33.13 5 -0.36 0.05 0.97 29.62 SUV Table 21. All found correlation for SA-10 (negative regression coefficient is shaded in gray). OM SA-10 Driver 19 20 21 22 23 R^2 F-stat 4 1.00 6.93 0.89 7.83 -0.57 -1.82 4.35 0.95 11.70 C-class 5 9 -0.18 0.06 1.44 0.97 21.56 1 -1.27 3.90 0.87 6.49 -0.18 0.68 0.96 22.59 D-class 3 5 -0.06 0.18 1.00 1.00 2675.32 2 0.75 -1.60 0.99 40.58 5 -0.11 -0.32 1.00 3325.66 SUV 6 0.89 -2.37 1.00 495.10 50 Links between Subjective Assessments and Objective Metrics for Steering Table 22. All found correlation for SA-11 (negative regression coefficient is shaded in gray). OM SA-11 Driver 19 20 21 22 23 R^2 F-stat 1 -0.65 -0.36 2.64 0.99 36.80 3 0.23 0.46 0.96 25.68 4 -0.16 0.62 0.98 63.99 D-class 6 -0.51 0.24 0.71 1.00 274.38 6 -0.25 -0.10 1.12 1.00 444.54 7 -0.36 1.35 0.96 22.59 4 -0.40 -0.17 0.89 0.93 13.04 E-class 8 -0.27 1.55 0.82 6.80 6 0.78 -1.91 0.99 33.91 SUV Table 23. All found correlation for SA-12 (negative regression coefficient is shaded in gray). OM SA-12 Driver 11 12 13 14 15 16 17 20 21 22 23 R^2 F-stat 4 0.09 -0.04 3.98 0.99 40.37 C-class 5 -0.14 0.01 -0.07 0.98 42.12 1 -4.55 -0.06 1.42 1.00 137.63 3 -0.03 0.02 0.04 1.00 190.00 4 -0.08 0.69 0.99 101.39 D-class 5 -0.12 0.92 0.99 106.21 7 -0.04 0.04 0.78 1.00 132.75 4 -0.33 1.38 1.00 7914.13 SUV 6 6.67 -0.19 0.98 48.97 As mentioned in the previous section one of the reasons to do metrics elimination is to leave suitable ones as regressors. However, even if the number of regressors in each equation is limited the appearing connected metrics in the equation differ a lot depending on the given driver especially for SA-7 and 12 where there are more objective metrics in the initial parameter pool. As a result, despite of some relationships regarding to these two assessments the correlated objective measures have very large change distribution and there is no tendency about the sign of their coefficients. On the other hand, for those assessments with relatively few initial measures, SA-6, 8, 10 and 11, the key correlated measures are easy to find. For example, in SA-11, OM-19, 21, 22 and 23 stand out quite much, and the sign of coefficients shows good agreement. In other words, it demonstrates that a correlated objective measure has unequivocally either positive or negative effect regardless of the given driver as long as there is no excess of initial regressors. Even across different best assessments part of metrics has a fixed tendency to a positive or negative sign. Close examination on each correlated measures is done later on. Another conclusion based on these two groups of assessments is that some true and significant correlation should be found if some of the initial parameters used for SA-7 and 12 51 can be cut. It is the same for the whole subjective assessment data set, it is improper to use all objective measures as regressors unless there is larger data size (the number of test vehicles). 6.2.5 Interpretation of Results Through the results of previous section the hypothesis could be verified that each driver has his or her own unique manner or preference compared with other drivers. Given the confidence interval for various vehicle-classes and the idea that drivers grade the steering performance according to different response or metrics is reasonable. The result of this verified hypothesis is that analyzing the average ratings of all test drivers might lead to unreliable information; some unreal correlations might be discovered while some existing ones might be ignored mistakenly. As shown in Table 24, for those found correlations between objective metrics and average subjective assessments the corresponding measure inputs differ a lot and show no consistency with individual driver study in the previous section. As an example of SA-11, OM-20 and 22 are included in the equation but the sign of OM-20 does not follow the tendency of majority of drivers. Due to the possibility of causing wrong conclusion the method of individual case study is more appropriate than adopting average ratings. Table 24. All found correlations between objective metrics and average subjective assessments. OM 2 6 8 Car 4 D-class 0.03 0.04 C-class 6 E-class -0.27 7 C-class 8 D-class C-class 10 D-class 11 SUV D-class 12 SUV SA 9 10 0.04 4.27 0.03 0.05 4.45 12 13 14 15 16 19 20 22 23 R^2 1.00 0.87 0.82 0.94 0.10 -0.04 0.04 0.99 -0.26 2.13 0.94 -0.42 1.37 0.33 1.00 0.18 -0.53 0.99 -0.05 0.63 0.99 4.29 -0.07 1.00 F-stat 486.25 10.32 8.89 25.42 25.85 23.62 837.61 45.28 117.16 283.72 6.2.6 Influence of Correlated Objective Metrics in Best Assessments Back to the correlations in these best assessments, some conclusions on the effect of objective metrics are drawn. 52 SA-6 torque feedback in straight-ahead controllability Shown in Table 18, SA-6 mostly shows negative correlation with torque buildup (OM-9) and positive with torque at 0g-lateral acceleration (OM-10). At the state with 0g or before the moment of starting turning, zero or too small steering torque (OM-10) means no feedback from the vehicle and usually does not give a good subjective perception. Links between Subjective Assessments and Objective Metrics for Steering On the other hand, the increase of steering wheel torque (OM-9) should not depend on lateral acceleration. Otherwise, an excessively high torque, which might be out of normal range, is needed with a sudden increased lateral acceleration. SA-8 response in cornering controllability SA-8 in Table 20 reveals a few correlations with OM-11, 12, 13 and 16. However, none of them is as universal as that for SA-6. Thus, the subjective feeling of the response during cornering probably depends on individual driving preference. Only D-class cars show somewhat uniform links on OM-12 and 16. Response gain understeer (OM-12) is looked on as needed steer angle with increasing lateral acceleration. For a better response feeling drivers of D-class do not mind giving more excessive steer angle that means a better way to achieve wanted turning status precisely. The other correlated parameter sine time lag Ay-SWA (OM-16) also interestingly shows positive relation. This could be because the drivers are influenced by some other properties of the car, such as roll and yaw motion. If the response depends too much on the preference of a driver he or she may also rate it lower because the car feels nervous, even though this is mainly an issue when driving straight or at small lateral acceleration. SA-10 torque feedback in cornering controllability As seen in Table 21, sedans (mainly C and D-class) and SUVs exhibit different multiple regression equations on SA-10. For all non-SUV cars, SA-10 shows negative correlation with OM-19, while positive ones with OM-22 and 23. Torque buildup into the corner (OM-19) should be quite small. The explanation is the same as in straight-ahead controllability (SA-6). The positive correlation between off-center hysteresis (OM-22) and effort level at 0.3g-lateral acceleration (OM-23), which has been clarified in the section 5.1, encourages both of the two to have positive effect on this torque feedback feeling. In the steady cornering larger steering torque or correction torque (off-center hysteresis indicates the effort to correct steering) make sure drivers can get enough torque feedback from the steering wheel. For SUVs, SA-10 has negative link with both OM-20 and 22. It is interesting that for SUVs they have more emphasis on low torque buildup in cornering (OM-20) instead of torque buildup into cornering (OM-21). However, all cars with higher torque buildup will not cause a likable torque feedback. The preference of off-center hysteresis (OM-22) is opposite to that for sedans. As in this segment, the range for this parameter is concentrating on [1.5, 3.5] as seen in Figure 26 (the SUV subplot), while for the non-SUV class most metrics locate in [1.0, 2.0]. Thus the explanation for this conflict of signs is that there might be an optimal vale (around 1.5 to 2.0), so when having smaller than this interval it is good to have higher off-center hysteresis but when having larger than this interval [1.5, 2.0] there is negative effect on subjective feeling. That is why for these 53 SUVs with growing off-center hysteresis the feeling of torque feedback gets worse. Figure 26. The range of off-center hysteresis (the upper one from D-class and the lower one from SUV). For E-class vehicles, no found correlation is based on two reasons. Too small data size for the multiple regression analysis and that the ratings on this assessment concentrate on some points makes difficult to produce regression results with high confidence. SA-11 returnability in cornering controllability The initial regressor input for this assessment seen as Table 22 is same as the last one. It is interesting to compare the significance of each metrics. Since not that many findings come from the SUVs the comparison for sedans shows that the significant objective measures (OM-19, 22 and 23) for returnability have the same negative or positive property as for torque feedback (SA-10). Only one added measure here is on-center hysteresis (OM-21), which is the only parameter deciding when the assistance force will disappear during turning the steering wheel back. A zero value on this measure makes sure the drivers feel no resistance torque unless the steering wheel is at the straight path position. That consequently leads to a feeling of easy returning. 6.3 Result Analysis Considering the small sample size of data available a case study is used to find out possible correlations according to findings from test data. Both simple and multiple linear regression analysis are employed. All of the links between subjective assessments and objective metrics, which have high possibility of existing, can be used to provide insights for tuning strategy on vehicle steering. Recall that, the main analyzing approach is individual case study and the conclusion is based on the findings that are confirmed from expert drivers. 54 Not that many correlations found among E-class cars, since the number of drivers who went through all vehicle tests is insufficient to become a statistical sample. The most common and uniform results come from torque feedback or returnability in straight-ahead and cornering tests (SA-6 and 10), where appropriate initial explanatory variables are set. For those subjective assessments with only one explanatory variable (i.e., roll control), more corresponding objective metrics are needed in order to explore regression Links between Subjective Assessments and Objective Metrics for Steering correlation. Parking torques (OM-26 or 27), known as mean value of steering torque, are preferred to be small for parking efforts perception (SA-2). For response assessments in both straight-ahead and cornering controllability (SA-4 and 8), the found correlations are not common for all classes of cars. More data samples are to be introduced if the positive correlations with response gain at straight path (OM-2 and 4), understeer (OM-12) and time lag (OM-16) should to be confirmed. The torques for initial steering wheel turning (OM-10) and for holding steady in cornering (OM-23) are preferred to be larger to give better torque feedback (SA-6 and 10). For all vehicles, lower torque buildup at straight path and cornering (OM-9 and 19/20) is favorable as a larger value means the steering wheel torque could increase too rapidly with lateral acceleration. On-center hysteresis (OM-21) mainly has negative effect on returnability (SA-11) because in the process of returning steering wheel, the less on-center hysteresis guarantees that the assistance force from steering wheel will disappear until the steering wheel angle is quite close to zero-degree position. On the contrary, a larger on-center hysteresis makes an unanticipated resistance force leading to worse returnability feeling. Off-center hysteresis (OM-22) has positive correlation on torque feedback and returnability (SA-10 and SA-11) for non-SUVs (C, D and E-class). Meantime, it shows negative link for SUVs. The reason is not there is different preference, but the objective value for SUVs is higher than optimal range, so if higher grade wants to be achieved, this objective value should be lower (closer to preferred interval), and vice versa for the non-SUVs. 55 56 Links between Subjective Assessments and Objective Metrics for Steering 7 Nonlinear Regression Analysis of Subjective and Objective Data In this chapter selected data from VCC will be used in Neural Network training to define preferred ranges of some objective measures that can give better subjective evaluation for steering. 7.1 Initial Results from Neural Network Model As illustrated previously Neural Network can be used to explore nonlinear subjectiveobjective correlation and optimize the value range of objective measures in product tuning. With the tentative trial of the data from a similar research at KTH it is already sure that at least 16 testing examples are necessary to achieve NN training. Based on present VCC data set in each vehicle class it is not possible to reach enough sample size unless different testing groups are merged. Hence, data from different vehicle classes was combined in order to train NN successfully. After some initial results from NN models are acquired they can be used to check the conclusion from our previous study and other research. 7.1.1 Data Used in NN Training To have 16 testing objects some groups have to be combined. However, the problem that the expectation from various classes varies a lot has to be considered. Thus, D-class, E-class and some C-class vehicles (compact SUVs are excluded) will be collected together since at least they share some sedan properties. Totally, this combination will give 15 samples that are barely enough to fulfill the requirement of building a NN model. Through all test drivers only three of them experienced all tests with these 15 vehicles. That means only their subjective data are valid to make NN training. By using case study method each of three drivers will have a result table that is the basis for further analysis. 7.1.2 NN Training The basic settings for NN training have already been shown in section 2.3.1 and parameters of NN can follow the previous design as well. Though different data were used the previous NN model still can be considered as the most reasonable design to explore nonlinear results. It consists of 27 objective measures, 12 subjective assessments and 3 drivers so 972 ( ) networks can be produced for a single input system. As seen in Table 25 and 26 the relationship between single subjective assessment and objective measure for Driver I is shown. Table 25 shows the regression coefficient (r-value) between target output and predicted rating for a given pair of subjective and objective parameter. That indicates if the learned function can relate the input and output well enough and the cells with r-value larger than 0.7 are marked. 57 Subjective assessments Subjective assessments Table 25. NN results, r-value for Driver I. Objective measures 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 1 0.1 0.4 0.3 0.2 0.3 0.6 0.4 0.8 0.7 0.6 0.4 0.4 0.5 0.4 0.6 0.4 0.3 0.5 0.6 0.7 0.4 0.7 0.7 0.6 0.5 0.8 0.6 2 0.0 0.4 0.5 0.1 0.4 0.5 0.5 0.5 0.6 0.5 0.4 0.5 0.6 0.3 0.5 0.5 0.4 0.6 0.4 0.7 0.6 0.6 0.2 0.5 0.5 0.8 0.6 3 0.6 0.9 0.6 0.7 0.3 0.6 0.4 0.7 0.6 0.7 0.4 0.3 0.8 0.5 0.5 0.5 0.5 0.1 0.7 0.6 0.6 0.8 0.4 0.7 0.8 0.3 0.7 4 0.7 0.5 0.3 0.6 0.3 0.1 0.2 0.5 0.5 0.4 0.2 0.3 0.2 0.5 0.2 0.5 0.7 0.2 0.5 0.5 0.5 0.4 0.7 0.1 0.1 0.4 0.6 5 0.3 0.1 0.4 0.1 0.5 0.3 0.5 0.5 0.3 0.4 0.2 0.3 0.5 0.4 0.4 0.5 0.8 0.3 0.2 0.1 0.5 0.5 0.2 0.2 0.3 0.8 0.1 6 0.4 0.2 0.4 0.4 0.5 0.4 0.3 0.4 0.3 0.4 0.6 0.3 0.5 0.3 0.7 0.4 0.4 0.3 0.2 0.7 0.3 0.2 0.1 0.2 0.1 0.4 0.2 7 0.2 0.2 0.5 0.2 0.7 0.3 0.3 0.3 0.2 0.5 0.8 0.3 0.4 0.2 0.6 0.5 0.5 0.5 0.3 0.6 0.2 0.3 0.2 0.3 0.2 0.2 0.2 8 0.3 0.5 0.4 0.3 0.5 0.5 0.5 0.4 0.2 0.4 0.3 0.4 0.2 0.4 0.4 0.3 0.3 0.5 0.3 0.4 0.3 0.4 0.5 0.3 0.5 0.6 0.0 9 0.3 0.1 0.3 0.1 0.3 0.2 0.6 0.5 0.1 0.4 0.3 0.3 0.5 0.4 0.5 0.5 0.6 0.4 0.1 0.1 0.5 0.4 0.2 0.1 0.7 0.8 0.2 10 0.5 0.7 0.6 0.5 0.2 0.7 0.3 0.7 0.5 0.4 0.7 0.2 0.8 0.5 0.6 0.5 0.4 0.4 0.7 0.5 0.3 0.7 0.2 0.7 0.5 0.4 0.7 11 0.8 0.8 0.7 0.9 0.4 0.2 0.3 0.3 0.6 0.9 0.4 0.4 0.8 0.4 0.4 0.4 0.3 0.1 0.5 0.2 0.6 0.3 0.5 0.2 0.4 0.3 0.2 12 0.3 0.4 0.3 0.4 0.6 0.8 0.5 0.2 0.4 0.2 0.5 0.5 0.4 0.4 0.6 0.5 0.2 0.4 0.2 0.8 0.4 0.2 0.2 0.4 0.2 0.3 0.1 1 2 3 4 5 6 7 8 9 10 11 12 Table 26. NN results, mean error for Driver I. Objective measures 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 1.0 0.9 0.9 1.0 0.9 0.7 0.8 0.5 0.7 0.7 0.9 0.9 0.7 0.9 0.8 0.9 0.9 0.8 0.8 0.7 0.8 0.7 0.7 0.7 0.8 0.6 0.7 1.0 0.7 0.8 0.9 0.8 0.7 0.7 0.8 0.7 0.7 0.9 0.8 0.6 0.9 0.7 0.7 0.9 0.7 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.4 0.6 0.8 0.5 0.9 0.7 0.9 0.8 1.0 0.6 0.7 0.7 0.9 0.9 0.7 0.8 0.8 0.9 1.0 0.9 0.7 0.8 0.8 0.5 0.8 0.7 0.5 0.9 0.7 0.3 0.4 0.5 0.3 0.4 0.5 0.5 0.4 0.5 0.5 0.5 0.4 0.5 0.4 0.5 0.4 0.3 0.5 0.4 0.4 0.4 0.4 0.3 0.5 0.5 0.4 0.4 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.4 0.6 0.5 0.6 0.6 0.5 0.6 0.6 0.5 0.3 0.6 0.7 0.7 0.5 0.6 0.6 0.6 0.4 0.4 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.5 0.5 0.5 0.4 0.4 0.5 0.4 0.5 0.4 0.4 0.5 0.4 0.5 0.3 0.4 0.5 0.5 0.5 0.5 0.4 0.5 0.6 0.6 0.5 0.6 0.4 0.6 0.6 0.6 0.6 0.6 0.3 0.5 0.5 0.6 0.4 0.6 0.6 0.5 0.6 0.5 0.7 0.6 0.6 0.5 0.6 0.6 0.6 0.5 0.4 0.5 0.5 0.4 0.4 0.4 0.5 0.4 0.5 0.5 0.5 0.5 0.4 0.5 0.5 0.5 0.4 0.5 0.5 0.4 0.5 0.4 0.5 0.5 0.4 0.5 0.5 0.5 0.6 0.6 0.5 0.5 0.5 0.4 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.3 0.4 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.3 0.5 0.2 0.2 0.2 0.3 0.3 0.2 0.2 0.2 0.2 0.3 0.2 0.3 0.2 0.3 0.2 0.2 0.3 0.3 0.2 0.2 0.3 0.2 0.3 0.2 0.3 0.3 0.2 0.4 0.4 0.5 0.3 0.6 0.6 0.6 0.6 0.5 0.4 0.6 0.7 0.4 0.6 0.5 0.5 0.7 0.7 0.5 0.6 0.5 0.5 0.6 0.6 0.7 0.9 0.7 0.3 0.4 0.3 0.3 0.3 0.4 0.3 0.4 0.3 0.4 0.3 0.3 0.4 0.3 0.3 0.3 0.4 0.3 0.4 0.3 0.3 0.4 0.4 0.3 0.4 0.4 0.4 The regression coefficient is not the only criterion to check the quality of learning. Even if the r-value is 1 the mean error (ME) between the predicted output and target value might be contradictorily large. The reason is that the regression line with perfect learning should not only have small r-value but also go through the origin with 45°-slope as seen in Figure 27. Therefore, it is also important to check the ME seen in Table 26 where the cells with mean error smaller than 0.5 are shaded as well. 58 Links between Subjective Assessments and Objective Metrics for Steering Figure 27. Regression between target rating and output rating learnt by NN. 7.1.3 Correlation Results Preferred objective measures can be defined according to the learning results from NN as seen in Table 25 and 26 if there are links with both good regression coefficient and ME (i.e., large r-value and small ME). Taking Driver I as an example, the r-value of OM-5 and SA-7 is 0.7 and the ME is 0.4. Then this link can be considered as valid. By following the same method illustrated in Figure 7 the preferred range for OM-5 that gives better score on SA-7 is larger than 1.4. All the preferred values for objective measures based on Driver I are listed in Table 27. The total results from all of the three drivers are collected in Appendix D. Table 27. NN results with good fitness and small error, preferred ranges for Driver I. Objective measures 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 1 2 <3.2 Subjective assessments 3 4 45~ 75 5 6 <3.8 5~6 7 28~ 32 >1.4 8 9 <3.8 10 11 12 >95 3.5~ 25~ 1~ 1.5~ 4.5 30 1.25 1.85 >28. 5 <2.2 <1.8 / >92 >2 >90 3~6 59 Regardless of various drivers or subjective assessments the preferred range for each objective measure can be defined if summarizing the findings in Appendix D. Since only the subjective rating of three drivers is applied in this analysis the found results must be quite limited and without very high certainty. The preferred ranges of some objective measures are shown in Table 28. With results from King et al. [11] more analysis and comparison will be done in the next section. Table 28. Overview of objective measure range causing high rating. No. Objective measures Preferred range Reference from other research 2 Response Gain Straight Path [°/s/100°SWA] 25~30 Yaw velocity gain: 20~25 6 Response Time Delay [ms] >95 8 Torque Deadband [°] <2.2 11 Yaw Response Gain [°/s/100°SWA] 28~32 20 Torque Buildup Cornering [Nm/g] 4~6 22 Off Center Hysteresis [Nm] 1.5~2.2 23 Effort Level [Nm] 3.6~4.5 Steering torque at 0.3g: <6.5 26 Parking Efforts Standstill [Nm] <3.3 27 Parking Efforts Rolling [Nm] >1.5 7.2 Result Analysis The value of each objective measure falling in a certain range will give drivers better subjective perception. The ranges of objective measures can be roughly defined by comparing nonlinear correlation results found from the data of three drivers as seen in Table 28. It is necessary to check the rationality of the results found in this thesis work. Therefore further examination is done by comparing the results with the findings from previous work. Though the results are not 100% reliable depending on small sample size they may still provide some insights for further study. Response gain straight path When the response gain at straight path is larger than 25°/s/100°SWA the subjective rating will increase. Compared with the similar objective metrics in another research [11] where the preferred yaw gain at 0.7Hz is within the range of 20~25 the result of this metric seems to map well when taking the difference of testing settings into account. Torque deadband The preferred range of torque deadband is smaller than 2.2°. The objective metrics larger 2.2° leads to imperfect subjective perception. On the other hand to specify the lower boundary of preferred torque deadband more data with extremely small value (lower than 1.0°) must be collected. Yaw response gain During the cornering state drivers preferred the yaw gain within the range 28~32°/s/100°SWA regardless of the assessments being judged. 60 Links between Subjective Assessments and Objective Metrics for Steering Torque buildup cornering The preferred range of torque buildup in cornering is defined by the NN models, which is 4~6Nm/g. However, in the previous linear correlation study there is no clear relationship for this metric. Off-center hysteresis As discussed in the section 6.2.6, a potential optimal range of 1.5~2.0Nm is identified for offcenter hysteresis. In this NN modeling, the favorable range is about 1.5~2.2Nm, which ties quite well with the former hypothesis. Effort level The optimal range for effort level at cornering is starting from 3.6Nm. This result fits with the positive correlation in earlier analysis. Based on current data the upper boundary of this range cannot be specified since there is no metrics larger than 4.5Nm. In the research of King et al. [11] it is defined that the objective measure steering torque at 0.3g should not exceed 6.5Nm, which is somehow the upper boundary of this effort level. Parking efforts standstill The negative correlation between parking effort at standstill state and subjective rating is revealed quite clearly in linear regression chapter. Here, through the nonlinear NN the upper boundary is defined as 3.3Nm. That means the steering torque larger than 3.3 Nm will lead to worse parking perception. Others (response time delay and parking efforts rolling) The preferred ranges of response time delay and parking efforts rolling found in this thesis seem confused when compared with normal driving experience. Response time delay found above 95ms is preferred that is normally expected to be small. The possible reason is that practical values of this parameter for our test objects are sufficiently low leading to more or less too fast response that is not pleased either. There should be an optimal range of this parameter (e.g. 95< response time delay<120ms) that cannot be defined based on our available data set. The same consideration can be applied to parking efforts rolling. 61 62 Links between Subjective Assessments and Objective Metrics for Steering 8 Recommended Procedure of Obtaining Ideal Data for Analysis In this chapter several methods to perform tests and obtain subjective data with the least uncertainty and the highest uniformity to achieve good NN training and get reliable results will be introduced. Among these methods the one considered as most appropriate for VCC to carry out will be proposed. 8.1 Track Test with Real Vehicle The data size must be large enough to carry out linear or nonlinear correlation analysis. If assessed items are standardized and maintained in the future tests according to the trail NN design the appropriate number of sample in subjective assessment tests should be at least 16 to draw a clear and valid conclusion. 8.1.1 Test Track In order to make the subjective tests consistent with measured objective data the test track should be able to cover all wanted test environments including 50m test, straight-ahead test and cornering test. 8.1.2 Various Vehicles Testing many different vehicles is not adopted as main method since the key problem is that it is not easy to collect sufficient vehicles for a research study. Considering the least sample number it will be costly to gather 16 test vehicles with various steering characteristics. In references [16] [17] where fewer than ten real cars are chosen as test objects, the purpose of them is to do simple linear regression, and the focus is more than on steering. Other factors than only steering system that should be taken into account are the differences of power train, chassis and car body. Test people cannot control these different parameters or attributes. As a result it could be a problem to confirm if the difference in assessment is due to some steering properties or other attributes with little interests (i.e., power, chassis, size and mass). The interferences cannot be excluded although these tests can be conducted simultaneously saving both time and cost. 8.1.3 Same Vehicle with Various Steering Characteristics In the previous research [4] [8] [18], various configurations or steering settings are used to vary the characteristics. The changes they made include different tires, dampers, yaw-inertia, bump steer and stiffness. If adequate statistical samples are wanted at least four factors have to be varied between two levels so that 16 configurations are acquired with full factorial test design as seen in Table 29. However, in this case the difference between configurations could be too small to give significant difference in subjective perception. Then, if still wanting to achieve 16 configurations (L16 matrix) instead of using fractional factorial test design [4] more factors can be varied for Resolution V ( ) or Resolution IV ( ) as Table 30 or 63 31. If this method is used the biggest challenges are time consumption and adjustment accuracy. Table 29. Two-level full factorial test design for orthogonal L16 matrix. Configuration Factor 1 Factor 2 Factor 3 Factor 4 No. 1 + + + + No. 2 + + + No. 3 + + + No. 4 + + No. 5 + + + No. 6 + + No. 7 + + No. 8 + No. 9 + + + No. 10 + + No. 11 + + No. 12 + No. 13 + + No. 14 + No. 15 + No. 16 Table 30. Two-level fractional factorial test design for orthogonal L16 matrix (resolution V). Configuration Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 No. 1 + No. 2 + No. 3 + No. 4 + + + No. 5 + No. 6 + + + No. 7 + + + No. 8 + + + No. 9 + No. 10 + + + No. 11 + + + No. 12 + + + No. 13 + + + No. 14 + + + No. 15 + + + No. 16 + + + + + Table 31. Two-level fractional factorial test design for orthogonal L16 matrix (resolution IV). Configuration Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6 Factor 7 Factor 8 No. 1 No. 2 + + + + No. 3 + + + + No. 4 + + + + - 64 Links between Subjective Assessments and Objective Metrics for Steering No. 5 No. 6 No. 7 No. 8 No. 9 No. 10 No. 11 No. 12 No. 13 No. 14 No. 15 No. 16 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 8.2 Simulator Experiments Real track test is abandoned in [15] due to the small tractor samples and simulator experiment is chosen. The way of varying characteristics for simulator is quite similar with real vehicles. The only difference is that one factor can be adjusted at more than two levels and the accuracy of variation is guaranteed. Besides in simulator experiments, the test maneuver can be included in simulation program before the test such as lane change, slalom, parking, etc. and also run all after each other in one simulation run. By comparing these possible test objects, the conclusion of their features is shown in Table 32. Table 32. Overview of test objects. Test object Monetary cost Time cost Influence Analysis Different vehicles Rent of vehicles Test simultaneously Difference of components Linear correlation Different configurations Rent of a vehicle and tire sets Test configurations and adjust settings Adjustment accuracy Linear and/or nonlinear correlation Simulator Rent of simulator Test configurations one by one Simulation feel Linear and/or nonlinear correlation 8.3 Suitable Test Objects In the current database there are already five vehicle models in D-class so the best option is to combine different vehicles and different configurations. To be specific, all these five cars are varied in some factors so that under each vehicle there are several distinct settings. Tire and mass adjustments are interesting since that is the easiest to adjust. Given the variation as seen in Table 33 with three adjustments and fractional factorial design totally there will be 16 or 20 different test objects. This is at least sufficient for linear and/or nonlinear regression analysis. Another important aspect is that once these test objects are determined collection of objective data should be conducted as well. 65 Table 33. Test objects with two-level factorial test design for orthogonal L4 matrix (resolution III). Vehicle parameter First level [-] Second level [+] Front tires Normal tires (tire pressure) Special tires (tire pressure) Rear tires Normal tires (tire pressure) Special tires (tire pressure) Mass Normal load Extra load (ballast/passenger) 4 or 5 vehicles Configuration 1 Configuration 2 Configuration 3 Configuration 4 Front tires + + Rear tires + + Mass + + 8.4 Test Subjects Studies from Agebro, Schoeggle et al. [19] [20] have revealed the huge difference of driving skill and rating ranges between expert and novice drivers. However, the final purpose is to give best driving perception to general customers. It is good to include some general drivers appropriately so as to check if these two groups have significantly driving preference. Both expert and novice groups should include at least six or seven drivers. That is the minimum statistical number. Expert drivers in this thesis are vehicle engineers employed by VCC and novice driver can be picked from normal customers, students or professors. Each driver should drive each car with another one who notes down the subjective feeling and with best efforts to follow the maneuvers. The questionnaire must be completed soon after tests and without allowing drivers to exchange opinions. The plus side of always having a codriver is that the feelings that the driver feels can be noted and captured right away. This is especially important when using less skilled drivers. 8.5 Pre-test with a Reference Vehicle In the previous section regarding individual assessment evaluation the importance of a reference vehicle (configuration) and rating range of each assessment is needed even for expert drivers if analysis should be done based on mean rating value. So a vehicle model with similar features to one of the test objects is required as a reference vehicle. For each driver after driving the reference vehicle, he or she evaluates the experimental one and compare with perception from reference rating value (which could be 7-point). Ideally, across all tests single-blind trails should be done to drivers but this might take too much time. Since several vehicles with various settings are selected at least one of the test objects could be blind. Meaning that no changes are done to this test object and it is thus the same as someone else of the test objects. If this is possible the repeatability of subjective assessment can be analyzed by studying difference in ratings from these two tests. 66 Links between Subjective Assessments and Objective Metrics for Steering 8.6 Normalization Work 8.6.1 Rating Mean Normalization As long as the reference is really adopted by all test drivers and their rating is repeatable, i.e., the difference in ratings from two single-blind tests on the reference vehicle is small. The mean ratings for each assessment and every driver should be very close to each other. Otherwise, some work to normalize the rating mean is inevitable. 8.6.2 Spread Normalization Even if the reference vehicle gives drivers the base line to start their assessment, the spread of their ratings or the sensitivity of their perception cannot and should not be controlled. The normalized subjective evaluation should place all ratings in the same range. As long as the reference perception is followed by test drivers sufficiently well the normalized evaluation can lead to reliable mean ratings so that the method of case study is abandoned to save time and improve validity. Back to the flow chart as seen in Figure 1 and for a specific assessment item SA-Y (m drivers make evaluation on n configurations, [ ], [ ]), first the average value of minimum and maximum rating given by all m drivers is acquired by Equation (21) and (22). Then, the ratio of normal rating range to individual range is defined for each driver as Equation (23). ∑ [ ] (21). ∑ [ ] (22). [ [ [ ] ] ] [ [ ] ] (23). (24). (25). With normalization work done the subjective data of drivers should be similar to each other as seen in Figure 28. If there is still someone with larger range or mean values than the others that data can be excluded as an outlier. The rest of the subjective ratings are considered as truthful and the average values are allowed to represent the comprehensive data. 67 Figure 28. Effect of rating mean and spread normalization. 68 Links between Subjective Assessments and Objective Metrics for Steering 9 Summary Results In this chapter the findings regarding subjective and objective data evaluation, linear links and preferred ranges will be summarized and presented. 9.1 Correlation of Objective Measures and Evaluation of Rating The aforementioned test objects in this study are different vehicles not one vehicle with various settings. Thus, it is understandable that found correlations between pairs of objective measures do not have extremely high correlation coefficients as shown in Table 7. In other words, these cannot support any elimination work of objective parameters. Nevertheless, these correlations can still reveal some confirmed relationships between two different objective measures. The work of subjective data evaluation shows that the data used in previous analyses is basically valid. Some tendency based on specific subjective assessments or drivers can be found according to the exemplified subjective data from D-class. However, the possible rating tendency of drivers will not affect the results in this thesis where the method of case study is applied. 9.2 Confirmed Subjective-objective Links for Steering All confirmed results about simple/multiple linear correlation and preferred range according to Neural Network are collected in Table 34. Most of the found linear correlated objective measures are under three categories torque feedback, response and 50m test. That means these objective measures are key to adjust the corresponding subjective assessments. Furthermore, nearly all confirmed links have pretty unequivocal pattern either purely positive or negative correlation. Some of them, for example on-center hysteresis can be used to infer the optimal value of objective measure. As for nonlinear analysis, based on current data set, the findings are basically agreed with results from linear part. Still, most found preferred ranges are about torque feedback, response and 50m test, regardless of drivers and subjective assessments. Besides, compared with results from King [11] the confirmed ranges of certain objective measures in this thesis are reasonable and even more detailed. So, the information about these range values is supposed to be useful when tuning the steering characteristics of a normal sedan vehicle. 69 Table 34. Confirmed linear and nonlinear links between subjective assessments and objective measures for steering. Level 3 Response OM Objective measures (Level 4) 1 Window [deg] 2 Response Gain Straight Path [°/s/100°SWA] 3 Lateral Acc. Resp. Gain [°/s/100°SWA] 4 Lateral Acc. Resp. Gain [°/s/100°SWA] 5 Gain Linearity [-] 6 Roll Control 7 8 Torque 9 Feedback 10 SA Subjective assessments (Level 3) Linear correlation 25~30 Response Time Delay [ms] Roll Control Straight Path [°/g/s] Torque Deadband [°] >95 <2.2 Torque Build Up [Nm/100°SWA] 6 Torque Feedback (Straight-ahead) Negative Friction Feel [Nm] 6 Torque Feedback (Straight-ahead) Positive 11 Yaw Response Gain [°/s/100°SWA] 8 Response (Cornering) Negative 12 Response Gain Understeer [°/g] 8 Response (Cornering) Positive 13 Response Gain Linearity [%] 8 Response (Cornering) Positive 14 Rel. yaw [email protected] lat. Acc. 15 Sine Time Lag Yaw-SWA [ms] 16 Sine Time Lag Ay-SWA [ms] 8 Response (Cornering) Positive 17 Sine Time Lag Ay-Yaw [ms] Roll Control 18 Roll Control Cornering [°/g] Response Torque Feedback - (50m test) 70 Preferred range 28~32 10 Torque Feedback (Cornering) Negative 11 Returnability (Cornering) Negative Torque Buildup Cornering [Nm/g] 10 Torque Feedback (Cornering) Negative On Center Hysteresis [°] 11 Returnability (Cornering) Negative 10 Torque Feedback (Cornering) Positive/Negative 11 Returnability (Cornering) Positive 10 Torque Feedback (Cornering) Positive 11 Returnability (Cornering) Positive Parking Efforts Standstill [Nm] 2 Efforts (Parking) Negative <3.3 Parking Efforts Rolling [Nm] 2 Efforts (Parking) Negative >1.5 19 Torque Buildup Into The Corner [Nm/100°SWA] 20 21 22 Off Center Hysteresis [Nm] 23 Effort Level [Nm] 24 Low Speed Response Gain [°/s/100°SWA] 25 Low Speed Torque Buildup [Nm/100°SWA] 26 27 4~6 1.5~2.2 3.6~4.5 Links between Subjective Assessments and Objective Metrics for Steering 10 Conclusions and Recommendations This chapter will briefly conclude findings by using linear and nonlinear correlation analysis. Based on the ideal testing procedure, some recommendations to future work will be made. 10.1 Conclusions Simple and multiple linear analysis are useful to detect if certain direct positive or negative correlation exists while Neural Network is an important tool to specify preferred range of objective measures even to find some nonlinear links that are not easy to be described by linear correlation. Both of the two analyzing approaches lead to the findings concerning almost the same objective measures. The preferred range for yaw response gain (OM-2 and 11) is located in a normal scope. That means for the purpose of good driving response neither too nervous nor too lazy steering wheel will be praised. Steering torque at low or high lateral acceleration, and off-center hysteresis (OM-10, 22 and 23) at high speed are preferred to be reasonably large. This fact gives reason to define the detailed ranges of these three measures. Torque buildup at the start of cornering and during cornering (OM-19 and 20) should not exceed a certain upper boundary. Otherwise, it will cause an uncomfortable torque feedback feeling. The steering torque at parking condition (OM-26 and 27) definitely has a limit on the maximum value. The steering torque below this critical value will surely guarantee a better perception of parking effort. However the lower boundary is difficult to define because practical steering systems cannot reach infinitely small parking torque. In addition to agreed findings in linear and nonlinear parts on-center hysteresis and response time delay should be investigated more deeply as long as more data set is available. Anyway, the approach of linear or nonlinear analysis in this thesis has shown a promising future to assess steering properties via objective metrics. An evaluation tool that uses only objective metrics as input to produce subjective rating for steering can be built based on the confirmed subjective-objective links. Thus, real vehicle tests are not necessary if this tool can be used during simulation. Furthermore, this tool can also be integrated into steering robot test, which can enable the robot to collect not only objective metrics but also subjective evaluation. 10.2 Recommendations to Future Work In conclusion the future tests must guarantee that the amount of data samples is sufficient to finish a complete NN training and achieve more convincing conclusion. The detailed suggestions for future work can cover several aspects. 71 MUST: Recollect both subjective and objective data due to new test objects 16 to 20 test objects (4 or 5 vehicles in the same segment each with 4 various settings) At least 6 or 7 expert drivers (more involved will result in less uncertainty) Unify the rating methods of drivers with best efforts after each test expedition (rating range and/or rating reference) Uniform data sheets with fixed content and format No cell regarding objective and subjective items in data sheets is supposed to be blank OPTIONAL: Novice drivers (at least 6 or 7) Single-blind test for test drivers (drivers have no idea what adjustments have been made on each vehicle) Pre-test to give drivers rating reference (with this one totally 17 or 21 test objects) The reference vehicle duplicated in test objects to check the repeatability of rating Focus on those assessments or measures already with found correlations (OM-1, 2, 3, 4, 11, 12, 16, 17, 19, 20, 22, 23, 25, 26 and 27; SA-2, 4, 6, 7, 8, 10, 11 and 12.) Have a Lotus Elise or extreme vehicle in a vehicle class to gain extreme ratings and wider spread of data The MUST steps make sure test data can be used for normal NN training and the OPTIONAL items can increase the confidence of final conclusion on preferred objective range. With this work and the method of NN able to produce wanted results further studies on subjective-objective correlation links are possible. There should also be a possibility to add several new vehicles to already existing data sets given that the process stay the same and the track and environment stay roughly the same. Thus an extensive model and database could be built up across all various vehicle segments in order to model vehicles with ‘perfect driving perception’ over the years. As long as the subjective-objective links are defined more accurately and comprehensively an evaluation tool can be built and used during front end simulations. During software simulation process the adjustment of steering configurations will not be limited by the safety consideration for test drivers. Therefore, wider spread of objective and subjective data could be achieved. In turn, this can help optimize the links. As a result, it is more likely to develop a vehicle with optimal steering design. 72 Links between Subjective Assessments and Objective Metrics for Steering Reference [1] S. Davies, "A Holistic Approach to Quality," Manufacturing Engineer, vol. 86, no. 1, pp. 8-13, Feb.-March 2007. [2] Subjective Rating Scale for Vehicle Handling, Vehicle Dynamics Standards Committee, 2007. [3] G. Helmut List and S. Peter Schöggl, "Method for Analyzing the Driving Behavior of Motor Vehicles". United Stats Patent 6079258, 27 June 2000. [4] Chen, David, "Subjective and Objective Vehicle Handling Behaviour," PhD Thesis, The University of Leeds, Leeds, 1997. [5] Rothhämel, M., IJkema, J., Drugge, L., "A Method to Find Correlations between Steering Feel and Vehicle Handling Properties Using a Moving Base Driving Simulator," Vehicle System Dynamics, vol. 49, no. 12, pp. 1837-1854, August 2011.. [6] Anthony Best Dynamics, "Steering Robot Systems: In-vehicle robot test systems," 2011. [Online]. Available: http://www.abd.uk.com/upload/files/2011-11-17_15-1153_SP6020%20-%20Steering%20Robots%20outline%20-%20Issue%206.pdf. [7] Beale, M. H., Hagan, M. T., Demuth, H. B., "User’s Guide (Neural Network Toolbox™)," [Online]. Available: http://www.mathworks.com. [Accessed 2011]. [8] H. A. S. Ash, "Correlation of Subjective and Objective Handling of Vehicle Behaviour," PhD Thesis, The University of Leeds, Leeds, 2002. [9] Rumelhart, D. E., Hinton, G. E., Williams, R. J., "Learning Internal Representations by Error Propagation," in Parallel Distributed Processing, MIT Press, ISBN:0-262-68503X, 1986, pp. 318-336. [10] Yam, Y., Chow, T., "Determing Initial Weights of Feedforward Neural Networks based on Least Squares Method," Nerual Processing Letters, vol. 2, no. 2, pp. 13-17, 1995. [11] King, R. P., Crolla,D. A., Ash, H. A. S., Whitehead, J., "Identfication of SubjectiveObjective Vehicle Handling Links Using Neural Networks for the Foresight Vehicle," in SAE 2002 World Congress, Detroit, 2002. [12] Sivanandam, S. N., Sumathi, S., Deepa, S. N., Introduction to Fuzzy Logic Using 73 Matlab, Springer; 1 edition, 2006. [13] Mariani, F., Grimaldi, C., Sgatti, S., De Cesare, M. et al., "Artificial Intelligence Methodologies for Oxygen Virtual Sensing at Diesel Engine Intake," SAE Technical Paper 2012-01-1153, 2012. [14] B.A.M van Daal, "Friction and Compliance Identification in a Vehicle's Steering System," Extern traineeship report, Eindhoven, 2007. [15] M. Rothhämel, "Capturing Steering Feel - A step towards implementation of active steering in heavy vehicles," Licentiate Thesis, US-AB, Trita-AVE 2010:57, ISSN 16517660, Stockholm, 2010. [16] Data, S., Frigerio, F., "Objective Evaluation of Handling Quality," in IMechE J. Automobile Engineering, Orbassano, Italy, 2002. [17] de Paula Eduardo, G., "Manual Steering Objective Reference Data Definition based on Subjective Evaluation Correlation," SAE Technical Paper 2011-36-0031, 2011, doi:10.4271/2011-36-0031. [18] Monsma, S., Defilippi, W., "Artificial Neural Networks for the Assessment of Driver Judgement," Vehicle System Dynamics, IAVSD 2011, Manchester, 2011. [19] Agebro, M., "Driver Preferences of Steering Characteristics," Licentiate Thesis, KTH, Stockholm, Trita-AVE 2007:66, ISSN 1651-7660, 2007.. [20] Schoeggl, P., Ramschak, E., "Vehicle Drivability Assessment Using Neural Networks for Development, Calibration and Quality Tests," SAE International, 2000. 74 Links between Subjective Assessments and Objective Metrics for Steering Appendix A F-Distribution Table for confidence level ( and when checking the regression significance). and is corresponding to 75 76 Links between Subjective Assessments and Objective Metrics for Steering Appendix B Correlation coefficient between two objective measures for C-class. R-value of Obj vs Obj Objective measures 1 2 3 4 5 6 7 8 Objective measures 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Correlation coefficient between two objective measures for D-class. R-value of Obj vs Obj Objective measures 1 2 3 4 5 6 7 8 Objective measures 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 77 Correlation coefficient between two objective measures for E-class. R-value of Obj vs Obj Objective measures 1 2 3 4 5 6 7 8 Objective measures 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Correlation coefficient between two objective measures for SUV. R-value of Obj vs Obj Objective measures 1 78 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 2 3 4 5 6 7 8 Objective measures 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 (T-stat) Q12 (T-stat) Driver 9 Q4 (T-stat) Q8 (T-stat) Q10 (T-stat) Q4 (T-stat) Q6 (T-stat) Q7 (T-stat) Q10 (T-stat) Q12 (T-stat) Driver 5 Q7 (T-stat) Q10 Driver 3 Q7 (T-stat) Driver 4 1 -4.09 -3.43 -10.7 -2.86 -7.70 -2.14 -4.32 -6.67 -0.08 -0.74 2.68 10 8.74 5.84 4.37 5.27 6.58 0.07 18.88 9 -12.3 8 5.80 7 -0.03 6 -12.5 -7.29 5 -3.01 -11.9 7.05 4 22.77 3 0.50 2 Multiple linear regression for C-class. Appendix C 0.04 2.98 2.57 -3.08 0.01 17 -10.7 16 -0.14 15 -0.20 -5.65 14 Objective measure -0.04 13 3.07 12 0.09 11 18 0.06 3.40 21 -4.13 -2.95 -0.07 20 -0.18 -5.75 -0.57 19 -4.64 -1.82 2.57 4.55 1.44 5.05 4.35 5.94 3.98 6.93 3.10 23 1.00 22 24 25 26 27 0.97 0.83 0.73 0.98 0.95 0.99 0.99 0.89 0.92 0.96 0.97 R^2 0.99 21.56 7.46 4.09 42.12 11.70 102.25 40.37 7.83 7.75 39.80 18.46 F-stat 179.66 Links between Subjective Assessments and Objective Metrics for Steering 79 80 Driver 1 Q8 (T-stat) Q10 (T-stat) Q11 (T-stat) Q12 (T-stat) Driver 2 Q6 (T-stat) Q8 (T-stat) Driver 3 Q10 (T-stat) Q11 (T-stat) Q12 (T-stat) Driver 4 Q8 (T-stat) Q11 (T-stat) Q12 (T-stat) Driver 5 Q7 (T-stat) Q10 (T-stat) Q12 (T-stat) Driver 6 Q7 (T-stat) Q8 (T-stat) Q11 (T-stat) Driver 7 Q7 (T-stat) Q11 (T-stat) Q12 (T-stat) -10.7 -41.5 3 21.14 2 1.32 1 -4.10 8 -0.57 278.0 0.08 7 3.57 -192 27.48 0.06 6 4.64 -3.05 -163 5 -2.61 4 Multiple linear regression for D-class. 10 7.34 5.21 9 -0.40 -5.84 2.64 -0.36 -8.03 -0.65 -6.06 6.17 39.24 1.00 14.24 -5.45 0.18 0.69 -0.08 -8.86 10.14 -0.06 0.62 -8.52 3.03 -0.16 0.46 4.85 6.45 -4.13 23 0.23 0.68 -0.18 8.47 1.42 9.42 3.60 22 3.90 21 -2.79 20 -1.27 19 0.04 7.81 -6.57 2.72 6.50 -0.04 0.02 0.05 6.45 9.86 0.78 1.35 -4.13 9.21 -17.2 -0.36 0.24 -0.51 12.19 0.71 11.60 18 0.92 4.94 17 -4.13 0.05 16.57 0.04 3.88 7.84 1.50 0.02 -6.85 4.94 -0.03 0.02 16 3.88 15 Objective measure 0.75 -3.14 -58.3 24.24 -0.06 -0.17 0.16 -7.92 14 13 -4.55 12 -0.12 11 24 25 26 27 1.00 0.96 0.91 1.00 0.97 1.00 0.99 1.00 1.00 0.99 0.98 0.95 1.00 0.96 0.96 0.95 0.96 1.00 0.99 0.87 R^2 1.00 132.75 22.59 9.60 274.38 33.13 43323.7 106.21 2675.32 2366.59 101.39 63.99 20.21 190.00 25.68 22.59 20.21 23.47 137.63 36.80 6.49 F-stat 2717.99 Q7 (T-stat) Q11 (T-stat) Q12 (T-stat) Driver 9 Q7 (T-stat) Driver 4 Q11 (T-stat) Driver 5 Q6 (T-stat) Driver 8 1 6 7 8 -4.01 -8.33 0.09 3.82 -3.38 5 -0.59 4 5.59 3 0.18 2 Multiple linear regression for E-class. -0.95 -5.60 -0.03 -9.38 2.34 0.04 10 9 11 12 18 19 3.20 5.34 1.55 -5.92 -4.27 0.89 22 -2.84 -0.17 -0.40 -0.27 21 20 3.88 17 1.10 16 2.12 15 0.13 14 Objective measure 13 23 24 25 26 27 0.99 0.89 0.82 0.96 0.99 R^2 0.93 133.18 12.65 6.80 36.21 49.11 F-stat 13.04 Links between Subjective Assessments and Objective Metrics for Steering 81 82 Driver 1 Q6 (T-stat) Driver 2 Q10 (T-stat) Driver 3 Q2 (T-stat) Q3 (T-stat) Driver 4 Q7 (T-stat) Q12 (T-stat) Driver 5 Q4 (T-stat) Q8 (T-stat) Q10 (T-stat) Driver 6 Q10 (T-stat) Q11 (T-stat) Q12 (T-stat) -7.19 -5.91 3 2.68 2 0.57 1 4 5 -42.9 320 7.43 -0.04 1.03 0.19 9 8 -9.03 7 -0.09 6 Multiple linear regression for SUV. 166 2.36 10 -0.19 -8.38 2.93 6.52 15 6.67 0.05 -7.43 52.2 14 Objective measure 13 -0.36 1.38 -95.6 12 -0.33 11 16 17 18 19 -0.32 -47.1 -2.37 -30.3 -1.91 -7.78 -31.4 0.89 23.1 0.78 6.42 -8.07 -0.11 -1.60 22 7.67 21 0.75 20 23 24 25 -3.58 -7.09 -4.07 -4.11 4.41 3.84 3.02 27 2.86 26 0.98 0.99 1.00 1.00 0.97 0.98 1.00 0.98 0.96 0.99 0.99 R^2 1.00 48.97 33.91 495.10 3325.66 29.62 31.35 7914.13 42.35 23.98 122.78 40.58 F-stat 55551.4 9 8 7 6 5 4 3 2 1 9 8 7 6 5 4 3 2 1 9 12 11 >22 >23 8 >26 2 >23 2 25~30 2 >25 <4.8 1 1 3.5~4.5 1 7 6 5 4 3 2 1 10 Driver 3 12 11 10 Driver 2 12 11 10 Driver 1 Appendix D Subjective assessments Subjective assessments Subjective assessments 3 3 1~1.25 3 4 4 1.5~1.85 4 5 5 >1.4 5 >95 >90 >95 >100 - 6 6 - >90 6 7* 7* 7* <2.5 <2.4 <2.5 8 <2 <2 8 <2.2 8 >25 9 >25 9 9 10 >1.3 10 <1.8 10 11 11 >28.5 28~32 11 2.2~3 12 12 12 13 13 >90 >92 13 - 14 - 60~72 - 14 14 15 15 15 Objective measures - 16 16 16 17 17 45~75 17 18* 18* 18* >10 11~13 >13 19 10~14 >15 >16 19 19 3~6 3~6 20 4~6 20 3~6 5~6 20 21 21 21 1.5~2.2 1.5~2.2 1.5~2.2 22 >1.5 22 22 3.6~4.5 23 >3.6 >3.6 >3.7 23 23 >24 - - 24 24 24 <13 - - - 25 25 25 26 <3.4 26 <3.8 <3.8 <3.2 26 Links between Subjective Assessments and Objective Metrics for Steering 83 - - >1.5 >1.5 27 27 >2 27

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement