Characterization of a Powertrain Test Bed in the Context of Virtual Validation of Drivability

Abstract

Technological trends in the automotive industry toward a software-defined and autonomous vehicle require a reassessment of today’s vehicle development process. The validation process soaringly shapes after starting with hardware-in-the-loop testing of control units and reproducing real-world maneuvers and physical interaction chains. Here， the road-to-rig approach offers a vast potential to reduce validation time and costs significantly. The present research study investigates the maneuver reproduction of drivability phenomena at a powertrain test bed. Although drivability phenomena occur in the frequency range of most up to 30$·$Hz， the design and characteristics substantially impact the test setup’s validity. By utilization of modal analysis， the influence of the test bed on the mechanical characteristic is shown. Furthermore， the sensitivity of the natural modes of each component， from either specimen or test bed site， is determined. In contrast， the uncertainty of the deployed measurement equipment also affects the validity. Instead of an accuracy class indication， we apply the ISO/IEC Guide 98 to the measurement equipment and the test bed setup to increase the fidelity of the validation task. In conclusion， the present paper contributes to a traceable validity determination of the road-to-rig approach by providing objective metrics and methods.

Keywords

powertrain test bed; virtual validation; modal analysis; powertrain-hardware-in-the-loop; drivability; measurement uncertainty

Major trends toward software-defined vehicles （SDV） and autonomous driving disclose new challenges in the automotive development process. New methods must speed up the development process to reduce costs and time to market. A potential solution for the central task of validation is the road-to-rig （R2R） approach. Testing is transferred from the road to the test bed^［1］. Depending on the test specimen， the unit under test （UUT）， a specific hardware-in-the-loop （HiL） application is realized. Such a setup is generally denominated as an X-in-the-loop （XiL） setup^［2］. Although， there are certain factors to consider， which lead to deviations between the actual road test and a simulation-based test bed scenario^［3］.

First， the test bed strongly influences maneuver reproduction in terms of system dynamics since the test bed components like adapters or dynamometers are not present in the reference car.

Furthermore， a simulation model， which is required to reproduce the residual vehicle dynamics， is not an exact representation of the real world. Instead， the residual vehicle model （RVM） must fulfill the optimal trade-off between computation demand and modeling accuracy.

Beyond that， a third factor is introduced by the measurement equipment. Even if the test bed matches the road testing maneuver exactly in combination with a precise simulation model， the measured signals within the XiL application are subject to measurement uncertainty （MU）^［4］.

The present paper is structured as follows： First， the differences between road testing and X-in-the-loop applications are discussed. In this context， a modal sensitivity analysis is conducted for a typical maneuver reproduction in vehicle drivability. Then， the relevance of measurement uncertainty is highlighted. Here， we consider the concepts of the ISO/IEC 98. Accordingly， both topics are consolidated into a new， objective measure for assessing the fidelity of a XiL application in general. For demonstration purposes， the XiL fidelity $R_{XiL}$ is calculated exemplarily. Finally， the findings of this paper are discussed， and guidance for future research is stated.

1 Differences between Road Testing and X-in-the-Loop-Applications

For discussing the deviations between the reference road test and the maneuver reproduction at the test bed， we utilize a setup according to Fig.1. The powertrain test bed consists of an Electric Drive Unit （EDU） as a UUT， side shafts， and adapters mechanically connect that to the two dynamometers （M2， M3）. Both dynamometers are controlled and supplied by the frequency converter. A battery simulator provides a high-voltage power supply， and a rest bus simulation emulates the Motor Control Unit （MCU） interface， which controls the inverter.

Fig.1  Powertrain test bed setup for drivability virtual validation

Torque and throttle control are the typical control modes for a specimen. The present case shows a setup with drive torque control and speed control at the load side.

Drivability refers to the subjective feeling of the vehicle’s response to the driver’s inputs focusing on vehicle longitudinal dynamics^［5］. A drivability maneuver reproduction at a powertrain test bed requires an adequate representation of the residual vehicle dynamics. Therefore， a residual vehicle model is incorporated into the control loop. Drivability maneuvers like tip-in or driveaway require at least a longitudinal dynamics vehicle model including simulation of the tire-road-interaction， the chassis dynamics， and a driver for vehicle control.

Major differences between the reference road test and the virtualized variant at the test bed occur due to：

（1） Test bed-specific components that are not present in the vehicle： Dynamometers， adapters， and measurement equipment.

（2） The limited accuracy of the simulation models of all residual vehicle components.

（3） Signal delay and dead time at the test bed within the whole control loop.

（4） Uncertainty in the measurement results at the test bed because of perturbations and standard uncertainties.

As a result， the system dynamics between the actual vehicle setup and the test bed substitution differ from one another. First， we discuss the differences in system dynamics in the upcoming Chapter 2. Chapter 3 provides a detailed analysis of the measurement uncertainty of the existent R2R setup. The other effects mentioned are not investigated in this paper， but a reference to relevant literature is made available. Studies regarding the simulation model accuracy of drivability models are given by Ref. ［6-9］. An analysis of the signal delay and system identification of a powertrain test bed is presented in Ref. ［3］. In summary， all aspects have been subjects of investigation， which are combined in a new R2R or XiL fidelity measure in Chapter 4.

2 Modal Analysis and System Dynamics of a Powertrain Test Bed

The fundamentals of modal analysis allow for the calculation of system dynamics in the frequency range. In this chapter， the basic equations are introduced， which are utilized for the evaluation of differences between road tests and reproduction at the test bed. In our case study， a battery electric vehicle （BEV） powertrain topology is examined.

2.1　Modal Analysis Fundamentals

A general mechanical system is stated in Eq. （1）.

$$\underline{M}\vec{\ddot{q}}+\underline{B}\vec{\dot{q}}+\underline{C}\vec{q}=\vec{\mathrm{0}}$$

（1）

where： $\underline{M}$， $\underline{B}$ and $\underline{C}$ mean the mass， damping， and stiffness matrices， and $\vec{q}$ is the vector of generalized coordinates^［10］. For a drivability model， which is characterized mainly by a torsional oscillation model， $\vec{q}$ is a vector of generalized rotational angles. For simplification， an undamped system is considered. In the undamped scenario， the specific eigenvalue problem leads to Eq. （2）.

$$\left({\underline{M}}^{-\mathrm{1}}\underline{C}-{\vec{\omega }}^{\mathrm{2}}\underline{E}\right)\vec{v}=\vec{\mathrm{0}}$$

（2）

where： E reflects the identity matrix and $\vec{v}$ refers to the eigenvector. Ref. ［10］ derives the demanded equation for the eigenfrequencies ${\omega }_{\mathrm{0}i}$ of the eigenvalue problem （see Eq. （3））.

$${\omega }_{\mathrm{0}i}=\sqrt[]{\frac{{\gamma }_i}{{\mu }_i}}=\sqrt[]{\frac{{v_i}^T\underline{C}{v}_i}{{v_i}^T\underline{M}{v}_i}},i=\mathrm{1,2},\dots ,n$$

（3）

We want to highlight at this point， that the eigenfrequencies of each of the $n$ modal modes are dependent on the ratio of modal stiffness ${\gamma }_i$ and masses ${\mu }_i$ in the same mode. Eq. （4） and （5） illustrate how the modal stiffness and masses are determined in general for a free torsional oscillation model based on the modal properties （torsional stiffness $c_{Tk}$， rotational inertia $J_k$） of each component $k$ in the dynamic system ［10］.

$${\gamma }_i={\displaystyle \sum _{k=\mathrm{0}}^n}{c}_{Tk}{\left(v_{ki}-v_{k+\mathrm{1},i}\right)}^{\mathrm{2}},i=\mathrm{1,2},\dots ,n$$

（4）

$${\mu }_i={\displaystyle \sum _{k=\mathrm{1}}^n}{J}_k{v}_{ki}^{\mathrm{2}},i=\mathrm{1,2},\dots ,n$$

（5）

Based on the concept of modal mass and stiffness， we can define the modal sensitivity of each component^［10］. Eq. （6） defines the modal sensitivity coefficients ${\gamma }_{ik}$ in terms of stiffness， whereas Eq. （7） shows the corresponding sensitivity coefficients ${\mu }_{ik}$ regarding mass.

$${\gamma }_{ik}=\frac{c_{Tk}{\left(v_{ki}-v_{k+\mathrm{1},i}\right)}^{\mathrm{2}}}{{\gamma }_i},i=\mathrm{1,2},\dots ,n$$

（6）

$${\mu }_{ik}=\frac{J_k{v}_{ki}^{\mathrm{2}}}{{\mu }_i},i=\mathrm{1,2},\dots ,n$$

（7）

The sensitivity factors of stiffness $k$ and mass $l$ can be used to estimate the impact on the $i$-th eigenfrequency （Eq. （8））^［10］：

$$\frac{∆f_i}{f_{i\mathrm{0}}}\approx \frac{\mathrm{1}}{\mathrm{2}}\left({\gamma }_{ik}\frac{∆c_{Tk}}{c_{Tk}}-{\mu }_{il}\frac{∆J_l}{J_l}\right)$$

（8）

We suggest the utilization of modal sensitivity coefficients to assess the difference in system dynamics by conducting an R2R approach.

2.2　Torsional Models of the Reference Vehicle and the Powertrain Test Bed

The simulation models used for the modal sensitivity analysis are shown in Fig.2. The reference vehicle model （a） consists of an electric motor （EM）， a gearbox （G）， a differential （D）， and drive shafts （DS） for the left and rear side of the driven axle. Moreover， both side shafts are connected to a wheel （W） and tire （T） subsystem， which represents the contact with the road. For longitudinal dynamics， the powertrain moves the vehicle body （V） in the longitudinal direction， meaning a mechanical connection of the vehicle body to the tire via the road. Each component is represented by its rotational inertia （$J$） and torsional stiffness （$c_T$）.

Fig. 2  Topology of torsional oscillation models: (a) Reference vehicle model (above); (b) Powertrain test bed setup (below)

On the opposite side， there is the powertrain test bed model （b）. The UUT is assembled up to the drive shafts， but additional components are required for adaptation and measurement： A wheel hub adapter （A）， a torque sensor （M）， the rotor shaft of each dynamometer （M2， M3）， and the speed sensor （n）.

2.3　Modal Sensitivity Analysis

A model sensitivity analysis is executed in the following as described in Section 2.1. All parameters are reduced to the rotor shaft of the UUT. The equations of motion are derived from Lagrangian mechanics as described in Ref. ［10］. The results are presented in Fig. 3.

Fig.3  Modal analysis of a powertrain subsystem: Analysis of the powertrain deployed in the reference vehicle (left) and assembled at a powertrain test bed (right)

In both dynamics systems， the first torsional oscillation mode occurs at 0 Hz， as both systems are not fixed. A comparison is more reasonable by starting from the fundamental mode. This actual first torsional oscillation mode is known as the shunt frequency^［11］. In a vehicle setup， this mode is characterized by oscillation of the drive side （engine， gearbox） against the combination of the wheel-tire-subsystem and the vehicle body. In the present example， the shuffle frequency is located at about 1.6 Hz in the battery electric reference vehicle. The second relevant mode is the wheel tire mode， which resides at about 46 Hz in the vehicle. We like to point out， that in our present case， some modes occur twice due to the almost symmetric design of the powertrain.

In contrast to the reference vehicle setup， the same powertrain assembled at a powertrain test bed shows a completely different modal behavior. Here， the shuffle frequency is shifted to a higher frequency of about 16.7 Hz. Furthermore， the torsional tire mode of the vehicle system is fully erased， because the system at the powertrain test bed has changed. There are no tires or the vehicle body existent. Only the gearbox mode （eigenfrequency number 6 in the vehicle） is also available at the test bed （test bed mode number 4）.

An in-depth study of the differences between both dynamic systems is provided by a modal sensitivity breakdown. Utilization of Eq. （4）－（8） provides detailed information about the distribution of the kinetic and potential energy. In Fig.3， the center row illustrates the kinetic energy distribution at the shuffle frequency in both dynamic scenarios. Analog， the bottom row demonstrates the potential energy distribution. In the latter case， the drive shafts are most relevant for the shuffle mode in both situations. On the other hand， the most important mass parameter is the combined mass of the wheel， tire， and vehicle body in the reference case. This changes at the powertrain test bed， where the rotor shafts of the load units are most crucial for the shuffle frequency.

2.4　Conclusion

By conducting a modal analysis， the differences in the dynamic systems of the reference vehicle and the powertrain test bed are evaluated. To reproduce whole vehicle dynamic maneuvers at a powertrain test bed， additional simulation by a residual vehicle model is required to account for the frequency shift of the basic test bed setup. Beyond that， a simulation model of the powertrain test bed assists in tuning the frequency response of the dynamic system at the test bed. This process is called frequency matching. Both approaches benefit from the knowledge of a modal sensitivity analysis， the implementation of the RVM， and the powertrain test bed model. Parameters， which are not sensitive to the natural mode to be reproduced at the test bed are not very important for modeling. Hence， the effort for adequate system identification can be reduced significantly.

3 The Role of Measurement Uncertainty

3.1　The Guide to the Expression of Measurement Uncertainty （GUM） ISO/IEC 98

Based on systematic and random errors， measurement uncertainty occurs in every measurement process. The term measurement uncertainty refers to a non-negative parameter that reflects the statistical distribution of quantity values considered with the measurement result^［12］. This definition is based on the ISO/IEC 98， the "Guide to the Expression of Measurement Uncertainty" （GUM）， a comprehensive guideline for determining measurement uncertainty. In contrast to classical error propagation methodologies， MU quantifies systematic and random errors and allocates a probability distribution to the same. A complete notation of a measurement result according to GUM is stated in Eq.（9）^［13］：

$$Y\approx y\pm U=y\pm k\cdot u_c\left(y\right)$$

（9）

$Y$ means the true measurement value， which is unknown in general. Therefore， the best estimation $y$ is determined with an extended uncertainty $U$. $U$ incorporates the measurement uncertainty as a combined measurement uncertainty $u_c$ alongside a probability distribution. A confidence level of 95% is usually sufficient， which correlates to an extension factor $k=\mathrm{2}$^［14］.

Having regard to the standard uncertainties $u\left(x_i\right)$ of each of all $N$ input quantities affecting the measuring result and their specific sensitivity coefficients $c_i$， the combined measurement uncertainty is calculated （Eq.（10））^［15］. In the case of non-independent input quantities in the measurement system， the term $B\left(x_i,x_j\right)$ is demanded to calculate the covariance of dependent signals （Eq.（11））^［15］. Furthermore， we determine the standard uncertainty $u\left(x_i\right)$ in Eq.（12） considering the corresponding weights $G_i$ and the maximum values $a_i$ of each relevant input quantity to the measurement system^［14］.

$$u_c\left(y\right)=\sqrt[]{{\displaystyle \sum _{i=\mathrm{1}}^N}{\left(c_i\cdot u\left(x_i\right)\right)}^{\mathrm{2}}+B\left(x_i,x_j\right)}$$

（10）

$$B\left(x_i,x_j\right)=\mathrm{2}{\displaystyle \sum _{i=\mathrm{1}}^{N-\mathrm{1}}}{\displaystyle \sum _{j=i+\mathrm{1}}^N}{c}_i·c_j·u\left(x_i\right)·u\left(x_j\right)·r\left(x_i,x_j\right)$$

（11）

$$u\left(x_i\right)=\sqrt[]{G_i}·a_i$$

（12）

The ascertainment of the sensitivity factor $c_i$ is based on data sheets from the equipment supplier or by utilization of the error propagation laws. In contrast， a probability distribution of the standard uncertainty provides information on the weights $G_i$ and maximum values $a_i$.The GUM approach is structured as follows^［14］：

（1） Problem analysis；

（2） Determination of the measurand；

（3） Data pre-processing （e.g. documentation of environmental conditions）；

（4） （Statistical） data evaluation；

（5） Build-up and evaluation of a system model for the measurement process；

（6） Calculation of the measurement uncertainty；

（7） Notation of the complete measurement result.

Applying the GUM framework is controversial because non-statistical quantities are evaluated for fidelity. Nevertheless， the GUM framework is recommended in many recent studies^［4，14，16］ and serves as a basis for calibration laboratories.

3.2　Application of GUM 98 at a Powertrain Test Bed XiL Application

In the following sections， we execute the standard GUM framework exemplarily for a typical drivability application. The setup shown in Fig.1 is selected. In the exemplary drivability scenario， the conditions in Tab.1 apply. $M_{M\mathrm{2}}$ and $M_{M\mathrm{3}}$ refer to the torque operation range of the load dynos for a typical full-throttle driveaway maneuver， while $n_{\mathrm{M}\mathrm{2}}$ and $n_{\mathrm{M}\mathrm{3}}$ define the angular speed range respectively. Furthermore， the environmental conditions are characterized by the ambient temperature $T_R$ in the test cell and parasitic loads acting at the torque transducers. The axial force $F_x$ is neglected. In contrast， the radial force $F_y$ and the bending torque $M_b$ are analyzed. The calculation of the last two components is performed based on a static beam model of each load side （M2， M3）. Since the beam model is statically undetermined， we utilize Castigliano´s method for calculating the bearing reactions^［10］.

Tab.1  Reference drivability scenario

Parameter	Load side	Value		Unit
		Min.	Max.
$M$	M2 / M3	0	1 500	N·m
$n$	M2 / M3	0	380	1/min
$T_R$	M2 / M3	20	25	°C
$F_x$	M2 / M3	0		N
$F_y$	M2	204.38		N
	M3	218.02
$M_b$	M2	34.48		N·m
	M3	29.11

We consider torque and angular speed sensors at the load dynamometers for a complete measurement uncertainty evaluation of a drivability Virtual Validation application case.

3.2.1　Analysis of the Torque Sensor

Each dynamometer has a torque transducer of type HBM T12HP 5 kN·m at the drive side. The torque transducer uses strain gauges at the rotor shaft and transmits the measured torque proportional via a frequency output. As an Ishikawa diagram， Fig.4 shows the input quantities affecting the torque measurement uncertainty. The combined measurement uncertainty is calculated using the information provided by the technical data sheet^［17］ and knowledge about the probability distribution of each parameter^［18］. Moreover， the measurement setup at the test bed is also considered for the parasitic loads， as described previously. The resulting standard uncertainties are shown in Tab.4. Finally， the input quantities lead to a combined torque measurement uncertainty of $U_{\mathrm{M}\mathrm{2}/\mathrm{M},\mathrm{95}\mathrm{\%}}\approx \mathrm{0.689}\mathrm{N}\cdot \mathrm{m}$ or about $\mathrm{0.046}\mathrm{\%}$ at the left load side and analog $U_{\mathrm{M}\mathrm{3}/\mathrm{M},\mathrm{95}\mathrm{\%}}\approx \mathrm{0.677}\mathrm{N}\cdot \mathrm{m}$ or about $\mathrm{0.045}\mathrm{\%}$ on the opposite side.

Fig. 4  Ishikawa diagram for the torque sensor HBM T12HP 5 kN·m

3.2.2　Analysis of the Angular Speed Sensor

In addition to a torque sensor at the drive side of each dyno， an angular speed sensor is deployed on the opposite side. In this specific case， sensors of type HEIDEHAIN ECN1313 with 2，048 increments in total come into operation. Analog to the torque sensors， the factors influencing the angular speed measurement result are indicated in Fig.5. The corresponding standard uncertainties are presented in Tab.4 as well and are predicted on the supplier´s technical data sheet^［19］. Execution of the same GUM framework for the speed sensor yields a combined torque measurement uncertainty of $U_{n,\mathrm{95}\mathrm{\%}}\approx \mathrm{1.597}/\mathrm{m}\mathrm{i}\mathrm{n}$ or about $\mathrm{0.42}\mathrm{\%}$.

Fig.5  Ishikawa diagram for the speed sensor HEIDENHAIN ECN1313-2048

3.2.3　Interim Summary

The example of a drivability Virtual Validation allows quantifying each sensor’s combined measurement uncertainty under actual conditions. As pointed out previously， the general acceptance of Virtual Validation methods strongly depends on objective and standardized metrics for fidelity assessment of the R2R approach. The utilization of the GUM framework for a precise rating of measurement equipment has been used more and more in recent research. It should be considered in Virtual Validation.

4 A New Measure of Test Bed Validity: The XiL Fidelity ${\mathit{R}}_{\mathbf{X}\mathbf{i}\mathbf{L}}$

Various measures exist to determine the accordance between a real system and a simulation model-based approach. For example， the goodness of fit can be expressed by one of the following measures：

（1） The Pearson correlation coefficient $r$^［20］；

（2） A （normalized） root mean square error （NRMSE/RMS）^［21］；

（3） The predicted sum of squares （$\mathrm{P}\mathrm{R}\mathrm{E}\mathrm{S}\mathrm{S}$）^［21］.

Nevertheless， those measures only consider differences between two sets of data samples for each data point in the ordinate. All the other aspects， which were shown in Chapters 2 and 3， of the deviation between a XiL application and the corresponding real-world test are not regarded. Hence， a new measure for the validity of such an application is needed to raise trust in the methods of Virtual Validation. In previous studies， Dos Santos et al.^［22］ suggested an alternative way to calculate the quality of validation of a HiL application. The so-called HiL representativeness is given in Eq.（13） for a reference test group $A$.

$$R_A={\displaystyle \sum _N^{i=\mathrm{1}}}\frac{t_i}{t_T}{K}_i{C}_i,$$

（13）

Where： $t_i$ is the test execution time for a sample and $t_T$ means the overall test run time of the reference group $A$. A shape factor is introduced by $K_i$ and $C_i$ refers to the test's reliability. The parameter $R_A$ is a characteristic value for HiL-based control unit testing applications， where faults and uncertainties are investigated in the I/O boards. In the process of a literature review on drivability as a potential field of Virtual Validation， the potential of such measures for deriving standards for Virtual Validation is discussed in detail^［11］. However， the effects of the test bed system， the quality of the simulation model for a residual vehicle simulation， and the uncertainty of measurement lead to a more sophisticated approach， which is defined and demonstrated subsequently.

4.1　Definition

We define the fidelity of an X-in-the-loop application $R_{\mathrm{X}\mathrm{i}\mathrm{L}}$ as Eq.（14）：

$$R_{\mathrm{X}\mathrm{i}\mathrm{L}}=\sqrt[]{{\displaystyle \sum _{i=\mathrm{1}}^{\xi }}{w}_i{R}_i^{\mathrm{2}}},i=\mathrm{1},\dots ,\xi$$

（14）

$${\displaystyle \sum _{i=\mathrm{1}}^{\xi }}{w}_i=\mathrm{1},i=\mathrm{1},\dots ,\xi$$

（15）

In this context， $w_i$ refers to a weighting function and $R_i$ means the fidelity of each of the $\mathrm{\xi }$ domains of the XiL application. The impact of each fidelity contribution is incorporated by utilizing weighting functions in Eq.（15）. By definition， a $R_{\mathrm{X}\mathrm{i}\mathrm{L}}=\mathrm{1}$ represents an ideal XiL application， the exact representation of an actual real-world test. In contrast， $R_{\mathrm{X}\mathrm{i}\mathrm{L}}=\mathrm{0}$ means no representation at all. All subsequently introduced partial fidelity measures are specified as $R\in \left[\mathrm{0,1}\right]$. It is crucial to note that the determination of $R_{\mathrm{X}\mathrm{i}\mathrm{L}}$ is only valid for a certain application or maneuver. Important to mention that a XiL setup can be utilized for various test scenarios， and the XiL fidelity of each realization may vary.

Considering a drivability Virtual Validation setup， the relevant domains for the fidelity calculation are： The system dynamics （SD） of the test bed， including the specimen or unit under test （UUT）； The model fidelity of the residual vehicle model （RVM）； The impact of the measurement uncertainty （MU） of the overall test bed setup.

Therefore， the fidelity of a drivability XiL application is determined as：

$$R_{\mathrm{X}\mathrm{i}\mathrm{L}}=\sqrt[]{w_{\mathrm{S}\mathrm{D}}{R}_{\mathrm{S}\mathrm{D}}^{\mathrm{2}}+w_{\mathrm{R}\mathrm{V}\mathrm{M}}{R}_{\mathrm{R}\mathrm{V}\mathrm{M}}^{\mathrm{2}}+w_{\mathrm{M}\mathrm{U}}{R}_{\mathrm{M}\mathrm{U}}^{\mathrm{2}}}$$

（16）

The partial fidelity for the system dynamics $R_{\mathrm{S}\mathrm{D}}$ is expressed by the deviation of each of the $m$ relevant signals in the time domain for the x-axis and y-axis （Eq.（17））. On the one hand， the first deviation refers to the time delay between the actual system dynamics and the replacement system. At the same time， the latter describes the gap in the ordinate axis （e.g. torque）.

$$R_{\mathrm{S}\mathrm{D}}=\sqrt[]{\frac{\mathrm{1}}{\mathrm{2}m}{\displaystyle \sum _{i=\mathrm{1}}^m}{R}_{\mathrm{N}\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E},i}^{\mathrm{2}}+R_{{\tau }_D,i}^{\mathrm{2}}},i=\mathrm{1},\dots ,m$$

（17）

For an exemplary signal that is representative of the maneuver reproduction， the ordinate deviation is determined by the NRMSE of $l$ sample points in Eq.（18）^［21］：

$$R_{\mathrm{N}\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}=\mathrm{1}-\sqrt[]{\frac{\frac{\mathrm{1}}{l}{\displaystyle \sum _{i=\mathrm{1}}^l}{\left(y_i-{\widehat{y}}_i\right)}^{\mathrm{2}}}{y_{\mathrm{m}\mathrm{a}\mathrm{x},i}-y_{\mathrm{m}\mathrm{i}\mathrm{n},i}}},i=\mathrm{1},\dots ,l$$

（18）

Where： $y_{\mathrm{m}\mathrm{a}\mathrm{x},i}$ and $y_{\mathrm{m}\mathrm{i}\mathrm{n},i}$ represent the maximum and minimum data points of a reference signal， $y$ is the reference signal and $\widehat{y}$ refers to the test signal.

In contrast to the ordinate deviation， the influence of the time delay ${\tau }_D$ is calculated by utilization of a modified version of the arctan-function （Eq.（19））^［23］：

$$R_{{\tau }_D}=\mathrm{1}-\frac{\mathrm{2}}{\pi }\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{t}\mathrm{a}\mathrm{n}(\chi ·{{\tau }_D}^{\mathrm{3}})$$

（19）

A robust calculation of a time delay between two signals is conducted by analyzing the signals' cross-covariance or comparing the step responses of both related systems.

As mentioned before， the XiL fidelity depends on the selected maneuver to be represented. Thus， the coefficient $\mathrm{\chi }$ allows for adjustment of the impact of the time delay concerning the maximum relevant frequency $f_{\mathrm{m}\mathrm{a}\mathrm{x}}$ excited during the test scenario. For good control loop response and stability， Lunze^［24］ recommends a minimum operation frequency $f_T\approx \mathrm{6}{f}_{\mathrm{m}\mathrm{a}\mathrm{x}}$. Hence， we suggest stating a time delay related to fidelity $R_{{\tau }_D}=\mathrm{0.5}$， if $\mathrm{1}/{\tau }_D$ reaches the minimum operation frequency $f_T$. Accordingly， the assignment of $\mathrm{\chi }$ is given by Eq.（20）：

$$\chi =\mathrm{t}\mathrm{a}\mathrm{n}\left(\frac{\pi }{\mathrm{4}}\right)/{{\tau }_D}^{\mathrm{3}}=\mathrm{t}\mathrm{a}\mathrm{n}\left(\frac{\pi }{\mathrm{4}}\right)/{\left(\frac{\mathrm{1}}{\mathrm{6}{f}_{\mathrm{m}\mathrm{a}\mathrm{x}}}\right)}^{\mathrm{3}}$$

（20）

An exemplary curve for the determination of $R_{{\mathrm{\tau }}_{\mathrm{D}}}$ for a maneuver reproduction with $f_{\mathrm{m}\mathrm{a}\mathrm{x}}=\mathrm{30}·\mathrm{H}\mathrm{z}$ is presented in Fig.6.

Fig.6  Exemplary determination of $R_{{\tau }_D}$ for a maneuver reproduction with $f_{\mathrm{m}\mathrm{a}\mathrm{x}}=\mathrm{30}·\mathrm{H}\mathrm{z}$

Analog to the system dynamics analysis of the test bed system， the same concept is applied to the residual vehicle model （Eq.（21））.

$$R_{\mathrm{R}\mathrm{V}\mathrm{M}}=\sqrt[]{\frac{\mathrm{1}}{\mathrm{2}m}{\displaystyle \sum _{i=\mathrm{1}}^m}{R}_{\mathrm{N}\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E},i}^{\mathrm{2}}+R_{{\tau }_D,i}^{\mathrm{2}}},i=\mathrm{1},\dots ,m$$

（21）

Here， the same definitions for calculating the differences in the x- and y-coordinate occur.

Finally， we consider the measurement uncertainty $U_i$ of each of the m relevant signals as the last component influencing the XiL fidelity：

$$R_{\mathrm{M}\mathrm{U}}=\mathrm{1}-\sqrt[]{\frac{\mathrm{1}}{m}{\displaystyle \sum _{i=\mathrm{1}}^m}{\left(R_{\mathrm{M}\mathrm{U},i}\right)}^{\mathrm{2}}},i=\mathrm{1},\dots ,m$$

（22）

$$R_{\mathrm{M}\mathrm{U},i}=\frac{U_i}{y_{\mathrm{m}\mathrm{a}\mathrm{x},i}-y_{\mathrm{m}\mathrm{i}\mathrm{n}}},i=\mathrm{1},\dots ,m$$

（23）

We normalize the measurement uncertainty of each signal by diving by the range of the maximum （$y_{\mathrm{m}\mathrm{a}\mathrm{x},i}$） and minimum （$y_{\mathrm{m}\mathrm{i}\mathrm{n},i}$） value in the measured signal.

After introducing the basic formula， an exemplary drivability application is presented next to demonstrate the usage of the XiL fidelity $R_{\mathrm{X}\mathrm{i}\mathrm{L}}$.

4.2　Exemplary Application

For maneuver reproduction， we consider an exemplary scenario of a driveaway event. As shown in Chapter 1， the relevant signals at the powertrain test bed are the torque signal M and the angular speed signal n at each load dynamometer （M2， M3）. As mentioned before， for such a drivability-related test case， the maximum frequency is stated with $f_{\mathrm{m}\mathrm{a}\mathrm{x}}=\mathrm{30}·\mathrm{H}\mathrm{z}$. In terms of the residual vehicle model， the signals to be compared to the road test are stated with the longitudinal vehicle acceleration $a_{\mathrm{v}\mathrm{e}\mathrm{h},x}$ and velocity $v_{\mathrm{v}\mathrm{e}\mathrm{h},x}$.

To execute the calculation of $R_{\mathrm{X}\mathrm{i}\mathrm{L}}$， an assumed data set based on knowledge from former test bed analysis and reference literature is chosen （see Tab.2）.

Tab.2  Exemplary data set for calculation of $R_{\mathrm{X}\mathrm{i}\mathrm{L}}$

Part 1： System dynamics $R_{\mathrm{S}\mathrm{D}}$
Parameter	Load side	Value	Unit
$R_{\mathrm{N}\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E},M}$	M2 / M3	95	%
$R_{\mathrm{N}\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E},n}$	M2 / M3	98	%
${\tau }_{D,M}$	M2 / M3	0.005	s
${\tau }_{D,n}$	M2 / M3	0.010	s
Part 2： Residual vehicle model $R_{\mathrm{R}\mathrm{V}\mathrm{M}}$
$R_{\mathrm{N}\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E},a_{\mathrm{v}\mathrm{e}\mathrm{h},x}}$	-	85	%
$R_{\mathrm{N}\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E},v_{\mathrm{v}\mathrm{e}\mathrm{h},x}}$	-	95	%
${\tau }_{D,a_{\mathrm{v}\mathrm{e}\mathrm{h},x}}$	-	0.001	s
${\tau }_{D,v_{\mathrm{v}\mathrm{e}\mathrm{h},x}}$	-	0.000	s
Part 3： Measurement uncertainty $R_{\mathrm{M}\mathrm{U}}$
$R_{\mathrm{M}\mathrm{U},M}$	M2	99.94	%
	M3	99.95
$R_{\mathrm{M}\mathrm{U},n}$	M2 / M3	99.58	%

By application of Eq.（16）－（22） to the data set， a calculation of each component of the XiL fidelity is conducted. On this， the selected weights for the drivability scenario are $w_{\mathrm{S}\mathrm{D}}=\mathrm{0.4},$ $w_{\mathrm{R}\mathrm{V}\mathrm{M}}=\mathrm{0.4}$ and $w_{\mathrm{M}\mathrm{U}}=\mathrm{0.2}.$ Thereby， the impact of system dynamics and simulation model deviations are emphasized against the measurement uncertainty. A final summary of results is given in Tab.3.

Tab.3  Summary of results for determination of the XiL fidelity $R_{\mathrm{X}\mathrm{i}\mathrm{L}}$

Fidelity measure	Value	Unit
$R_{\mathrm{S}\mathrm{D}}$	61.01	%
$R_{\mathrm{R}\mathrm{V}\mathrm{M}}$	77.65	%
$R_{\mathrm{M}\mathrm{U}}$	99.76	%
${\mathit{R}}_{\mathrm{X}\mathrm{i}\mathrm{L}}$	76.75	%

Various realizations of test bed setups for maneuver reproduction and Virtual Validation can be compared objectively. Besides， such a fidelity measure could be used for deriving a standardized process to compare such development tools.

5 Summary and Outlook

Virtual Validation is receiving continuously more attention as it shows a significant potential for cost reduction and short development cycles. Current trends like the software-defined vehicle demand smart and effective solutions and tools for faster development cycles and a higher degree of agility. In general， state-of-the-art methods for Virtual Validation do not provide objective and comprehensive approaches for estimating the validity of a road-to-rig approach. In this paper， we looked at vehicle drivability， or shuffle in particular， as a relevant subject for vehicle validation.

This paper deals with three key aspects for comparison of the overall closed-loop system of a powertrain in the case of road testing （reference） and at a powertrain test bed （R2R target）：

First， modal analysis of both dynamic systems is recommended for assessment of the sensitivity of each parameter for the torsional modes to be reproduced at the test bed. A practical example of a battery electric powertrain architecture shows a frequency shift of the shuffle frequency of about 15 Hz at the test bed， which has to be compensated.

Second， a precise analysis of the measurement uncertainty of the most relevant measurement equipment according to the actual maneuver is done. In the exemplary drivability R2R scenario， the measurement of torque and angular speed at the test bed is most essential. Here， we adapt the GUM resulting in a more specific evaluation of the systematic and random errors.

Finally， the first two aspects are merged with an evaluation of the model accuracy of the residual vehicle model to an overall XiL fidelity measure $R_{XiL}$. This measure is objective， can be generalized and traceable.

Future research should investigate into the XiL fidelity measure for other application cases or fields of research. Beyond that， this measure allows for objective benchmarking of various XiL setups. As a result， state-of-the-art XiL setups can be optimized and the complete development process of a powertrain test bed can be improved. Here， the XiL fidelity measure could be put into the requirement specification of future test beds with focus on Virtual Validation.