The kick-off of stage 4: testing and evaluating

The work package number 4 (WP4) can be seen as the core of the CLUG project and stretches over the entire duration of the project. Its main objectives are to test and evaluate the performance of localisation solutions (consisting of different receivers and sensors combined by different fusion algorithms) using real world data. The results will be evaluated against the actual speed and position of the train. Overall, it will produce the following outputs: 

• Statistics and graphs on the performance of different sensors  

• Statistics and graphs on the performance of different sensor fusion algorithms   

• Identification of environment and operational scenarios in which the performance of sensors is degraded  

• Identification of environments and operational scenarios in which the performance of fusion algorithms is degraded  

The launch of the WP4 took place at the end of July 2020. It identified 5 major steps:  

1- The test definition: “Where”, “When” and “How” to collect relevant data from a selection of sensors that will be installed on the train.   
WHERE: selecting lines and tracks where significant impacts on the evaluated sensors can be expected. The idea is to get a representative coverage of the variety of environments across the European rail network.  
WHEN: performing tests under time dependent conditions such as satellite constellations and weather or seasonal conditions.  
HOW: operational conditions that impact the sensors under evaluation such as standstill, speed, stopping and reversing in tunnels and rough driving.  
 
2- The train test and data collection: first, it implies the installation of the selected sensors on-board and second, the installation of the necessary equipment to properly record and store the data. Then, the data will be collected during commercial rides. Dedicated tests will also be scheduled to ensure that data is being collected under all environmental and operational conditions. This will result in the creation of a reference data set.  

3- The platform test: Siemens and Airbus are currently preparing the test platform to produce protection level, attitude, speed from data previously collected and algorithms provided from the work package 3. 

4- The ground truth: it prepares the software tools to generate the accurate position of the train during each test using data collected during the trip as well as GNSS referenced track data provided by the infrastructure manager. From that data, it generates the accurate train GNSS referenced position at regular intervals.  

5- Evaluation of test results: Siemens is currently preparing the software tools required to evaluate the test results. The tools generate graphical plots and statistics on the performance achieved during each test. Locations where significant degradation occurs will also be analysed to identify environmental conditions leading to these degradations.