Telemetry and shared memory tools

Discussion in 'Community Workshop' started by pixeljetstream, Jul 11, 2015.

  1. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    Update: the application is not compatible with current r3e version anymore. A future version will fix this, but I don't want to promise any date.

    Hi,
    to get a better understanding how the values behave and to help debug other tools, I've started a new github project r3e-trace-analysis.

    It can record multiple sessions while r3e is running and replay such recorded trace files.The replay should trick other apps using shared memory to think it's R3E running.

    There is a basic ui tool now that allows to browse through recorded sessions, as well as visualize recorded values of a single lap. It does require OpenGL 3.3

    [​IMG]

    Comparing laps is now possible

    [​IMG]
    There is also an option to animate the patterns

    [​IMG]

    When right-clicking another lap its values are directly compared to the current active lap and the compared lap is drawn colored based on these differences. Blue value means the other lap has a lower value than the reference lap. You can also modify the "Gradient" value prior the comparison to visualize where the differences changed.

    [​IMG]

    You can also visualize values next to each other. And again using Gradient operator on something like gears, you can also compare when you were shifting.

    [​IMG]

    Using the "Range" button, you can select to visualize a sub-range of the track. All plots will get cut by the duration the fastest plot took. For example in this image you can see we lost a lot of time in the beginning of the track in lap 4 (lap 4's left side vs. lap 5's right side).

    [​IMG]

    However when starting our analysis after the troubled section we can see that both laps ended up pretty similar in duration.

    [​IMG]

    With "File->Append" one can load another set of recorded laps to compare with. However, there is no compatibility check whether the same car or track was used. Below we compare the fastest lap from race with lap from qualifying.

    [​IMG]

    You can export CSV files to create plots for selected properties

    [​IMG]


    TODO
    • viewer: line graph, bar graph, 2d scatter plot
    • viewer: add support for custom scripted properties
    • consider zip encoding for trace files (currently raw is ~4 MB per minute)
     
    • Like x 9
    • Wonderful x 4
    • Winner x 2
    • Useful x 2
    • Love it! x 1
    Last edited: Apr 12, 2017
  2. mr_belowski

    mr_belowski Well-Known Member Beta tester

    Joined:
    Apr 23, 2015
    Ratings:
    +1,307 / 0 / -0
    Awesome, that's a huge help for all app developers using the shared memory block
     
    • Useful Useful x 1
  3. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    added full introspection on all shared memory properties. It is automatically generated from the C header and is required for properly displaying the state later.

    also bugfix, forgot the CutTrackPenalties in the r3e struct, not reflected in the sample dump below.

    the replayer dumps the info now every 2 seconds

    Code:
    PLAYER TIME    12.014
    r3e_float32     BrakeBias    0.41999998688698
    r3e_float32     BrakePedal    0
    r3e_float32     BrakeTemperatures.FrontLeft    20.682710647583
    r3e_float32     BrakeTemperatures.FrontRight    20.68271446228
    r3e_float32     BrakeTemperatures.RearLeft    20.60005569458
    r3e_float32     BrakeTemperatures.RearRight    20.600057601929
    r3e_vec3_f32    CarCgLoc    {106.7409286499,0.65895563364029,-124.18936920166}    163.75915087187
    r3e_float32     CarDamage.Aerodynamics    0
    r3e_float32     CarDamage.Engine    0
    r3e_float32     CarDamage.TireFrontLeft    0
    r3e_float32     CarDamage.TireFrontRight    0
    r3e_float32     CarDamage.TireRearLeft    -1
    r3e_float32     CarDamage.TireRearRight    -1
    r3e_float32     CarDamage.Transmission    0
    r3e_float32     CarOrientation.Pitch    0.0085065197199583
    r3e_float32     CarOrientation.Roll    0.00022280527628027
    r3e_float32     CarOrientation.Yaw    -1.0053857564926
    r3e_float32     CarSpeed    29.575429916382
    r3e_float32     ClutchPedal    0
    r3e_int32       CompletedLaps    0
    r3e_int32       ControlType    0
    r3e_int32       CutTrackWarnings    0
    r3e_int32       DrsAvailable    0
    r3e_int32       DrsEngaged    0
    r3e_float32     EngineOilPressure    258.64730834961
    r3e_float32     EngineOilTemp    43.904235839844
    r3e_float32     EngineRps    465.48831176758
    r3e_float32     EngineWaterTemp    37.810943603516
    r3e_int32       EventIndex    0
    r3e_int32       Flags.Black    0
    r3e_int32       Flags.Blue    0
    r3e_int32       Flags.Yellow    0
    r3e_float32     FuelCapacity    110
    r3e_float32     FuelLeft    50
    r3e_float32     FuelPressure    58.5
    r3e_int32       FuelUseActive    0
    r3e_int32       Gear    3
    r3e_float32     LapTimeBest    -1
    r3e_float32     LapTimeBestLeader    -1
    r3e_float32     LapTimeBestLeaderClass    -1
    r3e_float32     LapTimeCurrent    2.852073431015
    r3e_float32     LapTimeDeltaLeader    -1
    r3e_float32     LapTimeDeltaLeaderClass    -1
    r3e_float32     LapTimeDeltaSelf    -1
    r3e_float32     LapTimePrevious    -1
    r3e_vec3_f32    LocalAcceleration    {0.028581578284502,0.63809359073639,2.4627497196198}    2.5442320095941
    r3e_float32     MaxEngineRps    785.39819335938
    r3e_int32       NumCars    20
    r3e_int32       NumPenalties    0
    r3e_int32       NumberOfLaps    -1
    r3e_int32       PitWindowEnd    -1
    r3e_int32       PitWindowStart    -1
    r3e_int32       PitWindowStatus    2
    r3e_vec3_f64    Player.Acceleration    {-2.0687398261174,0.61713249292261,1.3463206766616}    2.5442319757427
    r3e_vec3_f64    Player.AngularAcceleration    {-0.075149766560623,-0.023070912310586,-0.64749374898337}    0.65224834946629
    r3e_vec3_f64    Player.DriverBodyAcceleration    {0,0,0}    0
    r3e_int32       Player.GameSimulationTicks    4801
    r3e_float64     Player.GameSimulationTime    12.014
    r3e_vec3_f64    Player.LocalAcceleration    {0.028581653402957,0.63809360557706,2.4627496799314}    2.5442319757427
    r3e_vec3_f64    Player.Orientation    {0.008504617193035,-1.0053855288309,0.00022639561478311}    1.0054215242132
    r3e_vec3_f64    Player.Position    {106.73530759505,0.66330861158477,-124.18580811176}    163.75280395185
    r3e_vec3_f64    Player.Rotation    {0.0099417858847045,-0.00034270534141877,-0.00077748930665658}    0.0099780280191184
    r3e_vec3_f64    Player.Velocity    {24.971996193958,0.33683698707416,-15.842728100798}    29.575430457438
    r3e_int32       Position    7
    r3e_float32     SectorTimeDeltaLeader.Sector1    -1
    r3e_float32     SectorTimeDeltaLeader.Sector2    -1
    r3e_float32     SectorTimeDeltaLeader.Sector3    -1
    r3e_float32     SectorTimeDeltaLeaderClass.Sector1    -1
    r3e_float32     SectorTimeDeltaLeaderClass.Sector2    -1
    r3e_float32     SectorTimeDeltaLeaderClass.Sector3    -1
    r3e_float32     SectorTimeDeltaSelf.Sector1    -1
    r3e_float32     SectorTimeDeltaSelf.Sector2    -1
    r3e_float32     SectorTimeDeltaSelf.Sector3    -1
    r3e_int32       SessionIteration    1
    r3e_int32       SessionPhase    5
    r3e_float32     SessionTimeRemaining    888.04278564453
    r3e_int32       SessionType    2
    r3e_float32     ThrottlePedal    0
    r3e_float32     TimeDeltaBehind    0.053832292556763
    r3e_float32     TimeDeltaFront    0.059962153434753
    r3e_float32     TirePressure.FrontLeft    179.94482421875
    r3e_float32     TirePressure.FrontRight    179.94476318359
    r3e_float32     TirePressure.RearLeft    179.94654846191
    r3e_float32     TirePressure.RearRight    179.94636535645
    r3e_int32       TireType    -1
    r3e_int32       TireWearActive    0
    
     
    • Like Like x 2
    Last edited: Jul 11, 2015
  4. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    lots of improvements today :)
    • major revision in fileformat
    • improved detection of session begin/end
    • improved lap markers
    • playback is frame- or time based
    • allow state filtering for "dumps" during playback
    • allow "only driving" recordings, that filter away frames in game menu or driven by AI (intended for telemetry tools)
     
  5. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    first version of the viewer is done :)

    double click the laps and use the slider to inspect the recorded values. Laps in brackets mean the time was invalid due to penalties... (as you see I tested that feature a lot)

    next up is something that draws the driven lap and allows to have a heat-map of the track based on whatever you want (increase of wheel temperature, velocity...), as well as a time-based plot. Though the tool will likely stay mostly programmer tool centric, as doing a MOTEC style tool is a bit more work than fun project ;)

    Actually to shortcut the graph plotting I will just add support to dump selected properties as ".csv" files for import in excel or libre/open-office
     
    • Like Like x 4
    Last edited: Jul 25, 2015
  6. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    Export of values via csv file export is now possible :) This allows for some poor man's telemetry in any charting software.

    [​IMG]
     
    • Like Like x 3
  7. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    uhh as you can see we have a track view of the values now, killed my weekend, but well :) (no zoom or pan yet)

    Using the gradient width > 0, you can do interesting things, for example show where you shifted up and down.

    [​IMG]

    next weekend I will try to rewrite the rendering code properly, to allow comparing two different laps on the track.
     
    • Like Like x 6
    • Wonderful Wonderful x 1
    Last edited: Jul 19, 2015
  8. mr_belowski

    mr_belowski Well-Known Member Beta tester

    Joined:
    Apr 23, 2015
    Ratings:
    +1,307 / 0 / -0
    Wow fella, that looks amazing, you've been *properly* busy :D
     
    • Agree Agree x 1
  9. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    thanks, now we "just" need AI to analyze the data so the crew-chief becomes a full racing engineer ;)
    Or longer multi-player races become a coop game, with one person being the crew-chief for real. Would be interesting feature, using spectator mode, to have an eye on the race, and then another monitor for real-time telemetry of the data... and then saying stuff like "compared to your best lap you are braking too early in corner whatever".
     
    • Like Like x 5
    • Winner Winner x 1
    • Wonderful Wonderful x 1
    Last edited: Jul 19, 2015
  10. ::SKRO::

    ::SKRO:: Well-Known Member

    Joined:
    Jan 29, 2015
    Ratings:
    +972 / 0 / -0
    THIS! THIIIIIS! This would be the apocalypse for organized leagues. Having someone who can actually live-monitor your performance remotely would be something epic.
    Thanks @pixeljetstream for the great job!
     
    • Agree Agree x 3
  11. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    As the images above show you can compare laps now

    [​IMG]

    pan & zoom is next up, but after that I will take a break and actually play the game again ;)
     
    • Like Like x 2
    Last edited: Jul 25, 2015
  12. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    Driving lines will use stippling and be printed "above" the pattern is time-based, which means it also gives a hint to velocity.

    Comparing other data will cause it to be printed next to each other (left to right in direction of travel), which you can see here. Up to 4 values can be compared, although the graph gets a bit crowded then. But that allows you to show values of all tires at once.

    Dealing with overlaps is not "ideal", but most of the time I hope the logic of stippling (full overlap) and shifting (partial overlap) should do the trick to see values most of the time. You can also play with the linewidth (last spinner).

    I wish I had the track's "left and right" limitations... hoping @Sector3 will release that data some day. Right now it would mean actually driving along the limitations and generate the data manually, which is a pain.

    [​IMG]
     
    • Like Like x 1
  13. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    Added animation for better driving line perception when things overlap, also visualizes speed well

    [​IMG]
     
    • Like Like x 4
  14. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    Using the "Range" button, you can select to visualize a sub-range of the track. All plots will get cut by the duration the fastest plot took. For example in this image you can see we lost a lot of time in the beginning of the track in lap 4 (lap 4's left side vs. lap 5's right side).

    [​IMG]

    However when starting our analysis after the troubled section we can see that both laps ended up pretty similar in duration.

    [​IMG]
     
    • Like Like x 3
  15. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    With "File->Append" one can load another set of recorded laps to compare with. However, there is no compatibility check whether the same car or track was used. Below we compare the fastest lap from race with lap from qualifying.

    [​IMG]
     
  16. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    Just took a while but finally the one feature I was looking for in all this ;) is finally done, coloring a lap directly by the difference to another lap (which is projected onto it). This will work on any active property, but foremost it's interesting for the driving line / time.

    [​IMG]
     
    • Useful Useful x 2
  17. GooseCreature

    GooseCreature Well-Known Member

    Joined:
    May 30, 2015
    Ratings:
    +670 / 0 / -0
    This my man is truly amazing, I'm no developer of any sorts but this opens up things for some real serious data.

    Loving this!
     
    • Agree Agree x 1
  18. shardshunt

    shardshunt Well-Known Member

    Joined:
    Jan 29, 2015
    Ratings:
    +490 / 0 / -0
    can someone help me I'm completely lost have rst files but i don't know how to do the data visualizations
    1.how do i get the colorful lines on the map i think I've pressed every button and nothing appears other than the label on the top left eg lap 3/2 Time [-0.22,1.05] ect
    2. what can i do the tables. i open it with excel and i just get the numbers eg 0;0.1294850; in each box with the ;?

    sorry im such a noob
     
  19. pixeljetstream

    pixeljetstream Well-Known Member Beta tester

    Joined:
    Jan 29, 2015
    Ratings:
    +412 / 0 / -0
    Sorry, the shaders I wrote required OpenGL4.3. I've just changed them to be OpenGL 3.3 so your hardware can run them. Grab the latest version, then you should see the track.

    As for Excel and csv import and so on, try googling a bit.
     
    • Like Like x 1
  20. shardshunt

    shardshunt Well-Known Member

    Joined:
    Jan 29, 2015
    Ratings:
    +490 / 0 / -0
    thanks so much
    ill test soon

    edit still nothing do i need to install some software or somthing?
     
    Last edited: Aug 15, 2015