App AI-based Virtual Race Engineer incl. automated Pitstop Handling

Discussion in 'Community Workshop' started by TheBigO, Apr 23, 2021.

  1. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    Hello all,

    with some support from this community, I finally integrated my software package for sim racing rigs, which also include an AI based fully voice dialog capable race engineer chat bot, which can handle a pitstop without any user interaction, with the RaceRoom Racing Experience simulation. Beside the race engineer the software comes with full supprt for controlling all the sim racing aspects with fully configurable and automated handlng of controller devices like button boxes or stream decks, and all the other technical aspects of the ingame immersion.

    The software is free, open source and is fully supported.

    You will find a download on GitHub: https://github.com/SeriousOldMan/Simulator-Controller#simulator-controller (see the README for a download link)

    Full and very extensive documentation can also be found on GitHub: https://github.com/SeriousOldMan/Simulator-Controller/wiki

    If you have questions or found something, that needs a fix or enhancement, please feel free to contact me...

    And now have fun!
     
    • Like Like x 5
    • Winner Winner x 1
    • Love it! Love it! x 1
    Last edited: Apr 23, 2021
  2. ducman888

    ducman888 Well-Known Member

    Joined:
    Jan 7, 2018
    Ratings:
    +171 / 0 / -0
    Holy smokes! I wish game/sim developers put as much effort into their documentation, well done.
     
    • Like Like x 1
    • Funny Funny x 1
  3. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    A question: Does anybody know, how I can create a rear aero damage? This is still an open area in damage repair handling in the Virtual Race Engineer for R3E. I only managed to create front aero damage so far.

    I tried to backup straight into a wall with different cars, but this apparently do not create any rear aero damage. I also tried to brake check AI cars, but they are smart enough to avoid any accident. So, any hints?
     
  4. majuh

    majuh Well-Known Member

    Joined:
    May 1, 2015
    Ratings:
    +219 / 0 / -0
    I guess it depends on the car, try other cars that are more fragile in the back, like the open-wheelers.
     
  5. yoori

    yoori Well-Known Member

    Joined:
    Feb 10, 2017
    Ratings:
    +103 / 0 / -0
    Join rookie / free content servers for lots od rear damage. No need to go past T1.
    But seriously I guess it would be the easiest to get someone help you in an MP session.

    Edit:
    @TheBigO Is there some sort of a video tutorial/demo how to use and set up the software.
    It looks very well documented when you read it. But then it doesn't when you try to run the software.
    I've only just downloaded it and havent tried to run it with a game.

    BTW. It looks like there is a hardcoded path to raceroom executable on your local system.
     
    Last edited: Apr 29, 2021
  6. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    No video yet, but it is on my wish list. And no, no hardcoded path, it can be configured:

    [​IMG]

    The software has a lot of features and a very complex configuration process, since every simulation rig is dfferent, so reading the documentation is necessary, sorry. But it will be worth it.
     
  7. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    With todays release 2.8.6, another cycle in the development of the Simulator Controller Suite is slowly coming to an end. With Assetto Corsa Competizione, rFactor 2, RaceRoom Racing Experience and now also iRacing, 4 simulations are fully supported, including integration for Jona, the virtual race engineer.

    The next cycle is again dedicated to artificial intelligence. If everything goes as planned, in the end we will see the driver fully supported by a virtual racing strategist. Just as Jona supports you as a race engineer with technical questions and automatically determines the settings for an upcoming pitstop, this virtual racing strategist will calculate the ideal time for a pitstop based on the positions and likely strategies of all other drivers. Completion of the automated setup database is also on the plan for the coming cycle.
     
    • Like Like x 1
  8. yoori

    yoori Well-Known Member

    Joined:
    Feb 10, 2017
    Ratings:
    +103 / 0 / -0
    I don't mind reading documentation to get what I want from the software. Although I would expect the software to work out of the box without the need for extensive research and configuration. People don't have time for it.

    I'm very interested in AI powered race engineer for pit stops and strategy, perhaps even driving tips. I work with machine learning, so I'm curious.

    But at the moment the software isn't good enough for me to use. The main reason being the voice is so low volume I can't hear it and there doesn't seem to be a way to change it. I'm also kinda disappointed that it uses windows TTS. (Spoiled by Jim from CrewChief). BTW, did you consider joining forces with CrewChief? That might have amazing results.
     
    • Dislike Dislike x 1
    • Agree Agree x 1
  9. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    Yes, the volume of Windows TTS is somewhat buggy, but you can adjust the level of the different sources with the volume mixer, meaning make the sound of the simulation somewhat more silent and you are fine.

    Regarding your other point: Hmm, it depends. You can use Word out of the box to write a simple letter, but formatting a complete book is something really different. Anyway, I am with you, I also like intuitive software solutions.
     
  10. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    With version 3.0 - as announced - the next development cycle begins, the further expansion of support for the driver. For some time now, the virtual race engineer has been collecting data from the various sessions and, of course only with consent, transfers it to a central server for consolidation. Release 3.0 now brings a first tool with which this setup information (tire selection and air pressure) can be queried depending on the vehicle, route and weather conditions. An extrapolation method helps to determine setup instructions for conditions that have not yet been "experienced".

    Race Engineer Setups 1.JPG

    Another note at this point: You always have access to your own data, you only get access to the (tire) setup information of the community if you also contribute to the development of the database. Therefore it may be time to reconsider your own consent.
     
    • Like Like x 1
  11. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    This week's release brings many changes "under the hood" to prepare for further steps in the development of the setup database and the virtual race engineer. The most important visible change concerns the virtual race engineer, who now also asks the setup database when the tire pressures are calculated for an upcoming pit stop. This is particularly advantageous in the case of drastic changes in the environmental conditions, because then the dynamic calculations based on the data from previous laps reach their limits. When querying the setup database, extra and interpolation methods are used and a factor is used to consider how resilient the derived tire pressures are if no exact data is available. The query of the setup database can of course be configured.
     
    • Like Like x 1
  12. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    This week's release brings more features to the setup database. In addition to information on the correct cold air pressures and the tire compound depending on the track and weather conditions, which have been collected for a long time already, the setup for mechanical and aerodynamic settings can now be uploaded to the setup database. If consent is given, this information will also be shared with the community. Consents can now be given separately for air pressure and setup information. For this reason, consent will be requested again with this release. In addition to the information mentioned, private notes can also be stored in the setup database, which are never shared with the community.
    Race Engineer Setups 2.JPG
     
    • Like Like x 1
  13. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    This week, too, there are further improvements to the setup database and the associated integration with the virtual race engineer. The settings for a current session can now be automatically saved in the setup database and automatically loaded from there at the beginning of a new session. Another page has been added to the general configuration tool, with which the dynamic decisions and statistical processes of the virtual race engineer can be influenced or configured.

    Unbenannt.JPG
     
    • Like Like x 2
    Last edited: May 28, 2021
  14. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    Release 3.1 took two weeks to mature. But a lot has happened, even if much of it is not yet fully visible. All changes serve to integrate a new virtual assistant into the platform. In addition to Jona, the virtual race engineer, the driver is now also supported by Cato, a future expert in racing strategy. Cato can currently hardly do anything, but when fully expanded he will have a complete overview of all drivers, their lap times and the development of the positions, even with different pit strategies. Cato will furthermore be able to develop appropriate pit stop strategies if you are stuck in traffic and he will be able to react to unforeseen events such as sudden weather changes and severe damage - all hand in hand with Jona, the race engineer. All of this will come step by step over the next few weeks / months.

    Since two different assistants are now available as conversation partners in Voice Chat, the speech recognition has been fundamentally revised. It now allows several simultaneous dialogues and can switch between the conversation partners with an activation phrase. In addition, many internal changes were made to make further development more flexible and the rule system for the AI kernel was modularized. As always, details can be found in the release notes.
     
  15. Maskerader

    Maskerader Well-Known Member

    Joined:
    Oct 6, 2019
    Ratings:
    +205 / 0 / -0
    This sounds awesome!
     
  16. yoori

    yoori Well-Known Member

    Joined:
    Feb 10, 2017
    Ratings:
    +103 / 0 / -0
    Maybe it's time to try it again. Have you tried to do something with the volume?
     
  17. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    I looked into that, but it looks like it's low level behaviour of the Windows speech recognition system. Surprisingly, I do not see this in all cases. If I use an external webcam microphone, everything is perfect.

    The only workaround is using the sound mixer to balance the volumes.
     
  18. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    Finally, I reproduced the same behaviour on my development notebook. As I said, the volume is reduced by Windows, if speech recognition becomes active. BUT, if you enable "push to talk" in the configuration, this will only be as long, as you hold the button to talk..
     
    Last edited: Jun 14, 2021
  19. Christian G

    Christian G Topological Agitator Beta tester

    Joined:
    Apr 8, 2015
    Ratings:
    +2,411 / 0 / -0
    Not quite sure if it's this setting causing your issues but I guess it's simple enough to give it a try:

    Go to Windows' control panel (the old one) and select "Sound".
    This menu should pop up:
    upload_2021-6-14_9-10-48.png
    Now go to the last tab called "Communication". Here you can define how Windows reacts to any communication it detects.
    upload_2021-6-14_9-12-6.png
    But this behaviour might also be caused by other programs that are running alongside the game, like Skype, Discord or Cortana which all have built-in settings similar to the one I showed above.

    Hope this helps.


    Hi BigO, as you said you can reproduce the issue, maybe you can verify the method above and if it's helpful, add it to your documentation/guide.
     
    Last edited: Jun 14, 2021
  20. TheBigO

    TheBigO Well-Known Member

    Joined:
    Mar 29, 2021
    Ratings:
    +45 / 0 / -0
    Hi, i tried it and was not the root cause, at least on my laptop. But I found a different option, that seems to be effective:

    Unbenannt.JPG

    If you uncheck the above option in the property dialog of "Microphone", the sound level reduction do not happen. But you might have recognition problems, though. Just give it a try.
     
    • Like Like x 1