App Adaptive AI primer/database tool

I mostly race against AI, so i should have some usefull data, but for some car/track combo i may have very bad times because i was just getting settled with them or was letting some friend play. Tell me if my file can be usefull anyways, i can upload it this evening

 

Well my file is 467kb at the moment and growing..

 

@hotak, your time won't matter mostly interested in the AI time

 

updated html once more, and generated some statistics
http://htmlpreview.github.io/?https...aptive-ai-primer/blob/master/docs/sample.html

We now have reached almost 1% of all possible combinations (40 ai levels * 65 tracks * 28 carclasses)
Total (track, car, ai) Entries: 746 (0.97%) Times: 786
Only have very little entries with more than one time, just shows how much variety in the 12 people who contributed so far
In terms of having at least one value per track & car we are at 16%
Track (track, car) Entries: 316(16.36%)

 

I think we may get good values for laptimes for 80 and 120 % AI level and just linearly interpolate between, maybe have 100% as another value. But when I look at some races the times scale quite linearly.
The more dense data we have, the more outliers/noisy our data gets.

 

Here's mine. Hope it doesn't make the data too noisy

 

Here's mine. I hope it is of some use.

 

here's mine

 

thanks for those contributing, we got 20% track/car coverage, although not all will be useful. We need at least two distinct ai - levels per track to have better data for the autogenerated values.

worked on that portion today, and the attached html report gives AI times per track/car that passed some "plausibility" criteria that I made up. Right now only a quarter of the content we have passes that criteria, maybe it's too harsh now we will see.

if you are brave you could use the attached table and make edits to your aiadaptation file as @CheerfullyInsane described. Otherwise wait until I have that part of the tool done which lets you modify the file via some UI (though warning this may be couple weeks away).

edit removed file, didn't check plausibility as well

 

Here's mine, I used to race in single player only but moved to MP only now. Anyway hope it helps.

 

Sure, I'll give it a whack or two during the week.
Btw, I think I may know why there are such large differences in the AI lap-times.
If you 'train' it like me using only one AI, the driver selected for each race differs.
And seeing as it is always the same drivers in the top ten during 'real' SP races, I suspect there might be something like the GTR2 talent files hidden away somewhere.
Not that it matters all that much. Since we have a span of six AI levels, we are mainly looking for a ball-park figure.

 

First impressions:
This actually works rather beautifully. Exceedingly well done @pixeljetstream
There is however a bug in the algorithm you use to generate lap-times.
Take a look at Aquilas@Raceway.
Starting from 120, it the laptimes slowly increase until around level 108.....
Then they start to decrease again.
Level 80 is in fact 25 seconds faster than level 120.
What I suspect is happening is that you've taught the algorithm to decrease the lap-times by X amount from a baseline, which is great as long as you want it to go faster.
But it should be reversed when going the other way.
Speaking of which, is there a way you can highlight (e.g. through text-colour) which times are player-generated, and which are algorithm generated?
In order to test the algorithm, it kinda helps if you know which lap-times you're comparing against.

First I tried to see if it works (which it does), then I tried to break it.

Couple of things to note.
If you haven't been following any of the other threads, one major thing to remember:
R3E cannot be running while you edit the XML file!
The XML file gets loaded into memory when R3E starts up, and after it shuts down, it dumps the memory into a new XML file that overwrites the old one.
So if you edit it while R3E is running, not only will the game not load any of the settings you've typed in, it'll also overwrite them afterwards, making the whole thing a colossal waste of time.

Next, in order to get full enjoyment out of the adaptive AI, you have to be fairly consistent.
The shorter the races you run, the more consistent you have to be.
The idea here is that if you spin in a 30 minute race, you have maybe 14 other laps to pull the average up.
If however you spin in an 8m race, you only have 3-4 laps to make up the difference.
One example was when I tried the generated lap-times in NSUs@Zandvoort. I plugged in the lap-times and got an index running:
110-111-112-113-114-115
Then I did an 8 minute race, and spun on one lap. After that the index ran:
107-111-112-113-114-115
So the AI threw in a 107 to deal with my slower lap.
Now, it's not the end of the world since the lap-times are weighted.
R3E takes note of how often each AI level is used in each race, and adds a value to them. So just because I had one spin, doesn't mean the AI will start at 107 in my next race.
However, if your lap-times are all over the place, the AI can't find a baseline to adjust to.

Next, I tried to see if you needed your own lap-times in there as well in order for it to work.
I simply plugged in 5 identical lap-times at the level i wanted and tried racing.
Then I tried it again with only the AI times.
Either works fine. As I kinda figured from my earlier testing, your own lap-times only comes into play AFTER the AI is trained.

Last thing I tried was to see if the number of decimals matter.
The XML file has 8 decimals, and the table supplies us with 4.
As it turns out, the AI doesn't care either way. You can plug in 4 decimal lap-times, and that'll work just fine.

EDIT: Updated version of my XML.
Lot more tracks for both the FRJ and Aquilas. (I seem to be the only one running them )

 

Sorry i just saw this Lars, if it is not too late i can contribute because i do a fair amount of AI and also because you guys are going to struggle populating data at levels lower that 120... I can give lots of stuff at 100, 105...

Will send my file later this week...

 

Great.
Everyone are volunteers, so no pressure.
Just as and when you get to it.

 

Thanks @CheerfullyInsane indeed there were some bugs. The tool will mostly never use values directly reported by users but will fit a curve to the data. Reason is that the data is kinda noisy anyway... I may change the fit logic over time, but the good news is that we can always change it.

I actually use the fact that my numbers will generate 0005 or say 000? or last 4 digits missing to detect values the tool generated, as it's unlikely real data has those properties. To avoid the problem of enriching database by values we generated that @mr_belowski mentioned before.

The tool is ready and should work for the brave once more. Just command-line so far (sorry no time for UI yet).

But in theory it works.

• Run the exe file (it comes with the database of times we gathered so far)
• Look at the results/processed.html file, find the ai levels you want to race against
• edit the "edits.lua" file (see below)
• BACKUP your old aiadaptation.xml !!
• Drag drop the edits.lua file onto exe or use commandline to pass it (preferred as you can read error messages)

Here is what the edits.lua file looks like, ideally self explanatory.

You can see the "safety" is on, as the target points to a file that won't exist for you, so you have to switch the if (true) to if (false) as well.

The sample does just one edit operation, but you could throw in multiple of course.

Code:

-- for now no UI
-- change what you want to edit here
-- lookup classid, trackid and your preferred ai level via results/processed.html

-- modifyAdaptive (filename, database to use, trackid, classid,
--                 ai start, ai end, ai spacing (1: e.g 90,91,92,93... 2: 90,92,94...))

print "executing edits"

local target

if (true) then
  -- absolute path
  target = [[D:\projects\r3e-adaptive-ai-primer\aiadaptation.xml]]
else
  -- danger zone
  -- make backups!
  -- this will look at the default location of the aiapdatiation.xml
  target = specialFilename([[$USER_DOCUMENTS$\My Games\SimBin\RaceRoom Racing Experience\UserData\Player1\aiadaptation.xml]])
end

-- this changes "RaceRoom Raceway - Classic Sprint" (265) for "Aquila CR1 Cup" (255)
-- it will generate ai for 97,99,101,103
-- use tighter spacing 1 instead of 2, if you are more consistent
modifyAdaptive(target, processed, "265", "255",  97, 103, 2)

 

Fair point.

Will do.
Going to have to wait until tomorrow though.
It would presumably be a good idea to be at least semi-awake for this.

 

ui version is done

 

Very impressive and advanced already, thanks @pixeljetstream @CheerfullyInsane; will give a try today; in the meantime and in case you are still collecting udata here is my current file. Thanks!

 

Woah, that's a bit awesome

Are the data points you've collected going to invalidated if / when the ai gets a kick up the arse? I believe Robert is working on their performance at the moment, which might make the current time->ai level data inaccurate

 

@Riverside yes keep it coming we have tons of gaps.

@mr_belowski yes good point. We don't have a "versioning" of sorts, although maybe I should add file-dates where the timestamps came from. R3E itself probably wipes the adaptation file clean whenever bigger changes occurred (or at least it should).

what I hope is that some day we get a dedicated server with ai support and then we could just brute-force fill the database with tons of AI values running a server in "accelerated simulation time". I'd hope the developers have such a mode, cause then you can tweak AI much easier by simulating long races in a minute or so depending on how fast the PC is.