The new adaptive AI

Discussion in 'General Discussion' started by Paradox Agi, Apr 18, 2016.

  1. CheerfullyInsane

    CheerfullyInsane Well-Known Member

    Joined:
    Oct 24, 2015
    Ratings:
    +344 / 0 / -0
    Apologies for the late reply, but work and a complete melt-down of my PC resulted in a somewhat harried week. :D
    Anyway.....
    The above is actually what the AI does when it adapts, only difference is that it starts at 80.
    (Or at least it did. Haven't had time to pick the new one apart yet)
    In my case, it usually jumps from 80 to 90-92 in the first race, 97-98 after the second and so on.
    The problem though is two-fold;
    First it starts in the wrong place. If the AI can go from 80 to 120 it should start at 100 to minimize the distance it has to travel.
    Though this only applies if your level is actually above 100, and let's not forget that a lot of less experienced racers could very well be discouraged if it took them 10 races to get something that even resembled a race.
    So I can live with that one (even if I still find it annoying :D )
    The second problem comes after it has actually found your level.
    It then has to truly adapt in order to get good racing, otherwise it doesn't work.
    To illustrate, let's say that my level for a particular combo is 108.
    After six races the index-file runs 80-92-98-102-105-108 (completely fictitious numbers, just play along)
    Now, if you're thinking that the AI has now adapted to you, you'd be wrong unless you happen to be obscenely consistent even when there is traffic around.
    If you try to race that index with a full grid you will at some point have to slow down to e.g. level 107 because another car is blocking a corner. And since the adaptive AI is made to consistently follow your lap-times it also has to drop down, but the next available level for it is only 105. Which means that for the next lap you'll be blitzing through the field.
    And then the AI will speed up again, repeat ad nauseum.
    And if you happen to beat you own level, the AI will then have to insert yet another level into the index.
    The AI will eventually get there on its own, it just takes a wee while.
    Make no mistake, the AI doesn't just use one entry in the index, it uses all of them if it has to. If you're reasonably consistent, it normally only needs two or three, but if there are large gaps in the index, you're going to get some decidedly weird races.

    And to muddy the waters even further, the starting point of 80 only applies if the index-file is empty.
    Once there is data in it, any new combo will start at what I suspect is the average of the entire index-file.
    Or in other words, if you've been running NSUs on all the tracks at an average of 105, if you switch over to say GTR3s it'll start at 105 as well, despite you not having any data for that particular combo.

    And of course.......All of the above are null and void since my training method doesn't work with the new AI. :D
    As I said earlier, I haven't had time to completely pick it apart yet, but from the experimentation that I've managed to do you can no longer just quit out after one race and get an AI entry.
    I think (read: wild-ass guess) that it's because they've corrected what was actually a bug. I doubt the AI was ever supposed to take the first lap into account, since that would mean a lot of short races would mean a lot of non-flying laps in there dragging the average down.
    So the good news is that they've fixed that. The bad news is that it'll take 2 laps on manual AI level, and probably 3 laps on adaptive to get it to write anything.
    Now, this is all very preliminary and assuming I can find the time, I'll sit down with the AI over the week-end and see if I can't figure out a new way to 'cheat' us into better adapting AI.
    But right now I'd say the fastest way would be to run a 2-lap race with fixed AI at your approximate level for that combo, and then run 3-lap races with adaptive until it has...well, adapted. :)
    I also suspect that since most people have a level that is more based on the car than the track (i.e. people usually run at one level with NSUs and another with GTR3s no matter what the track is), you're probably better off training one class at a time and then do some manual cut'n'paste or use @pixeljetstream s program to insert the data into a fresh index-file.
    If you're 108 with NSUs and 95 with GTR3s, and you do the NSUs first, the GTR3s will then have to adapt from 108.
    On the other hand, if you're capable of doing 108 with the NSUs fairly consistently on all tracks, you can get the data inserted, copy the file, and start over with a GTR3 one starting at 95 and repeat as needed.
    Hopefully I'll have something a little clearer after the week-end.
     
    • Winner Winner x 1
    • Informative Informative x 1
  2. Christian Göpfert

    Christian Göpfert Topological Agitator Beta tester

    Joined:
    Apr 8, 2015
    Ratings:
    +2,279 / 0 / -0
    Talked to Robert about that and iirc he said that it depends on the starting method you use. If you use standing the first lap will be ignored, if you use rolling it will be counted in.
    Maybe this helps sharpen the image a little.
     
    • Love it! Love it! x 1
  3. CheerfullyInsane

    CheerfullyInsane Well-Known Member

    Joined:
    Oct 24, 2015
    Ratings:
    +344 / 0 / -0
    *D'OH* Of course.....
    So I'm being stupid again. :D
    Thanks for that little tid-bit, that should severely cut down on the number of laps necessary. :)
     
    • Like Like x 1
  4. Christian Göpfert

    Christian Göpfert Topological Agitator Beta tester

    Joined:
    Apr 8, 2015
    Ratings:
    +2,279 / 0 / -0
    ^ Like does not apply to the being stupid bit. ;)
    Thanks for going through with all this, you're doing a great service to the community here, at least those who are into SP racing.
     
  5. CheerfullyInsane

    CheerfullyInsane Well-Known Member

    Joined:
    Oct 24, 2015
    Ratings:
    +344 / 0 / -0
    No worries.
    I've got all the coding skills of a demented rodent, and nobody in their right mind would look to me for racing advice. :D
    But this I can do. And as it happens, I enjoy doing it. :)

    But can you possibly get Robert to clarify something for me?
    As it is right now (or rather, was) the lap-times in the index-file are only used after the 'training' has been done.
    If I've deduced this correctly, the adaptive AI compares its last lap-time to the players lap-time, and then goes hunting through the index for an AI level that is roughly equal to the players lap-time and uses that level for the following lap.
    Pretty sure I've got that right, and that is all fine and groovy. But........

    If you manually edit the index-file, and only insert the AI level, it doesn't work.
    If you manually insert an AI level and a fictitious lap-time, it doesn't work (unless you're extremely lucky and manage to hit the right lap-time).
    So apparently the Adaptive AI needs an AI level AND a specific lap-time in order to use the entry.
    And yet, when using Fixed AI you can set it to a specific level, and it'll generate a lap-time out of the blue.
    It's even kind enough to write it for you in the index.
    So in other words, the AI itself must have some idea of the lap-time it's capable of doing, otherwise it wouldn't be able to differentiate between the various AI levels in the first place.
    So why does the Adaptive AI need a lap-time when the Fixed one doesn't?
    And is there some way to bypass that?
    If we could somehow get to the point where all you needed was to insert the desired AI levels without the lap-times, this whole training thing would be moot.
    All you'd have to do would be to edit the file before using each combo, and it would be fully trained.
     
    • Like Like x 1
  6. Christian Göpfert

    Christian Göpfert Topological Agitator Beta tester

    Joined:
    Apr 8, 2015
    Ratings:
    +2,279 / 0 / -0
    I'll see if can get a hold of him but it might take a few days.
     
  7. Robert Holm

    Robert Holm Sector3 Developer

    Joined:
    Jan 15, 2015
    Ratings:
    +160 / 0 / -0
    Not sure I understand properly :)
    An AI needs to know player's average laptimes; those I guess you could put in manually.
    It also needs to know how fast itself can go around the track at different skill levels in certain car class on certain track layout,
    so it can reverse engineere a skill level from player's time. If you knew what those times are, you could put them in, but how could you :) I guess you could try, remember to also put in amount of samples in paranthesis.
     
    • Like Like x 2
  8. CheerfullyInsane

    CheerfullyInsane Well-Known Member

    Joined:
    Oct 24, 2015
    Ratings:
    +344 / 0 / -0
    Maybe not. :D

    I get that, makes perfect sense.
    Except......
    That doesn't seem to apply when using a Fixed AI.
    If you use e.g. 105 for a particular combo, it'll run at 105 and it'll continue to do lap-times that are very close to each other.
    So the Fixed AI apparently doesn't need to know how fast it is with any given combo, it'll just go.
    Which, as far as I can figure out, either means the AI level controls something more than just the lap-times themselves, or that the Adaptive AI in theory should be capable of doing the same.
    So I guess my question is, since we already know the Fixed AI can run without having prior knowledge of its lap-times, why doesn't this apply to the Adaptive?
    Or in other words, why can't we simply plug in a range of six AI levels into the index-file, and the AI will fill out the lap-times as and when it gets to them?
    I realize that this would leave the AI without a clue as to which level to use for each lap, but that is no different than starting with a new index-file containing no data.
    I'm sure I'm missing something somewhere, but I'm having a little trouble wrapping my mind around what I perceive to be two different behaviours.

    Final thing, and I'll let you get some peace. :)
    The Adaptive AI starts out at level 80 when using a fresh index-file, but once the index-file starts to fill up it'll start somewhere else even if it has no data for that particular combo.
    Can you confirm that if the index-file contains data on other combos, the Adaptive AI will start any new combo at a level based on the average level throughout the index-file?

    And of course, thanks for taking the time to respond. :)
     
    • Like Like x 1
  9. Robert Holm

    Robert Holm Sector3 Developer

    Joined:
    Jan 15, 2015
    Ratings:
    +160 / 0 / -0
    Increasing skill level from 95 to 100 changes a whole lotta stuff about how fast they can go in corners etc, and the laptime is the result. And their laptime is also the only thing you can compare to player's "skill level", to try to match them up. Marko tries to match them to real life laptimes at a certain level. If you chose 95 in game instead of adaptive, it's not like we're gonna match to a specific laptime, only tweaks some parameters that make ai faster/slower.

    Secondly, yes, if you dont have any data for that specific layout/class combo, it'll take an average. First check if you have data for carclass, if not if you have for track layout, if not it takes average of all. Probably closer to truth than setting it to 80 :)
     
    • Like Like x 2
  10. CheerfullyInsane

    CheerfullyInsane Well-Known Member

    Joined:
    Oct 24, 2015
    Ratings:
    +344 / 0 / -0
    Aaahh...The penny drops.
    Okay, so a Fixed AI level will produce a certain behaviour which will result in a certain lap-time.
    The Adaptive AI, on the other hand, looks to reproduce a lap-time close to the player, and then reverse-engineer the process to find the AI level needed to produce the AI inputs that will result in that lap-time.
    But of course it can't do that unless it has something to base that on.
    Right, right, right.....
    Solving the problem from the other side means you need a different set of data to base the solution on.
    Okay, got it. :)

    Wait, what?
    Amazing how eight words can make ones life a whole lot easier. :D
    That is absolutely brilliant, thanks for that bit of info. :)
     
    • Like Like x 2
  11. Supermini

    Supermini Member

    Joined:
    Mar 31, 2016
    Ratings:
    +21 / 0 / -0
    Well poop. But that was enlightening nonetheless, thanks for the very informative explanations!
     
  12. CheerfullyInsane

    CheerfullyInsane Well-Known Member

    Joined:
    Oct 24, 2015
    Ratings:
    +344 / 0 / -0
    Now, now.
    Don't give up so soon, grasshopper. :D
    It may still work, it just needs some method-modification.
    But the last time I did this, the thread grew.....somewhat largish.... because I kept finding new stuff after I'd posted. And I'd really like to avoid that happening again, so this time I'm going to make reasonably sure that I'm actually right before posting. :rolleyes:

    So I'm 90% sure I've got it figured out, and since I need to train the KTMs anyway, I'm going to do some testing and see if I can't get to 100%. Add some time to get either a video or some illustrating done, and I wouldn't expect anything until probably Monday assuming something doesn't crop up in real-life.
     
    • Like Like x 2
    Last edited: Nov 26, 2016
  13. Andy Jackson

    Andy Jackson Member

    Joined:
    Nov 19, 2016
    Ratings:
    +14 / 0 / -0
    I like this.:)
     
  14. CheerfullyInsane

    CheerfullyInsane Well-Known Member

    Joined:
    Oct 24, 2015
    Ratings:
    +344 / 0 / -0
    Okay, so the first draft is ready for ridicule. :)
    I had originally intended to do a video, but most of the info is theoretical and the explaining involves text-files and spread-sheet, which would make for a somewhat sleep-inducing video.
    A forum post wouldn't allow the formatting needed to insert images at the right places.
    So instead I chose to whip up a small PDF explaining both the theory behind it, and how it applies in practical terms.
    It's actually not that complicated once you grasp the basis of the system, I just happen to think that knowing why you need to do something is far better than just knowing you have to do it.


    Now all we have to do is wait for Robert to come along and tell me I've misunderstood the whole thing. :D
    .
    .
    Told ya.... :D
     
    • Like Like x 2
    • Informative Informative x 1
    Last edited: Nov 28, 2016
  15. Christian Göpfert

    Christian Göpfert Topological Agitator Beta tester

    Joined:
    Apr 8, 2015
    Ratings:
    +2,279 / 0 / -0
    Very well done, kodus.
    Recommended to move this to the support section.
     
    Last edited: Nov 28, 2016
  16. Fairman

    Fairman Well-Known Member

    Joined:
    Feb 5, 2015
    Ratings:
    +870 / 0 / -0
    Generally in adaptive mode the AI is much to slow when not adapted to average times and much too fast when successfully adapted, ehm, (like before)...anyway, I don't care much cause best way is still to adjust the correct diff level that meets your skills...
    But what I wanted to state is that I like the AI very much now, they behave like MP Opponents and I had some really cool fights with 'em (GTO @ Midohio, Level 102)....mostly tough but fair (means no strange actions)...dunno how it feels if you're SP-only driver with less experience,but coming from a lot of MP (league) races throughout last year, the AI is definitely worth to use them for sparring again...;)

    Good job so far...
     
  17. Robert Holm

    Robert Holm Sector3 Developer

    Joined:
    Jan 15, 2015
    Ratings:
    +160 / 0 / -0
    @CheerfullyInsane there are a few wrong assumptions in that pdf, some I feel I have to correct:
    - ai doesn't adapt per lap, it's per race. otherwise it'd be a bit like rubberbanding
    - what you call sample rate is just the amount of samples that's been used for the stated average laptime, to know how much next added laptime will affect average

    Fastest way to get ai up to par is to download a ready file, which I assume is what you supply? Then all needed is player laptimes which you ofc have to set yourself.
     
    • Like Like x 2
  18. Robert Holm

    Robert Holm Sector3 Developer

    Joined:
    Jan 15, 2015
    Ratings:
    +160 / 0 / -0
    Think it's been said plenty of times before, but no it doesn't
     
    • Informative Informative x 1
  19. CheerfullyInsane

    CheerfullyInsane Well-Known Member

    Joined:
    Oct 24, 2015
    Ratings:
    +344 / 0 / -0
    Really?
    Uhm.....Okay, normally this is the point where I ask "Are you sure?", but since you designed the system, I'm going to skip that part. :D
    Could've sworn I saw AI behaviour that indicated it did, but I guess that means there's a flaw in my testing method.
    Fair enough, not the first time I've been wrong, and most certainly not the last.
    Good news is that this means you actually need fewer indices than I originally thought.

    So basically it smooths out the curve.
    Meaning that if the AI e.g. gets held back in a few corners that'll result in a slower lap-time. But if the average so far is already based on say 10 laps, the impact is lessened.
    So one bad lap will not lower the stated lap-time in the index-file, and thus won't necessitate a change in AI levels for the next race, which it might otherwise do.
    Okay, got it.

    A complete index-file listing every lap-time for every AI level for every combo would indeed be the fastest way.
    But not only would that result in a fairly hefty file-size, it would also take forever to get done.
    As I've found out, you can't just insert data into the file unless you have a general idea of the lap-times the AI will generate.
    @pixeljetstream designed a program that'll extrapolate probable lap-times from a data-set, and insert them into the index-file.
    But that relies on having a certain amount of data in the first place, which isn't always the case.
    And inserting fictitious data into the file results in.....odd behaviour. :)
    So what I'm trying to do is find the fastest way for the user to get the AI up to par for any given combo, no matter what the users level is and no matter if there is previous data available.

    Don't get me wrong, the adaptation process does work, and once you have data for a specific car-class from 2-3 different tracks, it adapts pretty quickly on its own. But it'll still take 4-5 races to get a good base-line, and a few more races until the data is solid enough for the AI not to shift levels from race to race.
    So my thinking was to exploit the algorithms already in place, and 'cheat' the AI into thinking it had already done hundreds of laps and it could now stop adapting.
    And now that my initial idea has been shot down in flames, and lies whimpering on the ground, it's time to rethink. :D

    So.....
    Apologies if I'm restating the obvious, but I'd really like to get this right one of these days.
    When you start a race against Adaptive AI, and there is no data at all in the file, the AI starts at level 80.
    Else, If there is data for the car-class the AI will start at the average level of the car-class in the file.
    If not, if there is data for that track, the AI will start at the average level of that track.
    Finally, if there is no data for neither the track nor the car-class, but there is other data in the file, the AI will start at the average level of all the data available.
    After the first race of a combo, the AI will then write an AI level and corresponding lap-time, and all further races will use that first entry as a baseline.

    Once you have data for a particular combo, the AI will take the average of your last 10 laps and compare it to the average lap-times it has for each AI level.
    If there is no AI time that corresponds to the players average time, the AI will then try a new level, and subsequently writes the data for that level into the file.
    How much it jumps depends on how big the difference between the times.
    If the AI has a time that corresponds to the players average, it'll re-use that AI level, and the lap-time will be added to the average for that level, ensuring you get consistency in the AI levels.
    Have I (finally) gotten that right?
     
    • Agree Agree x 1
  20. CheerfullyInsane

    CheerfullyInsane Well-Known Member

    Joined:
    Oct 24, 2015
    Ratings:
    +344 / 0 / -0
    Actually, never mind. I figured it out.
    Oh my god, have I been over-complicating things. :D
     
    • Funny Funny x 1
    • Winner Winner x 1