How to increase the Speed of a Deep Racer Car in Virtual Race(Time Trial)?

Question

Hello there Deepracers!(#aws-deepracer) I'm hoping you could offer some suggestions or guidance on my reward function. I'm seeking assistance and suggestions regarding my reward function. I've experimented with various approaches to create a reward function that handles the track better. I've developed two different approaches where the reward function performs admirably, allowing the agent to successfully complete the race without any resets or going off-track. However, I've encountered a challenge: the agent's speed is too slow. Despite conducting multiple tests, such as offering rewards relative to motivating the agent to reach its maximum speed, hard-coding speed values, and introducing conditions to monitor and manipulate speed, I haven't seen any improvement in the agent's speed. I'm open to any suggestions or advice you may have for me.  Any tips or solutions you can share with me would be greatly appreciated!

Answer

Please specify if you are in the deepracer league or deepracer student league. It would fundementally change our answer to you. What a lot of people do not grasp is speed is a fixed value from your action space. People will see the video and think they can reward above max speed. The other issue is reward = speed or reward = speed + some other reward behavior are bad reward function.

Without going into the math these reward function will prompt the car to go slower because highest overall reward is achieved. Ultimately the cars goal is to get the highest reward. This means analyzing your reward function is vital to understand if you shaped your reward correctly. Lower steps will be faster laps.

Answer

Hi,

You can get some samples to improve speed here:

1. https://blog.gofynd.com/how-we-broke-into-the-top-1-of-the-aws-deepracer-virtual-circuit-573ba46c275
3. https://medium.com/analytics-vidhya/training-deepracer-for-speed-52007aa03af5
2. https://dominguezdaniel.medium.com/aws-deepracer-parameters-of-reward-functions-4b6f4b7e9eaf
2. https://refactored.ai/microcourse/notebook?path=content%2FDeepRacer%2FAWS_DeepRacer_Reward_function_Additional_material.ipynb
2. https://www.linkedin.com/pulse/samples-reward-functions-aws-deepracer-bahman-javadi/

Best,

Didier

How to increase the Speed of a Deep Racer Car in Virtual Race(Time Trial)?

Relevant content