Original Source Here
Giving a Genetic Algorithm Control in a Physics Engine
A brief introduction to applications of AI in physics engines
The most common/commercial application of AI is to gain insights or make predictions on a dataset. Even though I think that this is interesting, these techniques have been widely covered by others.
This article covers a more experimental application of machine learning: this is allowing machine learning to manipulate a physics simulation.
There is a lot of different possibilities of putting an AI into a physics environment, but all of them involve manipulating and controlling objects and forces in the environment. To get good results from this project, we need to add limitations to the AI’s control in the environment.
The limitations that I decided to put are as follows: The AI must generate an n-sided polygon. Each point of the polygon has a limb, which rotates at a fixed rate. This rate of rotation is defined by the AI.
The genetic algorithm requires a fitness function to improve the agents over time. In this case, the agents will be judged on two factors: speed and area. The speed in this case is defined as the x-distance traveled in 100 frames. The area is the space enclosed by the points of the polygon. The area is taken into account, to prevent the more complex coding needed to add the effect of air resistance on the creature.
We want to maximize the speed and minimize the area. We can conclude that the fitness of the agent can be the speed/area. This would make it so that the fitness would be proportional to the speed and inversely proportional to the area. Hopefully, we should end up with “creatures” that move quickly, while being small. This means that a similar creature should hold up in reality, which gives the project more practical value.
I will only be adding interesting sections of the code because the rest of the code does not need any explanation. If you want the full code of the project, it can be found here.
Physics Simulation Basics:
The physics engine that I used for this project is based on the pymunk library, which contains all the parts that make this project work.
This script below is an example of the ragdolls that can be generated by the model.
The list “vs” is the list of points that form the main body of the ragdoll. The “limbs” list is a list that defines the length of the vectors of each of the limbs. The loop constructs the body by adding the limbs to each point of the body. Here is a gif of the simulation:
This is the fitness function that controls which agents are chosen to be retained. This fitness function runs the physics simulation for every agent.
Each agent has three separate neural networks, the “motor_net”, the “body_net” as well as the “arm_net”. The motor net generates the motor speed for each of the limbs, the body net defines the coordinate of each of the points on the body and the arm net defines the length of each of the limbs. Both of these neural networks take random values as inputs.
The models have a final hyperbolic tangent value (range of -1 to 1). This is then multiplied by a constant value, that acts as the range of the network. This is so that the model can create versatile results, while preventing very big or very small bodies to be made. This allows for some consistency for better comparison.
The definition of the body is in lines 26–29: for every point on the body, an arm is added, with a connection of type “Pivot Joint”. The pivot joint is just a joint that pivots around a single point. A “Simple Motor” simply means that the rate of rotation is consistent.
The body and the motors are set up before the agent is dropped into a large box. For a defined number of steps, the agent moves. The distance travelled by the agent can be defined by the x dimensional change in the position of its body.
I will cover the genetic algorithm separately, as it is more complex and I have used it a lot in recent projects.
Here are some of the more creatures created by the algorithm:
This creature has a thin body but the arms are short and does cause too much net movement, resulting in a lower fitness.
The thin body and different directions/speed of rotation made this creature get a higher score than others.
Symmetrical motor movement causes not much movement, and therefore a low fitness value.
This project took a long time to cobble together, because I tried many different means to use a physics environment. Luckily, I found pymunk before I gave up on the project.
A massive limitation of this project is the lack of joints/motors that can move by degrees instead of rotation. I am planning to create/find a feature that can have a changed rotation angle/direction for each step, allowing for the implementation of more complex models like LSTMs.
I still think that this project was interesting enough for me to continue looking into more physics-based optimization problems.
Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot