https://miro.medium.com/max/781/0*pFJLKubRo0Pq0dva

Original Source Here

# Deep Learning for Projectile Trajectory Modeling

*A review of our recent paper on generating simulated data for deep learning-based trajectory modeling*

*This article was authored by Ayaan Haque and Sajiv Shah*

In this article, we will review our recent work titled “Simulated Data Generation Through Algorithmic Force Coefficient Estimation for AI-Based Robotic Projectile Launch Modeling” by Sajiv Shah, Ayaan Haque, and Fei Liu. In this paper, we present FCE-NN, a novel method of modeling robotic launching of non-rigid objects using neural networks which are trained with supplemental simulated data, generated from algorithmic force coefficient estimation. This work has been accepted to ACIRS 2021. The paper is available in ArXiv, and the project website is here. We will first introduce our motivation and problem, then review the method, and finally present some short results and final thoughts.

**Background**

## Motivation

As robots begin to become more advanced, they are required to mimic human-like behaviors and therefore must obtain a human-like skill-set. A large part of this involves interactions with objects, including manipulation and launching. Unfortunately, real-world object modeling is difficult. This is because there are external factors and dynamics that govern the motion of an object which is unaccounted for in elementary physics models. Objects have an increasing amount of stress relaxation over time as well as varying stiffness under a multitude of non-constant conditions such as temperature or humidity. This creates a high amount of uncertainty as each object of a certain model will have varying conditions.

For object launching, aerodynamic forces and mechanism inaccuracies combine to create poor dynamics. To accurately determine lift and drag force coefficients, experiments using expensive tools and sensors such as wind tunnels must be used. Prototype or small-scale models are required to perform these tests, meaning the general shape and material of the object must be predetermined, making designing mechanisms that interact with objects of unknown or changing shape difficult. Without high-speed, accurate feedback from precise sensors and controllers, robots cannot imitate models well. In launching systems, friction from the mechanism and vibrations from actuation due to uneven mass distribution, in addition to unideal control loops results in variance from targeted parameters to actual parameters. The combination of noninclusive modeling and an imperfect mechanical system produces a great variance in experimental and theoretical results.

## Current Solutions

To address these problems, methods from various fields of Artificial Intelligence have been applied in robotics. In Reinforcement Learning (RL), robotics learn novel behaviors through trial and error, which removes the need for a human operator. However, RL has been known to be dangerous and costly in architecture, making it fairly unpopular in general robotics.

Other data-powered deep learning approaches have been investigated for various physics-based modeling tasks, but are reliant on large pools of data. For robotic applications, collecting and producing large datasets would render prediction models obsolete as programmatically referencing the collected data would produce far more accurate results than generating predictions based on that data.

Experimental data has often been paired with simulated/generated data to provide more data for deep learning models. However, strong generative models require access to large datasets in order to accurately learn and reproduce the representations of the data. We propose using a simple physics-based simulator that accounts for the known dynamics of the projectile trajectory to generate data for deep learning. To do so, we develop a set of equations and design the simulator, estimate coefficients using an iterative algorithm, generate data, then tune a neural network to create predictions.

# Methods

Our method is composed of 3 components. The simulator, which includes the coefficient estimator and the data generator, along with the neural network.

**Physics-Based Simulator**

While our method can be applied to a variety of objects, our work deals with a 7-inch foam ball, which is launched from a double-flywheel robotic shooter. Our objective, given by the FIRST Robotics Challenge and shown in the figure above, is to score as many foam, 7-inch diameter balls as possible into the 3 point target from different launching zones, which range from 3 to 30 feet away from the base of the target. The center of the target is ~8 feet high.

The known forces on the ball are gravity, drag, and lift, some of which act upon the ball in different magnitudes and directions depending on the position of the ball in the trajectory. The physics equations and simulator design are outlined in this article:

We assume access to measured launcher configurations X_r and launch outcomes Y_r, as well as simulated data with launcher configurations X_s and labels Y_s. X_r and Y_r together are used to form an experimental dataset D_r. X_s and Y_s are together used to form simulated dataset D_s.

We aim to train our neural network to predict an outcome based on a certain set of launch configurations X_r, each of which has 3 features: distance to the target (meters), motor speed ratio from range 0–1, and launcher angle from horizontal (degrees). More conventionally, X is the input for the neural network, which in this case are the launcher configurations, and Y is the label for the input, and in this case, it is the outcome of the launch. Knowing that the scoring target is a distance (d) away and that the center of the scoring area is at a height (h_target), we assign the outcome as follows.

**Estimating Force Coefficients**

We aim to use this trajectory simulator to estimate the coefficients C_l and C_d by comparing the outcomes of the generated trajectories from configurations X_r to their outcomes Y_r. To estimate the coefficients that best fit the dataset, we simulate the trajectory for each configuration of X_r using multiple coefficient pairs. We evaluate 1000² combinations, with each coefficient being in a range of (0.000, 5.000) inclusive with step-size 0.005.

For a given initial launch angle, launch velocity, and coefficient pair, the simulator is able to generate the object trajectory and determine whether it passed through the target. We can simply iterate through multiple coefficient pairs to determine which one models the trajectories of D_r best. Because the outcome dataset (Y_r) of D_r represents a range of points the object possibly passed through, a large number of coefficient pairs will model the trajectory of a single configuration from X_r.

We calculate the distance of the object from the center of the scoring target using the simulator and label this value as the deviation. To select a single coefficient pair to use for data generation, we evaluate the accuracies in the 3-point and 2-point regions and the mean and median deviations. Our algorithmic force coefficient estimation method is generalizable to other objects interacting with aerodynamic forces. The figure above exhibits the effect of different aerodynamic force coefficients on the object trajectories, as for a given subset of configurations, the outcomes vary based on the assigned coefficient pairs.

**Neural Network Training**

Once we generate additional data using our force coefficient estimator, we couple it with our experimental dataset Dr to train a neural network to predict whether or not the projectile will hit the designated target. The neural network we used in our paper is shown above, and it is a purposely barebones network with 3 layers. The input layer has 3 nodes for each launcher configuration, a hidden layer with 8 nodes, and an output layer with 2 nodes, each for a 2-dimensional vector with binary predictions on whether the 2-point or 3-point targets are hit. This network is a simple forward-feed network and can be replaced with a more advanced network. The use of a neural network allows for the consideration of inaccuracies in our mechanism and model. These inaccuracies include the non-rigid property of our projectile, inability to accurately model energy transfer, and many more. A neural network is specifically meant to account for the inaccuracies of the physics models. The hidden layers allow for the network to model non-linear systems and account for the non-linearities. The collected data is from the domain in which we deploy our AI system. If the data is collected from the target domain, it will account for the variances which the physics models cannot.

Our network is trained on a weighted loss function shown below:

where X_r and Y_r are experimental feature sets and labels, X_s and Y_s are simulated feature and label sets, F_θ is the model, L_R is the true loss term, L_S is the simulated loss term, and λ is the Lagrange multiplier for the simulated loss. Both L_R and L_S use conventional cross-entropy loss and are calculated identically. The λ term is significant in an approach that uses experimental and simulated data. Since D_s is significantly larger than D_r, the loss function must be appropriately tuned in order to optimize the model to perform effectively in the target domain.

**Results**

We use a double-flywheel launching mechanism (CAD shown above) to test the accuracy of our method.

D_r has exactly 100 configurations with an even distribution of data representing our 3 possible outcomes. Only 2 data instances had to be manually rejected due to mechanism flaws. For train and test splits, we used 20% of the dataset as an external test set to evaluate the model performance. This test set contains no simulated data. As stated, the simulated dataset has 900 evenly balanced samples. As an additional evaluation metric, we sampled 50 of the generated X_s and use the configurations on our robot to determine the experimental outcome on our field. We used this to form an additional test set. Acquiring the experimental labels for simulated inputs Xs allows us to not only evaluate the predictions from the simulator based on the estimated coefficients but also the predictions of the neural network.

*Force Coefficient Estimation*

In the table above, we show the performance of the simulator on the training and test set and prove that is it able to very accurately predict the force coefficients to match the training set, which is optimal for the neural network but performs poorly on the test set, which is the motivation to use the neural network. As stated, the simulator is meant to replicate the data distribution of the training set in order to generate new data that is indistinguishable from the training set, which would help the neural network performance the most. Therefore, because the simulator is very effective in accurately generating and modeling the training data, it can be a proper supplement for training a neural network.

The figure above shows a comparison of experimentally measured trajectories and trajectories generated by the simulator using estimated force coefficients (left). We also show the inaccuracy of generated trajectories using estimated force coefficients in comparison to experimental trajectories on the test set (right). This further proves that the generator is accurate at matching the training set distribution but inaccurate at modeling the test set.

*Neural Network Performance*

The table above shows the performance of the neural network at predicting whether the launcher configurations will yield a score on the target. The results show that with any amount of simulated data, the network has stronger performance than when the network is trained on no simulated data. However, after 900 samples, the performance begins to decrease, indicating overfitting to the simulated data which decreases the performance on the experimental test set. The results confirm the importance of using simulated data and show statistically significant increases (p-value ≤ 0.05) in performance. Moreover, the accuracy of the neural network is much higher than just the force coefficient estimation.

**Final Thoughts**

In this paper, we present FCE-NN, a method of modeling the trajectory of non-rigid objects launched by robotic systems using force coefficient estimation and deep neural networks. We introduce an algorithmic process for estimating force coefficients using experimental launcher data and a physics simulator. We importantly use this algorithmic simulator to generate additional data to train a data-starved neural network. We leverage deep neural networks trained simultaneously on simulated and experimental data to predict and model the outcomes of the launcher based on the input configurations, allowing for real-world modeling of non-rigid object launching which can account for external factors on object flight.

We envision such a system to be deployed on a teleoperated mobile robot with automated variable-launchers such that live measured launcher configurations can be fed to the model to predict the outcome of launching. The configurations could be sent to an operator dashboard where the web-deployed network could efficiently return predictions on whether to fire the launcher. It could further be used to provide suggestions on what configurations to adjust in order to make the shot, as opposed to simply returning a binary result. Our future work will investigate new neural network architectures as well as reverse-engineering neural networks to derive force coefficients.

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot