Bio-Inspired Optical flow

https://miro.medium.com/max/1200/0*D13dedV633DktPvG

Original Source Here

Bio-Inspired Optical flow

Take beetles as an example. Although having simpler nerve system they have an incredible ability to fly and response to spontaneous events. They are a perfect example of understating visual motion.

Photo by Filipe Resmini on Unsplash

The goal of visual motion estimation is to infer the motion of objects in a scene based on a sequence of images. It is closely related to Optical flow. Optical flow is made of vector field. The flow fields can be analyzed to infer the regions of containing motion. Of course, visual motions algorithms as they are inherently differential, are computationally expensive. The added complex is also from noise perceived from sensory inputs. This is in addition of real-time nature of the problem and presence of camera motion.

Why bio-inspired solutions might be useful?

One of the motivation behind this is to replicate the efficiency and resiliency of these systems operating under limited power. In fact replicated such system has a profound impact on robustness and energy efficiency of fundamental task like depth estimation, collision avoidance, semantic segmentation and optical flow in robotics.

Sensors operating in robotics environments must be real-time and physically carried by the agent and be reliable under egomotion. Another intriguing property biological systems is their reliability against faulty neurons unlike artificial systems where a single fault can impose great damage.

Hassenstein-Reichardt Detector (HRD)

Correlation-type motion detector (from ref. 1).

After World War II, a biology student and a physicist collaborated to investigate optomotor response of beetles. Beetles in their experiments, in response to an optical stimulus provided by clockwise, slowly rotating, black-and-white striped cylinder, the animal actively turns. The key observation was beetles compensate their mistaken perception of self-motion by moving aligned with animals’ visual surroundings [1].

Approaches to motion detection

In broader sense visual motions estimation techniques are three-folds. correlation methods, gradient methods, and frequency methods.

The common assumption under which these methods operate is brightness constancy. Brightness constancy states image frame within small period of time must preserve the level of brightness.

Correlations methods disentangling visual features in different point in time. In other words, the observer receives two image sequence at time t1 and at time t where t1=t2+∆t. Perceived features do not need to be continuous as in the case Hassenstein-Reichardt Detector [4], they are discreet to presence and absence of a feature. Brightness intensity and detecting the pattern the intensity is changing between observation is the common approach to use this method. For instance, to calculate velocity one can shift perceived image at time t2 to image at time t1 to maximize the correlation between them. The amount shift required for this divided by ∆t can determine velocity [3].

Gradient methods work by assuming brightness changes over small displacement over short period of time is zero.

which is known as the optical flow equation.

Frequency methods on the other hands leverage the relationship between temporal frequency, spatial frequency and velocity.

Let’s consider a point or Dirac Delta function moving in x direction. This point will trace out a line in the space domain with velocity as the slope. In frequency domain, on the other hand, the slope equals to the inverse of the velocity [2].

Now, in this ideal case where the stimulus is a point, it has equal energy along all spatial frequencies.

How does Hassenstein-Reichardt Detector (HRD) works?

Comparing mentioned approaches to motion detection, correlation methods are computationally simpler but requires memory to store the previous frames [2]. The Hassenstein-Reichardt is inspired by the motion-sensitive neurons found in the visual system of insects. These neurons respond to the motion of visual patterns across their receptive fields. The HR detector is a model of these motion-sensitive neurons that can be used to estimate the motion of visual patterns in an image. The HR detector consists of two filters, spatial filters at different location or phase and a “temporal filter” filter usually models as a “low-pass” filter. The temporal filter is used to introduce a delay between two observations of the scene. The output of these filter is then combined using multiplication or AND operation, and this correlation is used to estimate the motion of the visual pattern [2].

HR motion detector (from ref. 2).

The use of correlation information, which does not contain amplitude information, makes this method robust to noise and fluctuations in illumination conditions. Furthermore, by relying on changes in pixel information, the Reichardt motion detector avoids the issue of size selection. However, it is important to note that as an elementary motion detector, the Reichardt motion detector may not yield accurate motion information. Additionally, it suffers from the aperture problem, where correct motion direction can only be gleaned at corner points, with ambiguous results along edges and no motion direction information from other positions. The limited perception range of velocity also constrains the development of this detector [5].

In summary, the Hassenstein-Reichardt detector is a bio-inspired algorithm for visual motion estimation that is based on the concept of spatiotemporal filtering. The algorithm takes as input a sequence of images and applies a spatiotemporal filter to each frame to detect motion in the scene. The algorithm might be useful in various fields, including depth estimation and optical flow estimation, and it has been shown to be robust and efficient in detecting motion in real-world scenarios.

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: