Optimizing Omnimotion: Enhancing Efficiency and Speed in Dense Full-Pixel Tracking

Motion estimation in computer vision (CV) presents significant challenges, particularly in tracking points across occlusions. The Omnimotion model, introduced in the paper "Tracking Everything Everywhere All at Once," addresses this issue by mapping video points from local frames into a global 3D representation, allowing for accurate tracking. This model leverages Invertible Neural Networks (INNs) and Neural Radiance Fields (NeRF) to maintain visibility across frames, but its computationally intensive structure and reliance on RAFT for pre-processing demand significant computational resources and lengthy training times. This study proposes several optimizations to reduce these computational dependency. By eliminating non-essential loss functions and introducing an embedding mechanism for internal MLP layers, training complexity and time were significantly reduced. Additionally, the implementation of Tiny-CUDA and weight freezing techniques further enhanced performance, cutting memory usage by 50% and improving training speed by nearly twofold. Although a slight decrease in model quality was observed, these optimizations expand the potential use of Omnimotion for larger datasets and real-time applications.