Sparse Adversarial Attacks on Video Action Classification: An l0-/l1-Norm-Constrained Projected Gradient Descent Approach

Video models remain vulnerable to adversarial attacks, yet most of the adversarial attack litera- ture is image-centric, and the video-specific attacks that do exist treat sparsity only as a matter of which frames to perturb, leaving each selected frame densely corrupted. This thesis investi- gates sparsity also at the pixel level, proposing an l0- and l1-constrained PGD adversarial attack that enforces sparsity directly inside the optimization loop, combining an adaptive, gradient- informed frame-selection mechanism with a pixel-level hard-thresholding or l1-ball projection. We further study how a fixed sparsity budget should be distributed across a video’s frames, com- paring a Global allocation, where the budget is spent freely, against a Frame-wise allocation, where it is distributed uniformly per frame. Evaluated against Conv+LSTM, I3D, and Video- MAE architectures on UCF101 and Kinetics400, our attacks reach fooling rates close to the two existing sparse video-attack baselines while corrupting only a fraction of the pixels and at the lowest per-iteration computational cost among the compared methods. The two budget- allocation strategies prove almost equivalent on aggregate fooling rate, SSIM, and perturbation magnitude, with Frame-wise allocation retaining a structural advantage in upper-bounding any single frame’s perturbation. Examining where our attacks place their perturbation budget qualitatively reveals architecture-dependent patterns: perturbations align with VideoMAE’s patch-based tokenization, propagate through Conv+LSTM’s recurrent hidden state, and dis- tribute more broadly across I3D’s convolutional feature maps — evidence that sparse perturba- tion patterns are shaped as much by a model’s structure as by the attack’s budget.