[C74] Test-Time Fine-Tuning of Image Compression Models for Multi-Task Adaptability

Abstract

The field of computer vision initially focused on human visual perception and has progressively expanded to include machine vision. With ongoing technological advancements, this expansion is expected to continue further. Consequently, image compressors must effectively accommodate not only human visual perception but also current (closed-set) and future machine vision tasks (open-set). Many recent studies effectively address both human visual perception and closed-set machine vision tasks simultaneously but struggle to handle open-set machine vision tasks. To tackle this issue, this paper proposes a fully instance-specific Test-Time Fine-Tuning (TTFT) approach for adapting Learned Image Compression (LIC) to both closed-set and open-set machine tasks effectively. With our method, a large-scale LIC model, originally trained for human perception, is adapted to the target task through TTFT using Singular Value Decomposition based Low Rank Adaptation (SVD-LoRA). During TTFT, a modified learning scheme is used for the decoder to train only the singular values, preventing excessive bitstream overhead. This enables fully instance-specific optimization for the target task, even for open-set tasks. Experimental results demonstrate that the proposed method effectively adapts the backbone compressor to diverse machine tasks, outperforming competing methods.