Dual-Stream Adapters for Open-Set Segmentation in Driving Scenes

The task of segmenting novel categories in road scenes, often referred to as anomaly segmentation, has been recently addressed with great success by using mask-based architectures, but their efficacy is dependent on fine-tuning large transformer backbones. In this work, we design a specialized adapter for this task, which makes it possible to leverage even large backbones without re-training them. The key feature of our adapter is the separation of the adapted features in two streams, one specialized on the known categories (in-distribution) and the other that captures the characteristics of out-of-distribution categories. The out-of-distribution features adaptation is supervised by using synthetic negative data generated by a normalizing flow process. This dual-stream architecture allows to better disentangle features for known and unknown categories, preserving in-distribution performance while enabling direct and more accurate anomaly segmentation with fewer false positives. Experiments show that dual-stream adapters outperform previous methods while reducing training parameters by 38\%.