Enabling precise traffic filtering based on protocol encapsulation rules