|
|
|
|
|
|
|
|
|
|
|
|
|
| Learning modality invariant features is central to the problem of Visible-Thermalcross-modal Person Reidentification (VT-ReID), where query and gallery images comefrom different modalities. Existing works implicitly align the modalities in pixel andfeature spaces by either using adversarial learning or carefully designing feature extrac-tion modules that heavily rely on domain knowledge. We propose a simple but effectiveframework, MMD-ReID, that reduces the modality gap by an explicit discrepancy re-duction constraint. MMD-ReID takes inspiration from Maximum Mean Discrepancy(MMD), a widely used statistical tool for hypothesis testing that determines the dis-tance between two distributions. MMD-ReID uses a novel margin-based formulation tomatch class-conditional feature distributions of visible and thermal samples to minimizeintra-class distances while maintaining feature discriminability. MMD-ReID is a simpleframework in terms of architecture and loss formulation. We conduct extensive exper-iments to demonstrate both qualitatively and quantitatively the effectiveness of MMD-ReID in aligning the marginal and class conditional distributions, thus learning bothmodality-independent and identity-consistent features. The proposed framework signif-icantly outperforms the state-of-the-art methods on SYSU-MM01 and RegDB datasets. |
 |
| MMD-ReID: Structure of our two stream architecture for VT-ReID. Modality specific layers (L-0, L-1, L-2) have independent weights for each modality. Modality shared layers (L-3, L-4, Pool, BN, FC) have shared weights for both modalities, denoted by dotted by bi-directional arrows. Visible and Thermal features are extracted independently and ID loss is applied. Margin MMD-ID and Hc-Tri are applied on pooled features. |
 |
Chaitra Jambigi, Ruchit Rawal, Anirban Chakraborty. MMD-ReID: A Simple but Effective solution for Visible-Thermal Person ReID. In BMVC, 2021 (oral). (hosted on ArXiv) |
Acknowledgements |