MVF-PointCLIP: training-free multi-view fusion PointCLIP for zero-shot 3D classification
The remarkable success of Contrastive Language-Image Pretraining (CLIP) in zero-shot 2D vision classification inspires researchers to explore its potential application to zero-shot 3D classification. Some researchers project 3D point clouds into 2D images from multiple views to leverage CLIP. Howeve...
| Autores: | , , , , , , |
|---|---|
| Tipo de documento: | artigo |
| Estado: | Versión aceptada para publicación |
| Data de publicação: | 2025 |
| País: | España |
| Recursos: | Universidad Pública de Navarra |
| Repositório: | Academica-e. Repositorio Institucional de la Universidad Pública de Navarra |
| OAI Identifier: | oai:academica-e.unavarra.es:2454/55920 |
| Acesso em linha: | https://hdl.handle.net/2454/55920 |
| Access Level: | Acesso embargado |
| Palavra-chave: | Point cloud CLIP Multi-view fusion Zero-shot Training-free |
| Resumo: | The remarkable success of Contrastive Language-Image Pretraining (CLIP) in zero-shot 2D vision classification inspires researchers to explore its potential application to zero-shot 3D classification. Some researchers project 3D point clouds into 2D images from multiple views to leverage CLIP. However, by doing experiments, we find that this method suffers from two critical drawbacks: (1) noise views existing in multiview depth maps, which provide limited information that may mislead classification; (2) covariance inconsistencies between sample views, which can lead to misclassification when using cosine similarity. To address these issues, we propose a training-free MultiView Fusion PointCLIP (MVF-PointCLIP). It contains a Spatial and Frequency Attention (SFA) module and a Mahalanobis Distance module designed by us. The SFA module automatically assigns importance weights to views, effectively filtering out noisy information. The Mahalanobis Distance module models the distribution of views to tackle covariance inconsistencies. Experimental results verify the superiority of MVF-PointCLIP to SOTA models in zero-shot classification across ModelNet10, ModelNet40, and ScanObjectNN. |
|---|