In this paper, we show how support vector machine (SVM) can beemployed as a powerful tool for $k$-nearest neighbor (kNN)classifier. A novel multi-class dimensionality reduction approach,Discriminant Analysis via Support Vectors (SVDA), is introduced byusi