Deterministic bridge regression for compressive classification

Pattern classification using a compact representation is a crucial component of machine intelligence. Specifically, it is essential to learn a model with well-regulated parameters to achieve good generalization. Bridge regression provides a mechanism for regulating parameters through a penalized l_p-norm. However, due to the nonlinear nature of the formulation, an iterative numerical search is typically used to solve the optimization problem. In this work, we propose an analytic solution for bridge regression based on solving a penalized error formulation using an approximated l_p-norm. The solution is presented in primal form for over-determined systems and in dual form for under-determined systems. The primal form is suitable for lowdimensional problems with a large number of data samples, while the dual form is suitable for high-dimensional problems with a small number of data samples. We also extend the solution to problems with multiple classification outputs. Numerical studies using simulated and real-world data demonstrate the effectiveness of our proposed solution.