Abstract: In order to improve the robustness to variable factors such as illumination, expression, and pose, a novel face recognition method based on sparse representation with multi-directional Gabor feature maps was proposed in this paper. Firstly, multi-directional and multi-scale Gabor transforms were performed on face image, and the obtained Gabor features with different scales in the same direction were fused to generate multi-directional feature maps. Then, Gist features were extracted and adaptive weights were assigned to them for the fused feature maps in each direction. The adaptive-weighted Gist features of all directional feature maps were cascaded to form feature descriptors of face image. Finally, face recognition was implemented with a sparse representation classification method base on the face feature descriptors. Experimental results show that the average recognition rates of the proposed algorithm on Yale, ORL and Extended Yale B face databases are 99.8%, 99.7% and 100.0% respectively.