Abstract: The rotor shafting of high speed and high power permanent magnet synchronous motor (PMSM) is a nonlinear system involving electromagnetic and mechanical coupling, so the dynamic performance of the rotor is a problem that must be paid attention to in the PMSM design for high speed vehicles. A nonlinear model of the rotor system was established based on nonlinear rotor dynamics. The amplitude frequency characteristics of the lateral vibration of the rotor were calculated with the analytical method. The results show that the lateral vibration of the rotor reveals negative stiffness and instability amplitude jumping phenomenon. Finally, the results of equivalent analytical calculation were verified by numerical calculation.