Abstract: Aiming at the problem that the classical least mean square (LMS) filtering algorithm in the strong reverberation background is difficult to separate signal and reverberation effectively, an adaptive LMS algorithm based on fractional Fourier transform (FRFT) is proposed. Firstly, FRFT is performed to the reverberation signal and the reference signal of adaptive LMS filtering algorithm to find out the optimal transform domain, and the band-pass filtering is performed in the fractional order domain, then the fractional Fourier inverse transform (FRFIT) is performed to the obtained signal; and finally the variable-step-size LMS algorithm based on normal distribution curve is used for filtering in this reverberation condition. The verification results of simulation and sea-trial show that the algorithm can effectively filter out the reverberation under a signal-to-reverberation ratio of 0.3 dB, and the signal-to-reverberation ratio is improved by 6 dB.