Abstract: In this paper, a modified TOPS (test of orthogonality of projected subspace) algorithm based on beamspace, named BMTOPS algorithm, is proposed to overcome the shortcoming of TOPS algorithm that the false peak is easy to appear at low signal to noise ratio. A diagonal correction matrix is added to the orthogonality test matrix in the TOPS algorithm, and the beamspace method is applied to preprocessing the received data of the array, so as to achieve the DOA estimation of wideband signal. Simulation results show that compared with TOPS algorithm, BMTOPS algorithm can suppress false peaks in beam direction, improve resolution probability and reduce computation, which would be a new algorithm with application prospect.