Abstract: For the solution of large-scale acoustic problems, the fast multipole boundary element method (FMBEM) is an efficient numerical technique. It is typically coupled with iterative algorithms for solving the resulting linear systems; thus, its overall computational efficiency is strongly influenced by the convergence behavior of the chosen iterative solver. In this paper, we adopt the rank-one inverse Broyden quasi-Newton iterative method—a method exhibiting superlinear convergence—for the iterative solution within the FMBEM framework. Regarding the selection and storage of the initial approximation matrix in this iterative scheme, we employ incomplete LU (ILU) decomposition and propose a threshold-ratio-based elimination criterion to construct the preconditioner. Moreover, to avoid explicit formation and direct storage of the iterative correction matrix, we develop an efficient implicit update strategy. Furthermore, a multi-core parallelization algorithm is introduced to accelerate the entire iterative solution procedure. Numerical comparisons on representative acoustic benchmark problems demonstrate that the proposed iterative approach reduces the number of influence coefficient evaluations—both in the near-field and far-field interactions—compared with the widely used generalized minimum residual (GMRES) method. This reduction confirms the superior computational efficiency of the method presented in this paper.