Abstract: To address the prevalent issue of “compliance with standards yet still causing public annoyance” in China—and the limitation that traditional deterministic models cannot quantify the aleatoric uncertainty inherent in individual subjective responses—this paper investigates the factors influencing noise annoyance and proposes a novel probabilistic prediction method. Based on 3,371 valid responses from a socio-acoustic survey conducted in 2023, we analyzed the effects of sound pressure level, noise source type, and age on annoyance. Subsequently, we constructed a probabilistic prediction model based on a Multi-Layer Perceptron (MLP) architecture, incorporating a probability distribution layer to capture the stochastic nature of subjective evaluations. Results show that the daytime sound pressure level thresholds associated with a 10% high-annoyance prevalence are 57 dB(A) for social noise, and 60 dB(A) for road traffic and construction noise. The effect of age follows a “U-shaped” pattern, whereas gender showed no statistically significant effect. The proposed model effectively predicts individual high-annoyance probability (AUC = 0.7653) and supports regional adaptation via transfer learning. This study clarifies annoyance thresholds for common noise sources and quantifies the impact of demographic characteristics, providing a new tool for refined urban noise assessment.