The structure of exhaust system influences the loudness of exhaust noise significantly. This paper indicates the impact of the structure parameters of exhaust system on the loudness of exhaust tail noise and presents an evaluation model to conduct the improvement of exhaust system for improved loudness. Firstly, the twenty-seven structure samples are designed by orthogonal experiment and exhaust tail noises of those are obtained by simulation. The accuracy of simulation is also validated by the experiment successfully. Then the structure-loudness model is developed by the radial basis function (RBF) network technique. Subsequently, the contributions and main effects of the structure parameters on loudness are indicated in this work. Finally, based on the structure-loudness model, the robust optimum design for improved loudness is studied by the adaptive simulated annealing (ASA) algorithm. The result of optimization certifies that the structure-loudness model is useful for the optimization. The method applied in this paper can be feasible for other psychoacoustic metrics.