We investigate the moments of $L(1,\x_P),$ where $\x_P$ varies over quadratic characters associated to irreducible polynomials $P$ of degree $2g+1$ over $\mathbb{F}_q$ as $g\to\infty$. We extend to the function field setting the asymptotic formulas for all the moments of class numbers of quadratic function fields with prime discriminant that was established by Nagoshi in the number field setting. Moreover, we give asymptotic formulas for the complex moments of $L(1,\x_P)$ in large uniform range.