If AM and GM of roots of a quadratic equation are 8 and 5 respectively,

Let the roots of the quadratic equation be $a$ and $b$.

$\mathrm{AM}=8$

$\therefore \frac{a+b}{2}=8$

$\Rightarrow a+b=16 \quad \ldots \ldots \ldots$ (i)

Also, $G=5$

$\Rightarrow \sqrt{a b}=5$

$\Rightarrow a b=5^{2}$

$\Rightarrow a b=25$     ....(ii)

Now, the quadratic equation is given by

$x^{2}-(a+b) x+a b=0$

$\Rightarrow x^{2}-16 x+25=0$