(i) For two matrices A and B,

Solution:

(i)

Given : $A=\left[\begin{array}{lll}2 & 1 & 3 \\ 4 & 1 & 0\end{array}\right]$

$A^{T}=\left[\begin{array}{cc}2 & 4 \\ 1 & 1 \\ 3 & 0\end{array}\right]$

$B=\left[\begin{array}{cc}1 & -1 \\ 0 & 2 \\ 5 & 0\end{array}\right]$

$B^{T}=\left[\begin{array}{ccc}1 & 0 & 5 \\ -1 & 2 & 0\end{array}\right]$

Now,

$A B=\left[\begin{array}{lll}2 & 1 & 3 \\ 4 & 1 & 0\end{array}\right]\left[\begin{array}{cc}1 & -1 \\ 0 & 2 \\ 5 & 0\end{array}\right]$

$\Rightarrow A B=\left[\begin{array}{cc}2+0+15 & -2+2+0 \\ 4+0+0 & -4+2+0\end{array}\right]$

$\Rightarrow A B=\left[\begin{array}{cc}17 & 0 \\ 4 & -2\end{array}\right]$

$\Rightarrow(A B)^{T}=\left[\begin{array}{cc}17 & 4 \\ 0 & -2\end{array}\right]$          ...(1)

$B^{T} A^{T}=\left[\begin{array}{ccc}1 & 0 & 5 \\ -1 & 2 & 0\end{array}\right]\left[\begin{array}{ll}2 & 4 \\ 1 & 1 \\ 3 & 0\end{array}\right]$

$\Rightarrow B^{T} A^{T}=\left[\begin{array}{cc}2+0+15 & 4+0+0 \\ -2+2+0 & -4+2+0\end{array}\right]$

$\Rightarrow B^{T} A^{T}=\left[\begin{array}{cc}17 & 4 \\ 0 & -2\end{array}\right] \quad \ldots(2)$       $\ldots(2)$

(ii)

Given : $A=\left[\begin{array}{ll}1 & 3 \\ 2 & 4\end{array}\right]$

$A^{T}=\left[\begin{array}{ll}1 & 2 \\ 3 & 4\end{array}\right]$

$B=\left[\begin{array}{ll}1 & 4 \\ 2 & 5\end{array}\right]$

$B^{T}=\left[\begin{array}{ll}1 & 2 \\ 4 & 5\end{array}\right]$

Now,

$A B=\left[\begin{array}{ll}1 & 3 \\ 2 & 4\end{array}\right]\left[\begin{array}{ll}1 & 4 \\ 2 & 5\end{array}\right]$

$\Rightarrow A B=\left[\begin{array}{ll}1+6 & 4+15 \\ 2+8 & 8+20\end{array}\right]$

$\Rightarrow A B=\left[\begin{array}{cc}7 & 19 \\ 10 & 28\end{array}\right]$

$\Rightarrow(A B)^{T}=\left[\begin{array}{cc}7 & 10 \\ 19 & 28\end{array}\right]$ ...(1)

Also,

$B^{T} A^{T}=\left[\begin{array}{ll}1 & 2 \\ 4 & 5\end{array}\right]\left[\begin{array}{ll}1 & 2 \\ 3 & 4\end{array}\right]$

$\Rightarrow B^{T} A^{T}=\left[\begin{array}{cc}1+6 & 2+8 \\ 4+15 & 8+20\end{array}\right]$

$\Rightarrow B^{T} A^{T}=\left[\begin{array}{cc}7 & 10 \\ 19 & 28\end{array}\right]$

$\therefore(A B)^{T}=B^{T} A^{T}$          [From eqs. (1) and (2)]