Two charges –q each are separated by distance 2d. A third charge +q is kept at midpoint O. Find potential energy of +q as a function of small distance x from O due to –q charges. Sketch PE versus x and convince yourself that the charge at O is in an unstable equilibrium.
In the above figure, +q is the charge that got displaced from O towards (-d,0).
This is written as
$U=q\left(V_{1}+V_{2}\right)=q \frac{1}{4 \pi \epsilon_{0}} \frac{-q}{(d-x)}+\frac{-q}{d+x}$
$U=\frac{1}{2 \pi \epsilon_{0}} \frac{-q^{2} d}{d^{2}-x^{2}}$
At x = 0
$U=\frac{1}{2 \pi \epsilon_{0}} \frac{q^{2}}{d}$
Differentiating the equation with respect to x, we get
When x < 0, dU/dx > 0
And when x > 0, dU/dx < 0
Using this we can define, charge on particle to be F = -dU/dx
F = -dU/dx = 0
When
d2U/dx2 = positive, equilibrium is stable
d2U/dx2 = negative, equilibrium is unstable
d2U/dx2 = 0, equilibrium is neutral
Therefore, when x = 0, d2U/dx2 = (-2dq2/4πε0)(1/d6)(2d2) < 0
Which shows that the system is unstable equilibrium.