Coil whine is not due to poor quality capacitors. You only need ask the people who own some of the Seasonic X-Series power supplies that use all Japanese 105°C electrolytic capacitors and are afflicted with this problem.
Coil whine may also happen on the coils/chokes in VRM circuits of motherboards and graphics cards.
The coil/choke is a coil of wire wrapped around a ferrite core. If the core is loose it will vibrate at some harmonic of the PSU's switching frequency.
If the warranty on the device has already expired, I have heard of some people solving this problem by applying nail polish or hot-melt glue to the offending coil(s) to prevent the core from vibrating.
Coil whine is considered a manufacturing defect. Sometimes it's caused during shipping of the PSU to the end user. You really don't know if the package has been thrown around during its transport.