Did you check the headers at the chassis side?
From the description it sounds like a problem with USB header at the chassis (side).
In that case, you need to replace it.
Or, buy a 3.5'' front panel USB port set along with internal card reader. That will cost you 200/-
Cleaning won't do any...