CPU : Quite reliable; so only need to look out for physical damage; bent pins, I guess, would be the most common.
GPU : Not as reliable as CPU, but generally good used ones offer better VFM; especially in the current market. Things to look out for would be benchmarks, throttling, functioning of all ports, hardware info, etc.
RAM : Faulty memory is one of the most common issues, but easy to test. So, test with some credible tools.
PCB : Not as reliable as CPU, but better than GPU. Availability is probably the main issue in the used market, as people don't tend to upgrade the motherboard as often as other components. So situational, I'd say.
Look out for known issues in the specific model, signs of heat damage on VRM, whether used with a reliable PSU, UPS, etc.
STR : Gets worse with age. Check disk health, read/write speeds, physical damage (or even repair, like being bent back) on the port. Not usually recommended; should be fine for insignificant storage like backup.
PSU : Just avoid at all cost. Simply not worth the risk.