Heater/AC Blower motor

I have a 2003 expedition and recently have been having problems with the blower. It will sometimes be on the low setting and blow hi. or sometimes it works as it should. If it is blowing HI and I manually press the switch to lower the blower it will not lower, however if i press and hold the switch it will go down. I need some help here, do you think it is the switch or the blower resistor?

Thanks in advance!

I suspect it is the switch. Holding the swich down wouldn't lower the air speed if the resistors were bad. Gotta go with the switch. Anybody?

So here is my next question, if it is the switch how hard is it to switch those out? Does that plastic trim pop off in some sort of way and then just a electrical connection on the switch?

Thanks for the quick response.