Hi.
In this case, I'd say chances are it is the switch.
In high speed, the power bypasses the resistor.
In the diagram ,notice position 5 goes direct to the blower.
Also, with the issues you are having, it may be a good idea to make sure the blower motor is not drawing to much energy. It could be overloading the circuit and taking out these components.