a really easy way to tell if its a motor or not, is to take off the door panel and smack the motor with a hammer a few times while operating the switch. If it starts moving then you have a bad motor, if not it may still be a motor problem. If you have access to a test light backprobe the connector to the motor and operate the switch, it should only light up on one side going one way (ex: the left side lights up going down and the right side lights up going up). If one side lights up going both ways then its a bad motor.
And yes a switch could be the cause. If the passenger switch breaks in the up position then the motor is constitantly commanded in the up position. This may also burn out the motor.