Virus and bacteria are in our body. As long as we dress properly we don't catch the cold. When our body work more to keep the good temperature we become weaker and then get the virus or bacteria developp and we get sick.
Or why we tend to catch cold in winter and almost never in summer?
because of the proximity with others in winter given we stay indoors more, etc.
Maybe you’re right that that your body having to work harder when it’s cold, etc. Or that you need to have the “goldilocks” zone for a virus or bacteria to infect you. I don’t agree with that point of view but I’m no doctor.
edit: to add to this, I’m not a person who got sick much in my life. The last year and a half I have a consistent runny nose. Why you ask? A now 2 1/2 year old in daycare since he was 1...
edit#2: I understand that your body has to have the right condition for a virus to infect you, but I don’t think underdressing is going to be the reason your body falls ill. If like you say you are infected, it will run it’s course one way or the other.