To be or not to be a Hillclimber or Time Trialist?

A lot has been going on in my brain about trying to be a climber or a TTist or both. It is hard to be both. To be a great climber you need to light and thin, to be a TTist you can't be climber light and do super good in the time trials or can you? I have always struggled with my weight, I am a whopping 150 lbs. No matter how hard I try to get down the scale doesn't drop. I am afraid to try to drop weight now when my training is pretty tense and losing muscle or something. I have decided to focus on time trials and not completely on hill climbing. I have never really been that good on the climbs anyways...I have never won a hill climb, TTs I have. I am not going to cut out climbing, I am just not going to make it my main deal.