I believe there is a healthy balance between weight and comfort, but everyone's balance is different. I'm a cold sleeper so I need a warmer bag/quilt in all weather than the typical backpacker. But I know I will stay warm and comfortable so that weight penalty is worth it to me.
I have had knee surgery and cervical spinal surgery. My body can no longer handle heavy weights anymore either though. So while others say they could never backpack without camp shoes, I forgo that extra half pound or so. And so on. I will sacrifice certain comforts for less weight while not sacrificing all comforts.