Here are a few options based on my travels: 
Farmington, NM - Affordable town just an hour south of Durango with excellent fishing on the San Juan and the whole four corners region nearby. Can be hot in the summer, but you can escape to the mountains.
Laramie, WY - College town with a nice little downtown area. Laid back and has everything you need. Great hiking, hunting, and fishing opportunities in the Medicine Bow-Routt National Forests. Can be windy in the winter.
Cody, WY - Gateway to Shoshone National Forest and Yellowstone National Park. Plenty of fishing and hunting opportunities. 
Grand Juction, CO, as you mentioned is also a good option. In general it seems the further you are from a major airport or ski resort the more affordable towns become.