My entire life, I've had a problem with boots. I can find shoes that fit perfectly and I can wear all day in all conditions, but when I wear boots, I >always< get very painful cramps in my right foot in the muscles where my "ring finger" toe (aka the toe next to my pinky) attaches to my foot.
I don't now if it's something about the high tops around my upper ankle, lower leg, but this is something that makes all outdoor camping and hiking a literal pain. As I say, it has happened my entire life. When I was in Scouts, I couldn't go half a mile without having to stretch my toes. In the military, I was in pain all day when I wore boots. People always told me I was just trying to get out of work or effort or whatever.
Has anyone ever heard of this?
First, unless you are hiking in the winter, off-trail in rugged areas, or carrying very heavy loads, you probably don't need boots. Many people backpack in trail runners these days. However, part of what makes that work is keeping you gear on the lighter side.
As far as the specific foot pain I would go to a podiatrist and see if there is something they can do for you.
I'm a backpacker and switched to trail runners about 5 yrs ago and haven't looked back. The thing that convinced me to try was thinking...ten thousand thru-hikers can't be wrong! (I'm not a long trail thru hiker).
You might also think about taping trouble spots on your feet prior to putting on your boots and see how that works.
A few years back, I transitioned from boots to trail runners, for essentially the same reasons that you currently are enduring. However, this past summer my son introduced me to two boots that have made a huge difference.
1st Crispi boots (I use them mainly for off trail during the hunting season.) The nice thing about Crispi there is no break-in period. You can literally wear them out of the box.
2nd Oboz boots (made in Bozeman, Montana) their boots are made totally different from any other boot that I have ever seen. I can wear them with obsolete no pain at all! I also didn’t have to break-in my Oboz.
You might want to check both of them out!