You could raise your fitness with short, sharp rides, stay within Z2-Z3 levels on longer rides and you should sleep ok. Bike-packing sleep should be the easiest, after all those miles and a full stomach in a warm sleeping bag : )
Maybe the short sharp rides are not short and hard enough. A usual ride for me will be 2-4 hours at a pace about 10 - 15 bpm below my AT. If I go to the gym and do a 40 minute spinning class I am above my AT for virtually the whole thing. That is much harder.
I agree the sleep should be easy, but the combination of riding too long, too far, too hard and not being quite as fit as I need to be means the body is stressed too much.
Some good points have been made on here.
PS - I am really keen to read about your training for the TD and how you went about preparing for it, especially, given the daily mileage you were doing!