You might talk directly with the interagency permit office about this, but my recollection is that you are supposed to leave the TH at the same time, though that recollection could be faulty. I have called before and gotten a live person who has been helpful.

http://www.fs.usda.gov/contactus/inyo/about-forest/contactus

Eastern Sierra InterAgency Visitor Center
Visitor Services/Wilderness Permits
760-876-6222
760-876-6228 TDD
Junction Highway US 395 & SR136 (1 mile south Lone Pine)
Lone Pine , CA 93545
Hours: Daily 8:00am to 5:00pm