How is it in regards to permits staying a night before or after your permit date?

My wife and I are now thinking we didn't allow enough time to summit, hike down, and drive back 1 one day. We were thinking of going up a day early.