Re: leaving later, our main concern has been afternoon thunderstorms, so we were trying to make sure we were off the top part of the mountain by noon.
I had a bad experience a few weeks back on White Mountain (caught in a hail / lightening storm for 2hr on descent) so I was more concerned about that than cold.