I have done it both ways. If there is an opportunity to rest (potentially shower) overnight before the potentially long drive, I would much more prefer to leave in the morning (I prefer travelling in the morning vs night travel)
The body betrays and the weather conspires, hopefully, not on the same day.