You are taking enough time, and altitude issues take some time to show up. I would expect people to feel the effects in the morning after spending the night at altitude. At that point, they could just decide to remain at Trail Camp while others hiked to the summit.
Having a buddy go back with a sickened hiker is good, especially if the person needed it. However, if they were still doing "pretty well", I might consider letting them descend alone as long as it was daytime. You would need to evaluate how well the person might do alone, though.
As for the permit, take some note cards along and write the permit number down, and each member can carry one.
