Sam, it is more direct, but without snow it is an uneven rocky mess. One misstep and a rock could shift under you and you slip or fall. People prefer the trail because it is even and the footing is easy.
Granted, when it is fully covered with snow, people with glissade down in the right conditions -- they sit in the snow, and use an ice axe as a brake. It takes 15 minutes to get down this way. However, people have died trying to slide that slope on the snow. In the melt-freeze cycles, in the later afternoon, when the Whitney crest shadow hits the slope, soft snow can ice up, and it becomes treacherous.
Now Steve, the Chamber of Commerce is not going to be happy with that dying thingy.