Hi Chevyll,
I use a Zing Standard SLR Action Cover. This is a one-piece, wrap-around neoprene cover that stretches over the lens and self-secures without buckles, snaps or zippers. The case has a removable elastic waist strap. When used in conjunction with the camera neck strap, the waist strap holds the camera tight to the body so the camera does not bounce/sway/etc... The camera can be removed from the cover for shooting without having to unfasten the waist strap, and the waist strap secures the cover.
This way I always have the camera secured to my body (with the camera strap), and provides limited protection from the elements and from being banged around.
I guess your setup is a bit easier to grab your camera for quick shots, but taking off the cover is really simple so not a big hindrance.
See:
http://www.bhphotovideo.com/c/product/75792-REG/Zing_Designs_540_111_Standard_SLR_Action_Cover.html for more details.