There is a scenic drive back from the Caverns to the entrance. It is a good gravel road, but most people do not take it and we only encountered two other vehicles. Here are pictures outside the caverns and on the scenic drive:
The deer seem to ignore the "No Picnicking" sign: