Went there for the caves, but at the end, the whole experience was marvelous. The hotel is clean and family run. Tasty breakfast, specially don´t refuse to try local food, say yes to the Menemen (I believe it´s written like that) which is basically scrambed eggs with tomato, onion and spices.
The Goreme town is I think the best location to stay, or consider Urgup. For local tours, try to contact local agencies; better prices and more knowledge.
Even if you are not eager to do the ballon ride, as soon as you see the local scenery, you will understand why it is a MUST. The views and energy of the whole area will tatoo you forever. Having breakfast on a terrace with 15 to 20 ballons in the air, and unique rock formations... priceless...