The other danger is working with someone who has no clue what your sensibilities are, and has never been to the destination or resort you want to visit. Why would you trust your vacation to this person?
What I do is this: First, I stick with an agent who has been recommended by friends, or who knows me and wants my ongoing business. Second, I ask if they've been to the destination I'm going to, and if they know the hotel or resort first-hand. Third, I do all my homework first, with a guidebook, and then tell the agent where I want to stay, and let them make the bookings for me. When they come up with a price, I check it against what I can get online. And being the skeptic that I am, I then call the resort--both its 800 number and its local number--and compare rates. When I'm spending a bundle I may also check with a second agency. Some work closely with specific hotel chains and may be able to get you a better deal.