Including sweets (and not elevating them to special status) can help temper this response. A prudent amount is none to one serving (small portion) per day. Instead of freedom to run through the house with a lollipop, encourage your child to sit at the table while eating sweets. This sets up an expectation that desserts are treated similarly to meals and snacks, and it prevents mindless eating.
Another approach is to offer dessert at the same time as the meal or snack, making the sweet status on par with fruit, veggies, and other foods. Offer sweets in the manner that works for your family, but try to keep the overall average (over a week, for example) to less than or equal to one per day.