Offer fresh fruits and veggies with the skin on (if safe) like apples, pears, berries, and celery. Nectars (concentrated juices) like pear or apricot nectar have been effective at relieving constipation. Prune juice, warmed baby prunes, and dried fruit for the older toddler can help.
Be sure to work whole grain sources, like whole wheat bread, pasta, and crackers, into the diet. Offer milk with meals and snacks and water in between to ensure plenty of fluids. If your toddler drinks too much milk, it could contribute to constipation.