It’s possible, but you should see if there could be another culprit before assuming it was something in your diet.
There are lots of myths out there about breastfeeding, and one of them is that certain foods should be avoided by mom because they will not be tolerated well by baby when he or she nurses. The great majority of babies do just fine no matter what moms eat, and studies show us that some even prefer milk from moms who eat more strongly-flavored foods!
If you do notice that your baby gets fussy after you eat a certain food, first try to see the bigger picture. Could something else be going on, such as teething, a missed nap, or a growth spurt? Is your baby coming down with a cold, or did you just start a new medication that could have decreased your milk supply (for example, like Sudafed)? Is it the holidays and there are lots of visitors and your baby is overstimulated?
If you truly think it is related to something you ate, go ahead and avoid that food. Note that you will need to give it a few days to get out of your system. If you have definitely linked a food to a reaction in your baby, you should avoid it for the time being. Then you can try to reintroduce it again in a few months as many older babies will often grow out of these intolerances.