As long as your baby can't touch a lightbulb, it's no problem!
You'll probably see your baby staring at all kinds of things—everything is new to a baby! Still, it's normal to worry that staring at a lightbulb could somehow cause vision damage. In fact, your baby is probably staring at the lights because his or her long-range eyesight is still developing (depending on the age, of course), and lights provide wonderful contrast between bright and dark.
So for the first six months, it's common for babies to stare at lights, ceiling fans, and other moving and/or contrasting things. There's no harm in this, and it's not a sign of any developmental issue. Afterward, as baby learns to crawl and directly interact with the environment, you'll probably notice less staring because they're too busy exploring!