Not all oils are safe during pregnancy! You should always consult your doctor before using oils if you are pregnant. That being said, there are a few oils that are generally regarded as safe to use and even some that many midwives use to help the labor and delivery along!
Generally speaking, essential oils should not be used during the first trimester. Once you are past the first trimester, you can start to use some essential oils. Diffusing and massage are both great ways to make aromatherapy part of your routine. Common oils you can use during the rest of your pregnancy include lavender and peppermint, among others.
As far as using essential oils in the delivery room, a Mama in labor and after deliver has a lot going on and smells that were okay five minutes ago can be nauseating the next five minutes.
A baby's sense of smell upon delivery is super important for connecting to Mama and finding milk for the first time. I wouldn't want to interfere with that at all so it's important to remain very conservative with essential oil use around newborns and babies.
Instead of diffusers in the delivery room, opt for personal inhalers during labor and delivery. In the recovery room a diffuser would be wonderful with some gentle oils like lavender or chamomile or orange to help Mama relax.
you can download a full spreadsheet of oils and whether they are recommended during pregnancy here...