If you can't afford a generator for some reason ( ours is paying for my medicines). There are special cheap blankets. They are called mylar rescue blankets. You will not get cold I promise you. I put 1 around me last year. At the time our living room was quite cold. So I decided to test what I heard about them. The kind I got was very thin.
But they have better ones that are still cheap in cost. I wrapped one of the ones I bought around me & I stayed warm. It bounces your body heat back against you. I bought extras that I thought about handing some out to the homeless. I did give one away last year. The man seemed to be kept warm.

Talk with your electric company. It may surprise you how much help they can give you. I know when I talked with my Gas&Electric company they did a lot for me. I get a discount on my bill. They came out and gave me a new energy efficient refrigerator. And changed all my light 💡 to new ones. They do a lot of great stuff. It never hurts to ask. Especially since your both on oxygen. Please take care. I hope everything will work out well for you.

A generator would be great if you keep it outside. I live in Florida so it seems like a warm housecoat and blankets work well for me. Not sure about a gas heater. I guess as long as your not pumping oxygen into the room it would work. I have a gas stove and I always worry when I turn it on.
You could contact whomever it is that supplies your oxygen and see what they recommend. Take care.

Get a generator so you can have heat. Talk with the people who set it up so you can easily get it going and switch things out as you need them. Talk with your O2 provider and get them to bring several large tanks for when/if the electricity goes out. Call the electric company and make sure they know your needs. Talk with your children, friends and neighbors so they know you're making preparations; people are way more willing to help if they think you're doing all you can do to prepare for difficult times. And if you can help your neighbors, do that, too.......it really does take a village and not just for children; for all of us.

Do you have a deck or either front of back porch that is open? If so maybe a gas generator or a solar powered generator for that time. Just a thought