Here is your permission slip. Go. Put air in your nitrogen tires. I give you my full blessing.
Will the temperature swings affect your tire pressure a tiny bit more? Yes. Will you lose a microscopic amount of fuel economy? Maybe.
But you know what is worse than a 0.2% drop in fuel economy? Getting stranded on the side of the road because you were too scared to use the air pump. Don't be that person.
So next time you see that low tire light, just pull over. Don't overthink it. Your tires won't know the difference. Your wallet will be just fine. And honestly? You need to top it off anyway.