Imports of `ffast-math` compiled extensions change global FPU state

For example `import imops` breaks floating point subnormals flushing them to 0:
```python
import sys
sys.float_info.min / 2 != 0.0  # True

import imops
sys.float_info.min / 2 != 0.0  # False
```
