review memoryview usage

currently we're using [memoryview](https://docs.python.org/3/c-api/memoryview.html) in a lot of places, for zero-copy operations. We should check-in on how we're using native bytes type vs memoryview- and make sure our usage still aligns with optimal performance

this is low priority