From what I can tell from reading the (short!) source, there doesn't seem to be a way to use this on files too large to fit in memory. In particular, the ByteString's are strict, and if one simply tries splitting and sending in the parts, there's a possibility of getting errors like
Left (XenoParseError "Couldn't find the matching quote character.")
I'd rather not have Lazy ByteStrings because reasons, though that would probably be the simplest solution.
An alternative would be to add an eofF :: (Int -> m ()) to process, and pass it the index of the last fully processed element, and then call that wherever we now call throw, which would allow simply sending in arbitrarily sized ByteString's and stitch them using the eoF function. But most of the throw's happen in s_index – making that return a Maybe seems like it would hurt performance.
Or is it possible to make indexEndOfLastMatch an argument of XenoException's, and catch the throw's without losing the monad return value?
From what I can tell from reading the (short!) source, there doesn't seem to be a way to use this on files too large to fit in memory. In particular, the ByteString's are strict, and if one simply tries splitting and sending in the parts, there's a possibility of getting errors like
Left (XenoParseError "Couldn't find the matching quote character.")I'd rather not have Lazy ByteStrings because reasons, though that would probably be the simplest solution.
An alternative would be to add an
eofF :: (Int -> m ())toprocess, and pass it the index of the last fully processed element, and then call that wherever we now callthrow, which would allow simply sending in arbitrarily sized ByteString's and stitch them using the eoF function. But most of the throw's happen ins_index– making that return a Maybe seems like it would hurt performance.Or is it possible to make
indexEndOfLastMatchan argument ofXenoException's, andcatchthe throw's without losing the monad return value?