Shape or similar steps to propagate arbitrary input sizes and resize output correspondingly?

[Julian W. (nohype)]

Hi,
I hope this is the right place for my question and apologies in advance should this question simply be stupid.

TL;DR: Having a Resize() and LetterBox() step in the preprocessing of an arbitrarily sized input image, how would I add the postprocessing to go from a fixed-size output of the original model back to the (arbitrary) input size (i.e. undo the LetterBox() and Resize() steps)?

I'm a newbie to all things ML and started using image segmentation models with onnxruntime successfully so far. Now I've been trying to use onnxruntime-extensions to include necessary pre/post processing in those models and I was successful in having the extended model take an input image of arbitrary size, use Resize() and LetterBox() as you can see below for preprocessing.

input_size = (144, 256) # the original/unextended model's input size
pipeline.add_pre_processing(
    [
        ppp.Resize(input_size, policy="not_larger", layout="HWC", name="Resize"),
        ppp.LetterBox(target_shape=input_size, layout="HWC"),
        ppp.ReverseAxis(axis=2, dim_value=3, name="BGR_to_RGB"),
        ppp.ChannelsLastToChannelsFirst(has_batch_dim=False),
        ppp.ImageBytesToFloat(1.0 / 255.0),
        ppp.Unsqueeze([0]),
    ]
)

The model output is still a fixed size though. My question is how I would set up the postprocessing to take the fixed size output (shape [144, 256]), undo the LetterBox() step (somehow propagate the padding and use a CenterCrop() step?) and then resize to the original input's size (propagate the arbitrary original input size)?

It seems to resize to the original size, I should be able to use a Shape node to get the original input shape, Slice to extract the height and width individually, Concat to generate a (height, width)-tuple and feed that into a new final Resize node. I wouldn't know how to add those nodes to the model here though.
As for reversing the LetterBox() step, I don't really have an idea.

Thanks a lot for any help, hints, advice, pointers to resources or code to look at!

Kind regards