feat: Support for allocating GPU memory based on the selected profile

[anmolgupt]

The changes in the PR support 2 main items:

1. the GPU memory is allocated based on the selected TensorRT profile and not based on the profile that consumes max memory even when it's not selected.
2. Avoid the creation of profile 0 execution context if it's required.

[dongfengy]

For kUSER_MANAGED , the user (in this case the triton server) would need to actually allocate a piece of device memory and pass to execution context. Do you have support for this behavior? If not I would suggest you to only add kSTATIC and kON_PROFILE_CHANGE

[yinggeh]

As currently triton supports one context per engine, in order to set the specified profile_index, another context is created and the previous context is destroyed.

Is the comment still valid? From the code, each profile_index holds a context.

    if (profile_index == 0) {
      res.first->second.context_ = std::move(default_trt_context);
    } else {
      res.first->second.context_.reset(engine_->createExecutionContext());

[anmolgupt]

I tested at my end, the changes work for me.

[anmolgupt]

@yinggeh: New changes look good to me; I got the expected results on the models with these updates.

[yinggeh]

Updated README.md

[yinggeh]

LGTM. Thanks for your contribution.