Support AWS SigV4 in the http output plugin

[dlvenable]

Is your feature request related to a problem? Please describe.

We would like to enable Fluentd to write to Amazon OpenSearch Ingestion (OSIS). The current http output plugin in Fluentd can write to open-source Data Prepper's http source plugin, which OSIS supports.

All OSIS pipelines require AWS SigV4 authentication which the Fluentd http output plugin does not support.

Describe the solution you'd like

Provide an AWS SigV4 option in Fluentd's http output plugin. Create a new auth option named aws_sigv4. Add options with the aws_ prefix to support AWS configurations.

  <auth>
    method aws_sigv4
    aws_service osis
    aws_region us-east-1
    aws_role_arn arn:aws:iam::123456789012:role/MyRole
  </auth>

The aws_role_arn will be optional. If not provided, the plugin will use the default credential chain. If provided, the plugin will assume the role supplied by aws_role_arn.

Describe alternatives you've considered

One alternative considered is a new OSIS output plugin. However, this seems unnecessary as the http plugin is sufficient. Also, there may be other use-cases of writing http to other AWS services.

Additional context

No response

[dlvenable]

@ashie , @daipom , We have a customer who is interested in using this change on the 1.16.x series. Could we release this in a 1.16.6 release? What would be the process for that?

[ashie]

Usually we aren't so positive to add a new feature in patch release like v1.16.6.
So we are currently planning to ship this feature as v1.17 series.
As in this milestone, next release will be v1.17.0 scheduled in the end of this month.
Doesn't it satisfy your customer's requirement?

[dlvenable]

@ashie , Thank you for clarifying. We use semver so I fully understand not taking the change into a patch.