Auto flush on multiline codec not working

Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.

For all general issues, please provide the following details for fast resolution:

• Version: 7.6.2
• Operating System: Ubuntu 18.04.3 LTS
• Config File (if you have sensitive info, please remove it):

input {
    tcp {
        port => 10030
        codec => multiline {
                     pattern => "new_line"
                     negate => true
                     what => "previous"
                     auto_flush_interval => 1
                 }
    }
}
filter {

}
output {
    stdout {}
}

• Steps to Reproduce:
  • Write any line to tcp port. For example via nc localhost 10030 and line "some_test_line"
  • Nothing appear in stdout - auto flush is not working

[mj84]

I can confirm this with 7.10.0 using a tcp input as well.
4 years ago, someone mentioned that only file and beats inputs were supported with the multiline codec:
https://discuss.elastic.co/t/solved-auto-flush-interval-not-flushing-on-udp-input/60387

The current documentations of the tcp input or multiline codec don't mention such a limitation.

This is rather a limitation of the multiline codec and not the tcp input, and there is also an open issue for that:
logstash-plugins/logstash-codec-multiline#57

There also was a pull request work in progress regarding auto_flush, but it is closed and won't be merged for now:
logstash-plugins/logstash-codec-multiline#63

This issue provides a good roadmap in the last post which explains that an "input_type" option will be introduced for csv codec and later for the multiline codec:
elastic/logstash#11885

Sorry for pinging all the issues and PRs.
I though it was a good idea to summarize the state of auto_flush when using multiline with tcp since I ran into this behaviour today.
Google brought me to this issue, hopefully this explains the situation to others facing the same challenge.