I have a Magento2 instance for which I collect the logs using prospectors in filebeat-5.2.1. This is happening on a docker container based on ubuntu xenial (16.04).
The Magento2 report is a valid json file with a single line, but contain no newline at the end of the string which results in logs not being sent - https://www.elastic.co/guide/en/beats/filebeat/current/faq.html#newline-character-required-eof
Tried with single/multi line, tried changing the close_timeout and a few other suggestions found online but nothing works so far.
Could you provide some suggestions on how to approach it?
Problem is, lines are split on newline. The json parsing and multiline filters are applied thereafter.
Given JSON is structured, I would like to beats to not requiring the newline, but just stream through the JSON document, parsing object after object after object. Without requiring newlines, but also supporting pretty printed JSON. This is unfortunately not possible as of now. But we’re well aware of this use-case. For this and other common use-cases I would like the filebeat readers to be somewhat chainable, so we can plugin decompression, json, xml, multiline, newline splitting, content splitting on custom pattern in any order required.
Unfortunately I didn’t find a ticket, but feel free to open an enhancement request.
Thanks for your answer. I just opened an enhancement request: https://github.com/elastic/beats/issues/6132
This topic was automatically closed after 21 days. New replies are no longer allowed.