I'm not an elastic person, but I can shed some light on this: You're holding it wrong. It's not a bug. You have multiple config files in one directory - if you do that, all those files are combined to one, that means that each event gets handed to each of your individual outputs - multiplying the message. See https://www.elastic.co/guide/en/logstash/current/command-lin...
Feel free to hop on the IRC if you have further questions, there's usually somebody qualified to answer.
I appreciate the help, though isn't the point of `/etc//conf.d` directories generally that you have multiple config files? This is a common idiom that other packages handle correctly (differently?).
I have hopped on the logstash IRC at times to ask about some of this, though I guess not this exact item. In fact, there's a different (well-known issue) that the init script for logstash has the config path hard-coded:
https://botbot.me/freenode/logstash/2015-11-17/?msg=54338903...
There's also the problem that logstash (and forwarder) doesn't seem to let me do anything useful with the file names. I could work around that, sure, but it would be nice to have meaningful file names (not the "ls." thing that LS uses). Syslog-ng, for comparison, gives you a lot of control of that.
> isn't the point of `/etc//conf.d` directories generally that you have multiple config files?
Yes certainly. It's totally fine to place multiple config files there, I do as well. I split up my configs in the various outputs, inputs etc. It's just that logstash combines them to a single pipeline and does not run a pipeline per config file. Nginx doesn't run a webserver per config file either :).
It's certainly something that's unexpected and could be much better documented, but alas, I'm just a user :)
(and I do agree, your issue could have been handled much better, especially since it's not actually a bug)
Awesome, thanks for the clarification. This might help (if I ever go back to using logstash at this point!).
Yeah, my point was more that they accepted my issue, but there's no action and there are more than 600+ other open issues. Seems Elastic is too busy branding and pushing breaking changes to their APIs.
I do sincerely appreciate your clarifications and comments on this one, though.
Feel free to hop on the IRC if you have further questions, there's usually somebody qualified to answer.