Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Jacob Bogers
@Jacob_Bogers_twitter
hellloo
Attila Szakacs
@alltilla
@Jacob_Bogers_twitter Hi :)
Gábor Nagy
@gaborznagy
Hi
László Várady
@MrAnno
@gaborznagy Hi, what's up?
ccxcz
@ccxcz
Can I configure message dropping for a slow output? (specifically /dev/tty...)
Fabien Wernli
@faxm0dem
ccxcz: you can play with the queue size
László Várady
@MrAnno
@ccxcz yeah, if you don't use the flags(flow-control) flag in your log path, messages will be dropped after the destination's internal queue is full (you can change this queue size with log-fifo-size()).
However, using this on a file() destination may be tricky, because we have flow-control enabled implicitly.
Jacob Bogers
@Jacob_Bogers_twitter
I am at a loss, I want to replace on of the software components at work (node module uses posix bindings to "openlog", "syslog","setlogmask") with another package at seems to be more low level
we do use syslog-ng as a backend
the protocols are RFC3164, RFC5424, LEEF ,CEF ​​​​​​​
I am trying to find a alternative fo "openlog" without the need of reading all those RFC papers, maybe in the end i need to, but I hope i dont
Peter Czanik
@czanik
this is a test: balabit/syslog-ng moved to syslog-ng/syslog-ng, so I wonder if it still works...
Ah, OK. Activity on the right hand side seems to be up-to-date, so lets hope, that everything else works as well :)
László Várady
@MrAnno
we'll soon ask the Gitter support team to rename this room without removing the chat history (there is no rename option on the UI)
Gábor Nagy
@gaborznagy
Hi @Jacob_Bogers_twitter .
Just my two cents: just took a quick glimpse on the library "SyslogPro", and I don't think you can find a one-one mapping to POSIX APIs, e.g. "openlog" .
They are basically different.
While openlog/syslog is an API to the system logger, SyslogPro library seems like a network-based log forward library.
I guess you need to configure SyslogPro to send the logs to syslog-ng through the network instead of the system logger.
Jacob Bogers
@Jacob_Bogers_twitter
hi
yeah we have syslog-ng listening on 514
(udp)
it seems there is no other way as to learn syslog-ng configuration
or ask the devops to spend time on it to tell us how to use tha pi
thanks Gabor Nagy, can you tell me how long it would take (ballpark estimate) for someone to learn syslog-ng up to a good level?
thank you
Gábor Nagy
@gaborznagy
@Jacob_Bogers_twitter I don't think you need to adjust many things, if syslog-ng on your backend is configured to listen on UDP port 514 (as you stated on cyamato/SyslogPro#3), then you inly need to replace syslog() calls to SyslogPro's send methods, while where you used openlog(), you need to create the connection.
Jacob Bogers
@Jacob_Bogers_twitter
@gaborznagy the WHOLE POINT of my question is HOW you would replace it, there are (counting) 5 different protocols/message formats used, in "syslogpro" js lib , you already stated (quote) "They are basically different."
GUess i am reading the syslog-ng manual and RFC formats,
Stijn Vanorbeek
@StijnVanorbeek_twitter

Hi All, New here to the channel, and also new to syslog-ng.

I'm trying to accomplish the following, but not sure if it is possible:

I successfully have syslog-ng send logs from my embedded devices to a TCP socket, using network()
I see my logs coming in on a socket on the same machine running nc -l -k 8080

Now, I would like to post my logs to an endpoint I'm running in the cloud (using AWS API Gateway).
What would be a good design-pattern. syslog-ng doesn't seem to have a (rest) endpoint type of destination function.

Stijn Vanorbeek
@StijnVanorbeek_twitter
Any suggestions?
Attila Szakacs
@alltilla
Stijn Vanorbeek
@StijnVanorbeek_twitter
@alltilla Thxs! I wasn't yet able to make it work with http(). Ended up using program() to call a Python script to handle the http posts. Do you know if the native http() destination supports batch posting? As I'm working on embedded devices (over the internet) it seems a bit awkward to POST to an end-point for every log line..
Attila Szakacs
@alltilla
Yes, it supports! :) The main options are: body(), body-prefix(), body-suffix(), delimiter() and batch-lines().

The payload will look like:

body-prefix
body
delimiter
body
delimiter
body
body-suffix

where there are batch-lines() number of body

and each body is one log message, formatted as it is set in the config
The output will be one bigger POST request :)
Stijn Vanorbeek
@StijnVanorbeek_twitter
Thxs for pointing this out. I think I was looking at older documentation! The docs are very good, just a lot!
Attila Szakacs
@alltilla
@StijnVanorbeek_twitter You are welcome! :)
Stijn Vanorbeek
@StijnVanorbeek_twitter
@alltilla : I must be missing something. I keep on getting a Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng/3.20', module='http' when using http as destination.
I tried to include the module in the config file with @module http
I installed from Binary on Ubuntu, but can't seem to find a package like syslog-ng-http
Going to try build from source..
Any pointers?
Attila Szakacs
@alltilla
@StijnVanorbeek_twitter The package you are looking for is syslog-ng-mod-http. You can find the deb packages here: https://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/
GitHubKilla
@GitHubKilla
I am writing my own template which does its own file operations. When Syslog-NG is shut down (e.g., by a sigterm), the template sometimes gets interrupted during file operations. That's bad. :-) Is there an elegant way to either notify a template about the shutdown or temporarily delay shutdown?
László Szemere
@szemere
Hello @GitHubKilla , could you please elaborate on your use-case a little bit more. There is a mechanism inside syslog-ng to get notification about the shutdown, but unfortunately it has nothing to do with templates. Template functions are simply designed in a different way. I have other suggestions for you, but first I would like to hear your intention before taking this whole topic into a wrong direction.
GitHubKilla
@GitHubKilla
Thanks for your reply @szemere ! Here is some more information regarding the use case:
The template I am writing does some log data reformatting and massaging before data is then written to a destination.
Also, the template has a state which is used, e.g., to compute statistics and extract information from the data passing through the template.
Currently, I write this state to my own file using my own routines (like open(), write(), close()). Later, I use external software to analyze data in that file.
That works quite well, but if there is a high rate of incoming log data, and at the same time syslog-ng receives a SIGTERM, it has happened that my template routines were interrupted while writing into that file. This results in inconsistencies in that file.
So, for each log message, the template does some reformatting and passes on the result. In addition, the template updates its internal state and writes this state to my own file.
László Szemere
@szemere
Thank You!
My suggestion is to use "python-parser"-s, but first I would like to test your problem. The template functions should not be interrupted either.