I am looking for a way to do proactive monitoring of my network, what I am
specifically thinking about is receiving syslog msgs from the routers and
the backend engine would correlate certain msgs with output/data that i am
receiving through SSH/telnet sessions. What i am after is not exposed to
SNMP so i need to do it on my own.
I am sure there are many tools that can do parsing of syslog and acting
upon it but i wonder if there is something more flexible out there that I
can just re-use to do the above ? Please point me to known public or
home-grown scripts in use to achieve this.
My view on splunk,
+1 if you intend to have a human act on the reports, it does an excellent job of reducing huge amounts of audit data into the valuable bits.
-1 Seemed to be a pita to integrate with my scripting enviroment. I ended up kludging wget,awk and telnet together in a totally undignified way to make it reach out and act on something.
+2 Customizable ingestion/parsing, I'm feeding everything from linux audit data to weird proprietary serial output from a multiplexer into it.
-1 Proprietary database I would have liked to see an sql plugin for data storage, I would like the data in Mysql/Oracle but no-joy from splunk so that I can use other tools on it easily.
+1 Free demo. You can download an eval version that is rate limited and cripples itself after a fixed time.
-1 because The license costs are a bit high if your moving lots of data through it
Since you said you are willing to entertain home grown as well. I would
recommend looking at simple event correlator which is a perl script
designed to do the kind of thing you are talking about. I've used it in the
past to trigger bgp black holing and mail blacklists for example.
I wrote a script in Linux that watches for unauthorized login attempts and
adds the ip address to the blocked list in my firewall. You might want to
search sourceforge for a DYN Firewall and modify it from there.
Yes. Logstash shipper on your syslog proxy, forward to elasticsearch. Graylog2 is very cool. Tried kibana and didn't care for it.
Actually setting up graylog2 right now to do AD authentication.
So workflow is
End device -> syslog-ng vm -> graylog2/elasticsearch vm and other destinations (it corp security cloud for stuff they want to track, observium for anything matching my network gear hostname pattern, etc).
I have the middle syslog-ng box so I can have great control over where certain hosts ultimately send data. However that system can be used in any template, if I don't filter it just gets dumped to graylog.
Carlos Alcantar
Race Communications / Race Team Member
1325 Howard Ave. #604, Burlingame, CA. 94010
Phone: +1 415 376 3314 / carlos@race.com / http://www.race.com
So now Logstash and Elasticsearch will be even more integrated than before.
With Kibana on top of that, this seems like the ultimate log data "do
stuff" stack.
Actually I did the same. I use ipset lists (generally with a timeout) and take a regex or two and black / white list from a YAML file and just take (possibly multiple inputs) from piping tail -F. I also store addresses for future reference (by the script or otherwise).
This is quite maintainable as I can look at a list of people who have attacked the mail server and compare it to web attacks. Each process is a different type of service (different config file) and probably a different ipset. Due to ipset not actually doing anything until I make an iptables rule for it, I can run my script in a test mode (by default) and just see what happens (check it's logs and the ipset list it generates). I haven't found the need for this yet but I can use cymru to look up how big their net is (see geocidr for an example of how to do this in perl) and use a hash:net ipset type and cover a whole net.
Basically what I'm saying in doing it this way is quite expandable and isn't very hard and I can do tons of stuff that fail2ban can't (I don't think - it's been a while since I looked).
IDK enough about fail2ban to know whether I can assign a per proto or per
log type config (I assume I can). In which casethis does what my script
does and then some. I would probably dump out a ipset save on exit and try
to 'restore' on resume (which /I/ do) and I'm sure there's a way fail2ban
can check a store of addresses and check what network a host belongs to
(instead of just a host).
