Logs Bank

Mailman List Mirror (Read Only)
1

Hi,

If I may ask, is there any OSS that can serve as a log bank or log server, where it aggregate logs from different sources , and the logs can be accessed using the web from any location on the network and can do graphical presentations based on.the frequency or content os the logs.

Thank you

Joshua

2

You probably want spunk, but if you want to do aggregation in an OSS fashion, scribe or flume is the way to go.

-John

3

Do you mean like Splunk? http://www.splunk.com

4

Yes. Check out rsyslog and logstash.

5

Agree with Splunk, while not open source, is the most functional of these products. Be warned, while they offer a free license, once you start using it you'll be hooked, and their pricing beyond the free license is borderline extortionist.

6

Octopussy (8pussy.org) is another option as well. Natively ties into various network monitoring packages (Nagios, Zabbix) for alerting capabilities.

- Peter

7

http://www.8pussy.org/dokuwiki/doku.php -- free. open source.
http://logstash.net/ -- free. open source.
http://splunk.com (already mentioned, of course) -- pay to play. And
expensive, too.

There are far more out there.

8

Oh! And http://graylog2.org/ -- free, open source.
That's the last of the ones I can muster up.

9

To answer your question.

"yes"

However, with almost everything I can think of, there will be an element of
development required in order to achieve the results you're after. - at a
previous work place a few years ago we fed all event logs into hadoop, from
where we produced reports, initially just into excel files, and then later
created a webapp which produced near realtime stats/reports/graphs.

I've not looked recently at LogStash, or 8pussy, but primary concern would
be how well they deal with huge log volumes, how they scale when one server
is not big enough to hold all the logs any more, how they deal with many
users searching at the same time etc.

If you want to actually just get on with crunching logs, and drawing graphs
in a timely fashion, Splunk is proven, and works well up to big scale (we
were feeding almost 1TB/day of logs into it at my last company)...

Splunk is not cheap, but when considering the cost of development +
suppport if you went down the route of task of rolling something equivalent
in capabilities, its not bad value.

thanks

Andrew

10

<snip>

Do you mean OSS, or do you mean free?

</bugbear>

M