'lo all,
Is anybody out there aware of a piece of software that can take data from an arbitrary source and then present it, using a MIB or set of OIDs of your choosing, as an SNMP-interrogatable device?
We have some CPE that supports SNMP, but considers it to be a mutually-exclusive "remote management" protocol such that if you use another supported method for deployment and provisioning (e.g., TR-069), you cannot have both that AND SNMP enabled simultaneously. It's one or the other.
We currently monitor and graph some device stats for these CPE with Cacti, but we want to be able to provision using a TR-069 ACS. The ACS can collect some of the same data we are graphing right now, but cannot present it in a fashion that is nearly as useful as the way Cacti/RRDtool does (not to mention the staff is already used to navigating Cacti). We know what SQL database table the stats are being stored in by the ACS, though, so my thought was that there must be some way that we can have a host respond to SNMP gets and then have it turn around and collect the value to be returned from a database. Basically, an ODBC -> SNMP proxy. We'd then point Cacti at that IP instead of the individual CPEs. But I can't seem to find anything like this.
Thanks,
-- Nathan
It is fairly easy to extend the snmpd on a Linux host to retrieve data from
non-SNMP sources... For example:
http://www.adventuresinoss.com/2009/09/30/the-many-uses-of-net-snmp/
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/sect-System_Monitoring_Tools-Net-SNMP-Extending.html
Anything you can retrieve with a shell script can be turned into a
1-minute-interval cron job that outputs a text file with a number in it,
then the SNMP 'extend' parameter in the snmpd.conf can be used to 'cat' the
file.
For example something from a solar charge controller that only speaks RS485
over an RS485-to-RS232 (or USB) bridge. Or data retrieved via curl and sed.
I use this to graph WSDOT traffic driving times patterns and flows and feed
them into a charting system.
For smaller setups you can have all the shell scripts for data acquisition
on the same host that runs the snmpd.
Many, many years ago, I wrote a fairly minimal SNMP agent in Ruby
(http://theshed.hezmatt.org/rubysnmpd/index.html), which was
explicitly designed to be easily extensible (it didn't do anything useful by
itself). If your team's into Ruby, that might be a possibility.
Otherwise, net-snmp does come with some fairly decent extensibility
capabilities these days. It has a variety of mechanisms by which you can
feed extra data into it, and in general I'd recommend going down that route.
- Matt
'lo all,
Is anybody out there aware of a piece of software that can take data from an arbitrary source and then present it, using a MIB or set of OIDs of your choosing, as an SNMP-interrogatable device?
We have some CPE that supports SNMP, but considers it to be a mutually-exclusive "remote management" protocol such that if you use another supported method for deployment and provisioning (e.g., TR-069), you cannot have both that AND SNMP enabled simultaneously. It's one or the other.
We currently monitor and graph some device stats for these CPE with Cacti, but we want to be able to provision using a TR-069 ACS. The ACS can collect some of the same data we are graphing right now, but cannot present it in a fashion that is nearly as useful as the way Cacti/RRDtool does (not to mention the staff is already used to navigating Cacti). We know what SQL database table the stats are being stored in by the ACS, though, so my thought was that there must be some way that we can have a host respond to SNMP gets and then have it turn around and collect the value to be returned from a database. Basically, an ODBC -> SNMP proxy. We'd then point Cacti at that IP instead of the individual CPEs. But I can't seem to find anything like this.
I would move away from this CPE vendor. Your solution has merit in the short term, but monitoring through the ACS is pointlessly putting more load on a server that already has it's own responsibilities. You can't scale out with this. Well, not without deploying more ACS servers.. which are a bit more heavyweight than SNMP pollers.
As mentioned already, net-snmp can do this easily enough. The biggest problem you'll face is figuring out how you want to name OIDs to match up to each CPE and the elements you're graphing. .... you might be better off pulling the data out of the database via SQL queries to a remote host and proxying the data there. Or possibly have cacti run the SQL query directly. It looks like they have many general (non SNMP) templates that you could use to base it on.
http://docs.cacti.net/templates
http://forums.cacti.net/viewtopic.php?f=12&t=15067
Thanks,
-- Nathan
Thanks,
Robert
Hey, thanks guys! I had never really looked that deeply into Net-SNMP and had only ever installed it either to use as a client (snmpget/snmpwalk) or a basic agent w/ standard MIBs for the host it's running on, so I was unaware of its extensibility. And it even looks like it ships with a Perl module. That sounds like a perfect solution; thanks for pointing me in the right direction.
-- Nathan
I would move away from this CPE vendor.
I'm not thrilled with it either, but at this moment in time, this is easier said than done for many unfortunately good and unavoidable reasons. We will see how the future plays out, though.
[...] Or possibly have cacti run the
SQL query directly. It looks like they have many general (non SNMP)
templates that you could use to base it on.
Another interesting suggestion & possibility. Thanks.
-- Nathan
Hi Nathan,
You should probably write a cacti script to ingest the data instead of this SNMP proxy thing. Writing scripts to ingest data into cacti is simple, you just need to output the values you want in key: value format and then do some clicking in cacti. There are good docs for how to do this.
-- emj
Thinking of the recent conversation on ntp daemon precision and reliability
here on nanog, reminded me of:
https://www.pitt-pladdy.com/blog/_20140826-094101_0100_NTP_Monitoring_on_Cacti_over_SNMP/
There's a tiny shell script linked there on github which does nothing more
than run "ntpq -p" on your ntpd and parse the results of the 9th and 10th
columns (jitter an offset) into milliseconds. On a typical debian system
you'll need to install the 'bc' package for command line calculations.
Of course no need to use Cacti, though the XML template is ready to use.
You can use the newly exposed OIDs via SNMP with opennms or some other
charting/graphing system, or even something totally non RRA/RRDtool based
such as collectd and a time series database for long term storage of data
on a 60-second poller interval.
Another thing you can do with 'extend' is monitor an openvpn daemon.
There's no built in SNMP support in openvpn but it can be configured to
listen for a management CLI on localhost. Run tiny shell, perl or python
scripts that do something as simple as parse the openvpn-status.log, spit
out the list of currently active clients, pipe that into a one line script
with sed and "wc -l", feed the integer into a SNMP charting/monitoring
system.
Eric Kuhnke <eric.kuhnke@gmail.com> writes:
The Many Uses of Net-SNMP | Adventures in Open Source
Ha! I've never seen that article, thanks for pointing it out.
Note that the performance of Net-SNMP's extensibility mechanisms should
way into the decision. The fastest backend needs to be written in C,
and embedded perl is an easy second. Beyond that, pass_persist is
somewhere in the middle and pass/extend/other execs are the slowest
because of the need to exec a command for every incoming request which
is expensive. Great for bootstrapping and testing, but in the long run
look to the better coding solutions.
Tutorials for most of these exist:
Tutorials - Net-SNMP Wiki
[as a point of history: Net-SNMP has always been very extensible since
it was started based on my need to add extensibility to an agent way
back in 1995-ish in order to monitor some special cases on a map with HP
OV (as it was known back then)]
This doesn't scale on a large cacti installation with hundreds of hosts and
60-second poller intervals. Cacti data input method scripts spawn a new php
worker for each data acquisition target (they do NOT use the 'spine' SNMP
poller).
Exposing the data via SNMP on the host to be monitored distributes the CPU
load individually onto each host (example: an 'extend' script in the
snmpd.conf which runs "curl http://localhost/server-status" for apache2
status and parses the results) rather than centralizing it on the cacti
host.
This allows cacti or opennms or anything else to poll the hosts to be
monitored via something that scales better than php script workers, using
the 'spine' SNMP data acquisition method and the equivalent in other snmp
polling platforms.
Exposing the data via SNMP on the host to be monitored distributes the CPU
load individually onto each host
So much this. Most importantly, it removes the fork/exec overhead from
the monitoring server.
This allows cacti or opennms or anything else to poll the hosts to be
monitored via something that scales better than php script workers, using
the 'spine' SNMP data acquisition method and the equivalent in other snmp
polling platforms.
Point of fact for OpenNMS users: all our collectors are in-process, so
in cases where SNMP is impractical you can use the XML/JSON or JDBC or
WS-Management collector and still achieve great scale with a single server.
But SNMP still scales and interoperates like nothing else.
-jeff