Euphoria Audio, LLC

Recently I worked on a project where we moved a website to a load-balanced environment using a Cisco ACE module on a Catalyst switch. I wanted to setup monitoring so we could see how many people were connected to each server without having to login to the CLI or use Cisco Works as we didn’t have access to existing monitors or the money to buy our own. So I turned to the great Open Source tools, Cacti and Net-SNMP.

Cacti has a vibrant community of developers and users so I first searched for an ACE template that would be able to loop through the SNMP OIDs for my serverfarms and graph the connections. A user named TSY posted a template and script for parsing the ACE and when I loaded it into my Cacti install, it only found one out of six server farms and didn’t provide a graph in the format I was looking for. Thus I started poking deeper into SNMP and with the help of some folks in the Cisco Forums, figured out how to access the data that I needed. Here’s how I did it and hopefully it will help you.

 

-          Net-SNMP

-          MIB files for your device

 

First, for SNMP newbies like me, here’s a little information on how it works. The Simple Network Management Protocol is a standards-based system that allows a monitoring machine to connect to a network device and access its operating variables, configuration and to sometimes modify that configuration. To access SNMP information, you have to know the DNS or IP address of the device you’re connecting to and the community name that defines the rights you have when accessing the device. If your device uses SNMP version 3, you may also have a user name and password.

Every parameter for a specific device has a unique OID or object identifier that is a part of the SNMP tree. OIDs are referenced in a dotted decimal format, sort of like an IP address, and branch out like a tree. For example, one standard OID that almost all devices will define is .1.3.6.1.2.1.1.1. This is a description of the device in dotted decimal OID format. Another way to reference this object is SNMPv2-MIB::sysDescr, assuming the tool you use has the ISO standard MIBs in place. A Management Information Base (MIB) is a kind of map that links user-friendly names to the dotted decimal equivalent OID and defines what kind of objects and parameters the device supports.

 

So knowing that, let’s try to get the system description of our Cisco ACE module. I was provided the following information about the ACE:

 

IP: 192.168.1.1

Community: public

SNMP Version: 2

 

The public community is a default on most devices that provides read-only access to system parameters.

 

If you have Net-SNMP installed on your system, it will have a tool called “snmpget”. So open a command line and type:

snmpget -v 2c -c public 192.168.1.1 .1.3.6.1.2.1.1.1

The “–v” switch tells snmpget to use version 2c of the SNMP protocol. If you are using version 3, look at the snmpget help (type “snmpget –h”) for the proper format to enter your user name and password. The “-c community ip_address” switch tells snmpget what community and device to connect to. The final parameter is our OID and you can include or leave out the initial period.

If you run this command, you should get an error such as:

SNMPv2-MIB::sysDescr = No Such Instance currently exists at this OID

That’s because on the ACE and probably most boxes, its an indexed value which means there could be multiple system description entries. Indices are usually zero based, so if we append a 0 and try:

snmpget -v 2c -c public 192.168.1.1 .1.3.6.1.2.1.1.1.0

we get:

SNMPv2-MIB::sysDescr.0 = STRING: Application Control Engine Service Module

This is saying that we’ve received a value back from the device and it is of the STRING data type with a value of “Application Control Engine Service Module”. You may also have noticed that the numeric OID was translated into a nice text string. The Net-SNMP install includes a default set of standard MIB files under the \share\snmp\mibs directory. If you open the file SNMPv2-MIB.txt and search for “sysDescr” you’ll find the object definition, description and mapping stating that it is branch “1” of the “system” object. Snmpget was able to translate the numeric OID to text because this file provided the mapping. If you delete the MIB file from the directory, snmpget will not be able to translate. This brings up an important point: snmp calls always use the numeric OID format when communicating with a device, so if you provide a text OID, whatever tool you use must be able to translate it to a numeric OID or the call will fail.

 

So with that information, we next go in search of device-specific MIB files that we can use to translate between text and numeric OIDs. Cisco has a nice page where you find MIBs for almost all of their devices. If you’re device comes from another vendor, contact them for MIB files. If you lookup the ACE under Cisco’s security and VPN devices, you’ll find a long list of files to download. Some are already on your machine, such as SNMPv2-MIB and while there are a number of Cisco private MIBs, you won’t have to download them all. Because SNMP is a tree structure, each of the Cisco MIBs will depend on MIBs closer to the root of the tree. A tool we’ll be using later on will tell us what MIBs we’re missing, but for now there are a couple of ways to determine dependencies. The first way is to open one of the MIB files and look at the IMPORTS section at the top. If you see a list of object names, such as “InetAddressType” followed by a line such as “FROM SNMPv2-SMI” then you know that you’ll need a copy of the SNMPv2-SMI MIB in your directory. So download any MIB files that you need and save them to that directory.

 

Then next tool we’ll look at is called snmpwalk. This utility will scan a device for OIDs and return the names and values. We can use it’s default behavior to scan for standard device parameters such as names, interfaces and basic statistics. Use the following command:

snmpwalk -v 2c -c public 192.168.1.1

If you connect successfully, you should see a big list of OIDs starting with something like:

SNMPv2-MIB::sysDescr.0 = STRING: Application Control Engine Service Module
SNMPv2-MIB::sysObjectID.0 = OID: CISCO-PRODUCTS-MIB::ciscoACE10K9
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (1080447866) 125 days, 1:14:38.66

You may want to redirect the output to a file so you can browse the results easily. Usually just append “ > \path\tofile” to the end of the command line. While you’ll probably see a list of standard objects, you may not see the private, device-specific information you need. So the next step is to tell snmpwalk to include any MIB files you downloaded. The “-m” switch lets you include MIBs, either a list of a few MIBs you want to load, such as “-m ENTITY-MIB CISCO-PROCESS-MIB” or load everything in your MIB directory by using “-m +”. So try:

snmpwalk -v 2c -m + -c public 192.168.1.1

This may return a few extra device-specific OIDs, but you may still be missing what you’re looking for. In my case, I wanted a list of the load balancing server farms and the connections to each server in the farm. Unfortunately all devices are unique and you’ll have to do a lot of searching through documentation, user forums and even MIBs to find the object you’re looking for. I eventually found an entry called “cesServerFarmRserverTable” in the CISCO-ENHANCED-SLB-MIB that contained exactly what I needed, a table of server farms, real servers and the status of each server. In my ignorance, I figured that I could just use snmpget to grab a table of data and then parse it. But SNMP only allows a single value to be returned per call, so that wasn’t going to work. Besides, when I tried it, snmpget said the OID could not be found. Thankfully yjdabear and Joe Marcus Clarke helped me out in the Cisco forums, describing SNMP tables something similar to a spreadsheet where each “cell” of the table is a unique OID. Thus, when using snmpget to call cesServerFarmRserverTable, SNMP doesn’t have a value to return because it’s just a node on the branch that points to the start of a table of OIDs. Instead, we have to browse each of the values in the table using GET-NEXT queries that try to grab the next OID in a branch or object and we can use two tools to do this.

 

Snmptable is a tool that will iterate through a table of content and print the results in a nicely formatted, tabular output. Try something like:

snmptable -v 2c -m + -c public 192.168.1.1 CISCO-ENHANCED-SLB-MIB:: cesServerFarmRserverTable

This call is similar to our other snmp tool calls but now we’re using text OIDs instead of numeric. The format is to put the MIB file name, followed by two colons and append the object name at the end, thus “CISCO-ENHANCED-SLB-MIB::cesServerFarmRserverTable". The output should be a nice table with a header consisting of individual object names, such as “cesServerFarmRserverOperStatus” and values printed below. You could use this kind of output in a program if you’re good at string maniuplation but I needed to just dump a single OID into Cacti and grab a single value at a time. So let’s return to snmpwalk which will help us find individual OIDs.

snmpwalk -v 2c -m + -c public 192.168.1.1 CISCO-ENHANDED-SLB-MIB::cesServerFarmRserverTable

This will return a list of text OIDs, data type and the current value of each object in the table. Here’s an example of what might be returned

CISCO-ENHANCED-SLB-MIB::cesServerFarmRserverTotalConns.2."WebServerFarm"."Webserver01".443 = Counter64: 1570
CISCO-ENHANCED-SLB-MIB::cesServerFarmRserverTotalConns.2."WebServerFarm"."Webserver02".443 = Counter64: 1233

This tells me that on my ACE, there is a server farm called “WebServerFarm” with two real servers named “Webserver01” and “Webserver02”, with 1570 total connections to the first one since a counter reset, and 1233 on the second. Note the quotes around the server farm and real server names. This indicates ASCII encoding where each letter in the name will translate to a different dotted decimal number in the OID. If you want to plug this value into snmpget or another tool, you have to escape the quotation marks with a back-slash to notify the tool that you are providing ASCII values instead of MIB values. So if we want the total connections for Webserver02, (port 443 for SSL) we would plugin:

Snmpget -v 2c -m + -c public 192.168.1.1 CISCO-ENHANCED-SLB-MIB::cesServerFarmRserverTotalConns.2.\"WebServerFarm\"."\Webserver02\".443

If you need to get the full numeric OID to pass to another tool that doesn’t have access to your MIB files, you can use the snmptranslate tool:

snmptranslate -m + -On CISCO-ENHANCED-SLB-MIB::cesServerFarmRserverTotalConns.2."WebServerFarm"."Webserver02".443

will return:

.1.3.6.1.4.1.9.9.470.1.1.3.1.11.2.13.87.101.98.83.101.114.118.
101.114.70.97.114.109.11.87.101.98.115.101.114.118.101.114.48.50.443

So there you have it. You can now access a specific cell in a table using your SNMP tools.