CCIE or Null!

If there is anything I find more enjoyable then doing some type of network design or writing on whiteboard, it’s thinking about  network management and creating some new alert or poller that let’s me know when something changes that shouldn’t. It would seem over the last few years Data Center technologies have really become popular:

• Leaf/Spine models – A scalable growth model
• Virtual Switches – VMWare NSX / Cisco Nexus 1000v, ASA1000v, VSG, etc
• vPC – Active/Active forwarding to trick spanning-tree
• OTV – another way to extend Layer-2 connectivity across physical locations

And one the thoughts I have in the back of my mind:

Yea, this stuff is great but I can’t monitor this stuff natively in my management systems. How I am going to know when something goes wrong?

Now, from here I am going to focus on Overlay Transport Virtualization (OTV)

Luckily, Cisco publishes a lot of their MIB structures. For example we can find the OTV MIB File here, this gives us a lot of information and tells us what information is exposed via SNMP.

The first thing, this tells us is where to find the OTV information

1.3.6.1.4.1.9.9.810

One thing I want to point out is the fact, I was unable this find this using a MIB Browser, “810” just was not listed. In order to find this I had download all the SNMP MIB Tables from one my OTV VDC’s. At this point I am not sure if it was an issue with the SNMP MIB Browser I was using or if it is an NX-OS thing.

Now, we will want to take a deeper look at what CISCO-OTV-MIB can tell us. We can find this info here.

That might look a little intimidating and it may look like a lot of reading but luckily it is to the point and tells us the following:

• What OID’s (Object Identifiers) are available
• What each OIDs is with a proper description
• The OID Syntax or variable type (IE: Gauge, Interger, String, etc)
• What possible values the OIDs can return and the meaning of those returned values.

For example:

The above snippet from the CISCO-OTV-MIB structures tells us, cotvOverlayVpnState is an Interger and will return a value of 0, 1, or 2 where:

• 0 = Other
• 1 = Down
• 2 = Up

So this tells us, if this specific OID returns a value of 1 we have problem our OTV Overlay interface is down.

We can even go another step further and poll an additional OID to identify the reason the Overlay interface.

cotvOverlayVpnDownReason is another Integer, they will return a value of 0-19 depending on the specific reason regarding why the Overlay is down , the MIB file goes further and describes each of the down reasons. (Small snippet below)

Now, by polling for these specific OID’s in your Network Management Server (NMS) you can much easily get the status of your OTV Overlay interface, and if the Overlay does happen to be down you can also find out why just as quick.

In the event you have multiple OTV Overlays running, you can poll this information on an overlay by overlay basis. If we go up a level in the MIB file, we’ll notice it contains a sequence for each OTV Overlay:

And within each sequence is where can find a lot of the information we probably need:

• The name of the Overlay
• The state / status of the Overlay
• The reason the Overlay is down
• Much more

Now, that we know how to monitor the Overlay interface that really solves half the problem in my mind. We may also want to monitor OTV Adjacencies, or an AED Status. Information that is also contained in the MIB file to poll the additional data.

Another important thing that may be worth monitoring depending on your deployment is multicast. I’ll cover multicast monitoring in a future post, but if your running OTV in multicast mode and you start losing PIM neighbors or mroutes start getting pruned you may definitely have some issues going on.