Archive for 2012
Let’s look at NHRP.
DMVPN, a staple in any large scale VPN network, or a cheap internet backup solution for a primary MPLS network. We all know the great thing about DMVPN is the fact it creates a single mGRE tunnel interface in which multiple connections can be made to/from (trust me if you ever have to manage a router with 500+ GRE tunnels you will quickly learn to love DMVPN) and the fact that mGREs can create tunnels automatically between spokes putting less load on the hub router is another major advantage. It also increases network performance due the fact in a typical hub/spoke topology if two spokes need to communicate between themselves the hub router must decrypt the traffic from spoke1 and then encrypt it again to send it to spoke2, just for spoke2 to decrypt the data again when it receives the data (assuming you are running IPSec over your tunnels). So you have some double encryption/decryption/encapsulation going on there, not very optimal at all.
The question is how does DMVPN know how to create these tunnels, and what mechanism is it that allows DMVPN to create all these tunnels, well that would be NHRP Next Hop Resolution Protocol. NHRP works off a server/client relationship, where the NHRP clients (let’s call them next hop clients/NHCs) register with their next hop server (NHS), it’s the responsibility of the NHS to track all of its NHCs this is done with registration request and reply packets. When two spokes need to communicate between themselves the spokes send over a resolution request packet and the NHS responds with a resolution reply packet, with that information the spokes need to build a tunnel directly between themselves.
So let’s take a quick look at how process occurs between some Cisco routers:
Here is an overview of 2 NHC’s registering with their NHS:
10.1.1.1 / 192.168.1.1 – NHS
10.1.2.1 / 192.168.1.2 – NHC
10.1.3.1 / 192.168.1.3 – NHC
Taking a closer look these NHRP packets they are comprised of a few different parts:
From top to bottom this packet starts off pretty normal, you see the normal L2/L3 information, and the GRE tunnel (since NHRP runs through a GRE tunnel) right after that we get to the NHRP portion of the packet. Now RFC 2332 defines these portions:
- Fixed Header – This portion is always the same, and just covers some basic information NHRP version, Ethertypes of the physical medium, address family information, and etc.
- Mandatory Part – This is where the NHRP packet type is defined (Registration request/reply, resolution request/reply, etc) and tunnel interface (Displayed as the protocol address) and physical interface (displayed as the NBMA address) IP addresses are contained.
- Client Information Entries CIE: Which contains specific information for clients such as MTU and hold down times.
NHRP Packets
Above is a quick screenshot of some of the NHRP packet types. You can the NHC at 10.1.3.1 attempts to register with 10.1.1.1 (the NHS) until the NHS sends its registration reply. Then you can see 10.1.3.1 sending a resolution request to the NHS, this represents the NHC at 10.1.3.1 attempting to reach the NHC at 10.1.2.1 after 10.1.3.1 receives its response from the NHS, 10.1.3.1 sends its it reply directly the NHC at 10.1.2.1. Clearing the NHRP cache on the Cisco router actually forces it to transmit a purge packets.
And now for a closer look:
The fixed header shown above, always includes the same information no matter which NHRP packet type is sent. Not too much to look at the NHRP version, address family, protocol type, and so forth.
The Mandatory Part of the header is what defines the NHRP packet type, this packet was the NHRP resolution request sent from the 10.1.3.1 (a NHC) to 10.1.1.1 (the NHS) when 192.168.1.3 attempted to ping 192.168.1.2 another NHC on the DMVPN network. Something I want to clarify here is the fact there are a lot of different IP addresses shown here. The Source NBMA address is the IP of the physical interface, the Source Protocol Address is the address of DMVPN tunnel on the NHC, and the Destination Protocol Address is the IP address of the DMVPN tunnel of the other NHC we are trying to reach. So NBMA Addresses are for physical interfaces and Protocol Addresses for the logical tunnels. After all that you’ll notice the Client Information Entry at the bottom. (Also keep in mind there are many other parts to the NHRP packet I did not show)
So there was a real quick and dirty run down on NHRP the protocol that makes DMVPN possible. It all really boils down to NHCs registering with the NHS, the NHS keeping all the NHC information in a cached local database, and the NHC’s asking for NHRP resolutions from the NHS when they want to spin up tunnels directly between NHCs. You can find a much more in depth discussion of NHRP in RFC 2332
My CCIE reading list!
A few of the CCIE books I have next to my bed!
Well, I’m still working towards my CCIE: Route/Switch, I figured I’d post an update about it, and why not focus on the books I’m using to prep for the CCIE: R/S Written exam:
- Routing TCP/IP Volume I 2nd Edition
- Routing TCP/IP Volume II
- Cisco LAN Switching
- CCIE: Routing & Switching Certification Guide
- Troubleshooting IP Routing Protocols
- QoS Certification Guide
- BGP Design & Implementation
- Developing IP Multicast Network
- MPLS Fundamentals
- Deploying IPv6 Networks
- IPv6 for the Enterprise
- Optimal Routing Design
- Cisco Firewalls
- CCNP: Security SECURE
I’m sure that will be enough to cover the objectives of the CCIE:R/S Written. I’ve actually already read through a good portion of the books mentioned, hopefully in the next month or two I will be ready to tackle the written exam!
Anyway back to reading and labbing, I’m working my way through the Multicast objectives, then I’m going to tackle IPv6, and review VRF/MP-BGP and then I’ll be ready to take on the exam!
I guess it’s time I should update my library page.
Securing NTP
Like all technologies in a network there are ways to secure it from being used maliciously, and NTP is no acception. We actually have a few different ways to secure the communication between our NTP peers and servers.
- Authentication
Yep, we can configure our NTP clients to authenticate to our NTP server/master and make sure we only accept the time from NTP servers that we have control over. This ensures our routers will not get the time from any unauthorized sources and prevents a malicious user from spoofing their own NTP server and changing the time on the network. Do keep in mind NTP authentication only supports a PSK type authentication method and will use an MD5 hash so were not doing anything over the top here. Configuration below:
NTP authentication on the NTP master.
NTP authentication on the client
Do keep in mind, I’ve built this configuration off my previous NTP lab post so not every NTP command is shown above.
Now, when I configured this I forgot the command ntp trusted-key 1 on the NTP server, but I did specify the key for the ntp server on the NTP client, so obviously my configuration did not work right off the bat. I ended up debugging NTP to troubleshoot, so I issued the debug ntp packet and debug ntp auth, which provided me the following:
NTP debug with wrong key on the master.
From there you can see we transmitted a packet to 172.31.1.1 (our NTP master) with authentication key 1, however the packet we received from 172.31.1.1 had an authentication key 0 (No authentication). So once I saw that I quickly realized where my problem was. I went back to the NTP master and issued the command ntp trusted-key 1 and then went back to client check the ntp status.
After reviewing the debug output again, both packets are using an NTP authentication key. Keep in mind the NTP key # does not have to match but the MD5 password within the keys has to match.
- Access Control Lists
Good old ACLs what can’t they do for us? Well NTP can be configured with an ntp access-group command, this command will limit the number NTP peers we perform transactions with, which can be useful and if you consider the IP Scheme during your design this won’t be hard to implement (Especially if you source NTP packets from loopback address). Here is my configuration:
NTP access group on the NTP Master
NTP access group on the NTP client
I’m a big fan of using named ACLs whenever possible, it just makes it easier to manage in my opinion. My NTP client is a Cisco Catalyst 2960 and NTP on this device only supports numbered ACLs, which is why I had to use a normal numbered ACL there and not a named ACL. The only slightly complex portion of NTP access-groups is the fact we can have different types of access-groups:
- Peer: Peer access-groups allow both request and control queries to be processed meaning the router will be allowed to update its time from the allowed peers.
- Query-only: This only allows control queries to be accepted, control queries don’t actually the effect the date/time so I’m going to skip this one. See RFC 1305 for addition information about this.
- Serve: Allows the router to reply to request as well as control queries.
- Serve-only: Does not allow control queries and only replies to NTP requests.
So from our example our NTP master is setup to serve-only our NTP clients, and our NTP master has full access to our NTP clients.
One last thing I want to touch on concerning NTP access-groups is the fact if your setup an ACL between 2 NTP servers, you must remember to allow the local NTP server address, in this case it is 127.127.1.1 this address might different depending on the router model and IOS version so it’s always a good idea to check it out.
The NTP Master Internal IP 127.127.1.1
Configure your router to watch for login attacks.
Even though we can configure SSH for secure management, this still does not stop a random person from connecting to your device in an attempt to guess the password. In some cases this could be a bored user who is just curious or in worst cases it could be a malicious user trying to perform a dictionary attack on your password. Something we can do is to configure our router to start blocking login request after so many failed attempts occur within a specific time frame. Now this isn’t going to secure your router indefinitely but it is a simple preventative measure they will slow down any real dictionary attacks and will most likely discourage any bored employees in hopes that they lose interest or remember they have other work to do.
So now let’s say we have an existing router with SSH enabled and we want to configure this login block feature, well let’s enter the following command:
Yep one command. This will block all logins for 120 seconds when 5 failed attempts occur within 60 seconds of each other. Below is a breakdown of the block-for command, and notice all the timers are in seconds.
Now when the router notices these failed attempts it enters a “quiet mode”. This quiet mode is what tells the router to block all further login attempts until the timer expires.
So now I hit my router with a bunch of invalid attempts and noticed the following error get logged:
This message states quiet mode is now on and also notice it actually logs the username I attempted to log in as (ss in this case), the source of the login attempt (192.168.1.109), the port I connected on (22 default for SSH), and the reason (failed login authentication)
From this point I cannot even connect to the router over port 22, putty flat out tells me the connection is refused:
The following error message will also get logged when I attempt to login to a router that is currently in quiet mode:
Stating our login attempt has been blocked by an ACL, but wait we didn’t configure an ACL on this router. Let’s check the VTY lines:
We didn’t configure this ACL but the login block feature did. The cool thing about this is when “quiet mode” is turned off the ACL dissappears:
Pretty cool right? We have now taught the router to protect itself.
While this is a nice feature to have configured the last thing you want is to find yourself locked out of your own equipment. So what we can do is configure the router to accept login attempts even when quiet mode is enabled, while that might sound counter-productive to this whole feature it will be something you definitely want to do.
And we do it this way:
The first thing I did was create an ACL (SSH_MGMT) that permits my management workstation (or management subnets) and then I issue login quiet-mode access-class command and reference the ACL I created. This tells the router to accept login requests from the hosts allowed within the quiet-mode ACL when quiet mode is on and the router is blocking login attempts from everyone else.
So now when quiet mode is initiated this custom ACL is applied to the line VTY lines:
You can verify your configuration with the show login command, this provides you a nice summary of what has been configured as well as the current state of the router:
A few things to keep in mind:
- While the quiet mode ACL will allow login attempts from hosts matched in the ACL, those very same hosts can trigger the router to enter quiet mode.
- If you already have an access-class assigned to the VTY lines and quiet mode kicks in, the quiet mode ACL will be applied during the block-for time. However when the timer expires the quiet mode ACL is removed and the original ACL back in place.
A quick look at WRR.
Weighted Round Robin our method of LAN QoS on the older switching platforms (2950’s/3550’s). Replaced by Shaped (or Shared) Round Robin on our newer platforms but WRR is still a big player out there so let’s take a quick look at it. Before we take a look at any configurations lets quickly take a look at how WRR functions from an overview perspective.
So if we start from top to bottom the first thing we notice is the fact we have 4 queues to work with. Remember these are egress/outbound queues so we are working with outgoing traffic. The switch first checks the fourth queue to see if to see if there are packets in the queue and then it transmits its configured amount of packets. Once the configured amount of packets have been transmitted for that queue it moves along to the next queue. It works this way to ensure queues do not get starved and the switch is not constantly transmitting one queue all the time. After it gets done with queue one it will then reset and start back at queue 4.
So from looking at the above flowchart you can probably guess a few things that need to get configured:
- What belongs in each queue
- How much from each should be transmitted
So in order to configure our queues we go into the interface we are looking to configure (Thank you Cisco for interface range command) and from there we configure the queues as followed:
So from the above we are configuring the following:
- Queue 1 will contain packets marked with CoS 0 & 1
- Queue 2 will contain packets marked with CoS 2 & 3
- Queue 3 will contain packets marked with CoS 4 & 5
- Queue 4 will contain packets marked with CoS 6 & 7
You don’t have to break it up so evenly you can put up 8 CoS markings in a single queue. After all QoS will be different for every network.
So now that we have our queue defined we need to define how many packets from each queue the switch will transmit before working on the next queue. This to, is also done from within interface configuration mode.
When we are configuring the individual queues they start from queue 1 to queue 4 (left to right), so in this example we configured the switch to transmit:
- 10 packets from queue 1
- 20 packets from queue 2
- 30 packets from queue 3
- 50 packets from queue 4
Now we can configure queue 4 as the priority, meaning the switch will check the priority queue and transmit all packets in the priority queue after it checks any other queues. So the switch will check queue 3, transmit the packets then check queue 4 since its got priority transmit those packets, then check queue 2 transmit those packets and then go back to queue 4. We can do that with the following command:
You just have to be careful with what traffic is placed in the priority and how many packets are sent from that priority queue to avoid starving other queues or causing unnecessary latency for other traffic.
You can verify your configuration by issuing the following command sh mls qos interface queueuing
This command will show you the QoS configuration on an interface by interface basis, I only included interface fa0/1 for this example. You see’ll that the expedite queue is enabled (queue 4) and the bandwidth per queue. Right after that we will see the CoS mapping to each of the queues.
Who’s got the time? NTP – Network Time Protocol
Anyone who has ever had to troubleshoot any type of issue on a Cisco router knows they have to look through log messages, and anyone who has ever done this knows what’s it’s like to look through a log where the time is off by minutes, hours, days, or even years. It tends to add on another layer of complexity in itself & typically you are going to be in this type of situation:
- Ok, I’ve got a problem let’s troubleshoot
- Lets look at the logs
- HHmm those dates/times are all wrong
- Ok let’s issue the sh clock command to see what time the router thinks it is
- Now how far off is the router’s internal clock from the real world current time
- Alright now what what time did the problem occur in the real world and what time did the problem occur in the router’s time zone
- There we go!
It’s not overly complex but it can be a little time consuming and it can cost you a good amount of time depending on the type of issue you are troubleshooting. In my mind this is completely unnecessary and should just be corrected. How do we correct this though? Well we configure our devices to check in with a time server and get the correct time using NTP (Network Time Protocol).
The first thing we will want to do is pick one (or more) of your routers to be the NTP master for the network, and the other routers will connect to this NTP master for the correct time, this will provide us with some type of hierarchical NTP design. Before we configure any clients to check in with the NTP master we decide on we’ll need to get the correct time on the master. To do this we can peer to some of the Stratum 1 public NTP servers. In terms of NTP and Stratum levels, Stratum 1 will be the most accurate time sources, these Stratum 1 servers are directly attached to some type of GPS, WWV, or CDMA device which are stratum 0 devices and are known to have the most accurate time. (Note: Cisco devices do not support the use of these devices if directly attached)
Here is our current topology:
Now let’s configure our router:
From here we configured a timezone for our router, I named the timezone EST I figured I’d keep it simple and set it to -5 which is where the Eastern time zone fits in the world. Then I set daylight saving time with the clock summer-time EST recurring command. After that I start configuring some NTP parameters I start by turning on NTP logging, I did this just for the sake of showing off some of the NTP logging messages. Then I configure the router to update it’s hardware clock with the time it receives from the server with the command ntp update-calendar, the hardware clock runs off an internal battery and will keep the correct time even when the router loses power. Now we configure our router to check in with ntp.you.org this is just one of the stratum 1 servers I pulled off the webpage mentioned earlier.
Now, shortly after I finish with those commands I see some of the NTP logging messages flash through the console:
The messages are simply telling us the clock was reset, and synchronized successfully with the server we configured. We can verify this by looking at the sh ntp status & sh ntp associations output.
You can see from the sh ntp status output, that our time is synchronized, our stratum level is 2, and when the last update was. From the sh ntp associations we can see we are peered with the NTP server 204.9.54.119 (Confirmed with the * before the IP address) and the NTP server we are peering with is using a CDMA device for it’s clock source. Also notice the 127.127.1.1 address, which is the local router.
So now in SW1 and SW2 I am going to issue the following commands:
This tells the other devices to check in with the router we configured as the NTP master, very similar to the commands we issued earlier on the master. We can verify this again by issuing the sh ntp status & sh ntp associations:
Notice there is less information, but it still tells us what we need to know. We see our clock is synchronized and to whom, as well as our stratum level. Our stratum level here is 3, since we got our time from a stratum 2 source and we are one more “time hop” away from the stratum 1 source.
So essentially our NTP design looks like this:
We can do the same thing with even more complex designs:
However NTP would be behave like this:
RFC 1305 goes into NTPv3 much more in depth if you want to read about it, also keep in mind NTP runs off UDP Port 123. (I just had to throw that in there somewhere before I ended this post)
CCIE or Null! 