Use BGP maximum-paths eibgp with Caution

Under the BGP command process, "maximum-paths" enables BGP to perform load-sharing among multiple paths. There are 3 options:

1. maximum-paths: only allow equal-cost path load-sharing
2. maximum-paths ibgp: enable both equal-cost and unequal-cost path load-sharing for internal BGP
3. maximum-paths eibgp: enable both equal-cost and unequal-cost path load-sharing for both internal and external BGP

If you have 2 or more edge BGP routers on same AS and is using both iBGP and eBGP, use the "maximum-paths eibgp" with caution. The Cisco routers would assign iBGP routes with same administrative distance (i.e. 20) as eBGP. You could end up with a routing loop if you design eBGP to be preferred over iBGP paths.

Part 3: Configuring RIP routing on Windows Server 2008

This is following up on part 2 that described the concept of network high availability for single  non-clustered Windows server using dynamic IP routing. This final part is to outline implementation steps. On the Cisco routers, ensure that auto-summarization of RIP routing is disabled, as we are using /32 host addresses on the server loopback adapters.

Step 1: Install Windows loopback adapter

Add microsoft loopback adaptor using hardware wizard (hdwwiz.exe) with elevated admin rights. Assign a host IP address on it using /32 or 255.255.255.255 subnet mask. Add a host record statically on the DNS server using this IP address.

Step 2: Install “Routing and Remote Access Services” Server Role 

Open up Server Manager and add a server role named "Routing and Remote Access". Check on the following items:

Step 3: Add new routing protocol “RIP”

On the Server Manager console, right click on “Routing & Remote Access”, choose “Enable and Configure Routing ….” Click “Next”, select “Custom Configuration”. Select “LAN routing” and complete the wizard. Under “Routing and Remote Access > IPv4”, right click on “General” and choose “New Routing Protocol”. Select “RIPv2 for Internet Protocol”.

Step 4: Enable RIP on Network Adapter

Click on RIP and add new adapters (only public interfaces and loopback that you want users to access – never put in your management interface!). By default, Cisco routers would perform RIPv2 routing on multicast protocol, so select this option on the General tab.

For route optimization, it may be a good idea that your network routers to simply inject default routes into your server routing table. Click on "Advanced" tab and enable to following options:

You would also want to allow the server to advertise the host address (remember the loopback interface in step 1), which can be statically map to a host record on the DNS server.

Step 5: Securing RIP routing (Optional) (Important for better stability & security)

To prevent your server from listening to "rogue" RIP router/server that give bad routes, specify your trusted RIP neighbors on the Security tab (Right-click on RIP and choose Properties).

Furthermore, you probably won’t want your servers to become routers and carry network traffics unknowingly. You can further secure it by limiting it to advertise its own addresses only. For each interface added to RIP routing, choose “Properties”. Permit only the NIC address and the loopback address for outgoing routes.

Step 6: Verify IP routing

There are two ways to verify the IP routing table on your server, which should be populated by the network routers. You can view the IP routing table through the Routing And Remote Access console or through a command prompt. In the Routing And Remote Access console, expand the IP Routing node, right-click the Static Routes node, and then click Show IP Routing.

On the client network, do a continuous ping check (ping -t x.x.x.x) on the loopback address of the server. To simulate a NIC failure, disable one NIC and you may just see a couple of timeouts before resuming.

Part 2: Network Redundancy for Windows Server: Dynamic IP Routing

Besides NIC teaming in earlier part 1, another way to achieve network redundancy for Windows Server is to employ dynamic IP routing. Windows Servers support both static and dynamic IP routing. In dynamic routing, new routes are learned dynamically from the route peers (typically routers and L3 switches). Any changes in network topology (including failure of server NICs or switches) would trigger a routing update for all participating routing nodes. In other words, if there were any failure in any switches or NICs, traffic would have avoided the broken path and continue to route on unbroken links.

Examples of dynamic routing include Open Shortest Path First (OSPF) and Routing Information Protocol (RIP). Windows Server 2003 supports both OSPF and RIP while Windows Server 2008 only supports RIP. Besides achieving network redundancy, you need not configure routing statically, which is otherwise prone to network topology changes and configuration errors. Consider the following example:

Dual-homed ServerX is connected to 2 different switches on different subnets. RIPv2 routing is enabled on both NICs. By default, ServerX would dynamically register both DNS records with AD-integrated DNS servers. With DNS round-robin, network load-balancing can even be achieved among multiple clients. However, in the event of either NIC or switch failure, half of the clients would lose connection with ServerX, which is usually not desirable. Consider the next example:

MS Loopback adapter is installed. RIPv2 routing is enabled on two physical NICs, as well as the loopback adapter. Dynamic DNS update is also disabled. Single static A record is created with loopback IP address.  As dynamic routing supports equal-cost multi-path load-balancing, multiple clients reach ServerX loopback via both NICs. If either NIC or switch fails, dynamic routing would have to re-converge. After a short converging time, clients can regain access to ServerX on its loopback address via the sole remaining NIC.

In conclusion, with both dynamic routing and static DNS record on loopback adapter, both network load-balancing and network redundancy can be achieved. In my next post, I would summarize the steps to achieve this in part 3.

Faster OSPF Convergence using iSPF

According to this Cisco config guide, OSPF uses Dijkstra's SPF algorithm to compute the shortest path tree (SPT). During the computation of the SPT, the shortest path to each node is discovered. The topology tree is used to populate the routing table with routes to IP networks. When changes to a Type-1 or Type-2 link-state advertisement (LSA) occur in an area, the entire SPT is recomputed.

In many cases, the entire SPT need not be recomputed because most of the tree remains unchanged. Incremental SPF (iSPF) allows the system to recompute only the affected part of the tree. Recomputing only a portion of the tree rather than the entire tree results in faster OSPF convergence and saves CPU resources. Note that if the change to a Type-1 or Type-2 LSA occurs in the calculating router itself, then the full SPT is performed. Incremental SPF is scheduled in the same way as the full SPF. Routers enabled with incremental SPF and routers not enabled with incremental SPF can function in the same internetwork.

Given only pros and not cons, we should enable iSPF by default. iSPF can be easily enabled using ispf command under each router ospf process.

1. router ospf 1
2. ispf
3. !

To verify:

1. show ip ospf 1 | inc SPF
2. ........
3. Incremental-SPF enabled 
4. .......

Cisco Performance Routing (PfR)

For most medium-to-large networks, you would usually have more than 1 ISP to connect the enterprise network to all the remote sites. Traditional routing (such as OSPF, BGP etc) could route network traffic through a preferred path or load-balance among different paths. It can also automatically change path when a link is dead.

However, traditional routing alone can't detect and take actions on links that are "half-dead" e.g. congested, flappy link etc. To overcome it, Cisco introduces Performance Routing (PfR) or f.k.s Optimized Edge Routing (OER) and add intelligence bits into the classical IP routing. There are two components to PfR: a Master Controller (MC) and a Border Router (BR). In a typical installation, there is one MC and one or more BRs. In many smaller designs, you can have the MC and BR on the same physical router.

The MC is the brain of PfR. The MC receives performance data from the BRs, compares it against the configured policy, selects the best route for the data and sends commands to the BR to forward traffic to that path.

The BR is the foot soldier of PfR. It collects the performance data and sends it to the MC. The MC compares the data against the configured performance policy. If it is out of policy, the MC sends commands to the BR to change the data path. PfR can monitor link performance based on passive mode (using netflow) and active mode (IP SLA).

There is this good wiki article that explains the fundamental of PfR.

Route filtering using route tags

In enterprise routing, route-filtering is often used to prevent routing loops and sometimes for security reasons. Instead of solely relying on ip access-list and addresses, route filtering can also be performed by route tagging. In fact, this method is more scable for a larger network when managing access-lists can be a challenge over a large number of routers. Consider this corporate network (see below pic). The corporate has 3 remote sites with IP subnets of 1.1.1.0, 2.2.2.0 and 3.3.3.0 respectively. You have a corporate policy that states Network A should link to all 3 remote sites via ISP X. Network B should link to the first 2 remote sites via leased lines and the last remote site via ISP X only. Network A is peered with ISP X on eBGP. IGP between internal networks is OSPF and remote sites via leased line is RIP. To implement such routing policy using route-tag:

1. Router A


2. access-list 1 permit 1.1.1.0 255.255.255.0


3. access-list 1 permit 2.2.2.0 255.255.255.0


4. access-list 2 permit 3.3.3.0 255.255.255.0


5. !


6. route-map route-tag permit 10 ‌


7. match ip address 1 ‌


8. set tag 111 --tag the 1st two remote sites with 111


9. !


10. route-map route-tag permit 20 ‌


11. match ip address 2 ‌


12. set tag 222 -- tag the 3rd remote site with 222


13. !


14. route-map route-tag permit 30 -- without this, all other routes will be dropped


15. !


16. router ospf 1 ‌


17. redistribute bgp 65001 subnets route-map route-tag -- redistribute ISP routes into IGP


18. ...


19. ...


20. Router B


21. route-map tag-filter deny 10 ‌


22. match tag 111 -- filter off sites with tag 111


23. !


24. route-map tag-filter permit 20 ‌


25. match tag 222 --permit only sites with tag 222


26. !


27. router ospf 2 ‌


28. distribute-list route-map tag-filter in

To verify, perform the necessary "show ip route" commands on both router A and B to ensure the route entries are in order. Do note that tagging does not work with BGP. The alternative in BGP is to use community string in AA:NN format (e.g. 100:300). For the adverting routers (typically on customer edge), use "set community" in place of "set tag" in the route-map statement. For the recieving routers (typically on provider edge), use "ip community-list" to describe the community string and "match community". For further example on using BGP community, see this Cisco example.