In
Intro to VRF lite, we looked at how virtual routing and forwarding (VRF) instances can be employed to logically separate the layer three topologies of unrelated entities sharing a single physical infrastructure. VRFs work at layer three much like VLANs work at layer two, by assigning interfaces to a virtual domain isolated from other virtual domains at the same layer. This is a simple concept to grasp so long as each of the VRFs are maintained in isolation. However, things get a bit more complex when you need to route traffic from one VRF to another.
Consider the following topology, which might be found at a small business park or college campus.
The network pictured serves three customers: we'll call them customers Red, Green, and Blue. Each customer has its own Internet connection, housed in building one. Customer Green has employees in building two, customer Blue has employees in building three, and customer Red has employees in both buildings two and three. Additionally, VOIP service and connection to the PSTN (represented for now by a loopback interface) will be provided in the future by the network operator to customers Red and Green, but not Blue.
Buildings two and three are connected to building one via 802.1Q trunk links. All three customers rely on the same physical network infrastructure for connectivity, but the traffic from each customer must be isolated from the others for security reasons. The IP prefixes in use are as follows:
- Red: 172.16.0.0/16
- Green: 172.17.0.0/16
- Blue: 172.18.0.0/16
- VOIP Services: 192.168.99.0/24
The design question posed by this scenario is as follows: how can we provide local VOIP connectivity to some customers while keeping all of the customers isolated from one another?
Modifying the Multilayer Switches' SDM Template
Before we can implement VRFs on our multilayer switches (Catalyst 3550s, in this case), we have to modify how the internal ternary content-addressable memory (TCAM) is partitioned to store IP routes. Observe what happens if we attempt to enable VRF with the default SDM template:
S1(config)# ip routing
S1(config)# ip vrf Red
S1(config-vrf)#
%L3TCAM-3-SIZE_CONFLICT: VRF requires enabling extended routing
VRF routes require that an additional unique identifier, a route distinguisher, is prepended to each VRF route in the routing table (we'll cover this in more detail shortly). To accommodate for this, the TCAM must be repartitioned. (This step is only necessary when configuring VRFs on certain multilayer switches.)
S1(config)# sdm prefer extended-match
Changes to the running SDM preferences have been stored, but cannot take effect
until the next reload.
Use 'show sdm prefer' to see what SDM preference is currently active.
S1(config)# ^Z
S1# show sdm prefer
The current template is the default template.
The selected template optimizes the resources in
the switch to support this level of features for
8 routed interfaces and 1K VLANs.
number of unicast mac addresses: 5K
number of igmp groups: 1K
number of qos aces: 1K
number of security aces: 1K
number of unicast routes: 8K
number of multicast routes: 1K
The template stored for use after the next reload
is the default extended-match template.
The selected template optimizes the resources in
the switch to support this level of features for
8 routed interfaces and 1K VLANs.
number of unicast mac addresses: 5K
number of igmp groups: 1K
number of qos aces: 1K
number of security aces: 1K
number of unicast routes: 4K
number of multicast routes: 1K
Note that the supported number of unicast routes has been halved from 8K to 4K.
We need to apply this change to and reload all three switches before proceeding.
Creating VRFs
We need to create four VRFs on S1: one for each of the three customers, and one for the shared VOIP network. Each VRF must be assigned a unique route distinguisher. A route distinguisher is simply a unique number stored beside each route in the routing table to associate it with its VRF and maintain uniqueness should two or more VRFs have overlapping address space (for example, if all three of our customers were addressed out of the 10.0.0.0/8 network).
There are two formats in which we can define a route distinguisher: <ASN>:<number> or <IP address>:<number>, where
number is an arbitrary decimal number. Which method we choose is arbitrary. While the format of route distinguishers is an important design consideration in "real" VPNs, in VRF lite (i.e. VRFs without MPLS) it is only a locally-significant value. We'll use the <ASN>:<number> format with the private ASN 65000 (which will also be used for our BGP configuration later in the article) and number customers sequentially beginning from 1.
S1
ip routing
!
ip vrf Red
rd 65000:1
!
ip vrf Green
rd 65000:2
!
ip vrf Blue
rd 65000:3
!
ip vrf Shared
rd 65000:99
Next, we assign the physical and logical interfaces to VRFs and address them appropriately. We're using a loopback interface in place of a physical interface on S1 to emulate the shared VOIP network, however there is no effective difference with regard to VRF configuration. Note that interfaces must be assigned to a VRF before being addressed; assigning an interface to a VRF wipes any IP addresses already configured on that interface.
S1
vlan 16
vlan 17
vlan 18
!
interface Loopback99
description VOIP Services
ip vrf forwarding Shared
ip address 192.168.99.1 255.255.255.0
!
interface FastEthernet0/1
no switchport
ip vrf forwarding Red
ip address 172.16.1.2 255.255.255.252
!
interface FastEthernet0/3
no switchport
ip vrf forwarding Green
ip address 172.17.1.2 255.255.255.252
!
interface FastEthernet0/5
no switchport
ip vrf forwarding Blue
ip address 172.18.1.2 255.255.255.252
!
interface FastEthernet0/13
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface FastEthernet0/16
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface Vlan16
ip vrf forwarding Red
ip address 172.16.0.1 255.255.255.0
!
interface Vlan17
ip vrf forwarding Green
ip address 172.17.0.1 255.255.255.0
!
interface Vlan18
ip vrf forwarding Blue
ip address 172.18.0.1 255.255.255.0
The physical trunk interfaces F0/13 and F0/16 are not assigned to a VRF, because they are not routed interfaces. VRF configuration is only relevant at layer three.
We can use show ip vrf to examine the VRFs and their assigned interfaces:
S1# show ip vrf
Name Default RD Interfaces
Blue 65000:3 Fa0/5
Vl18
Green 65000:2 Fa0/3
Vl17
Red 65000:1 Fa0/1
Vl16
Shared 65000:99 Lo99
S1# show ip vrf interfaces
Interface IP-Address VRF Protocol
Fa0/5 172.18.1.2 Blue up
Vl18 172.18.0.1 Blue up
Fa0/3 172.17.1.2 Green up
Vl17 172.17.0.1 Green up
Fa0/1 172.16.1.2 Red up
Vl16 172.16.0.1 Red up
Lo99 192.168.99.1 Shared up
We only need to create two VRFs on each of the two other switches, as each connects only two customers, and assign the appropriate interfaces.
S2
ip routing
!
ip vrf Red
rd 65000:1
!
ip vrf Green
rd 65000:2
!
vlan 16
vlan 17
vlan 216
vlan 217
!
interface FastEthernet0/13
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface Vlan16
ip vrf forwarding Red
ip address 172.16.0.2 255.255.255.0
!
interface Vlan17
ip vrf forwarding Green
ip address 172.17.0.2 255.255.255.0
!
interface Vlan216
ip vrf forwarding Red
ip address 172.16.2.1 255.255.255.0
!
interface Vlan217
ip vrf forwarding Green
ip address 172.17.2.1 255.255.255.0
S3
ip routing
!
ip vrf Red
rd 65000:1
!
ip vrf Blue
rd 65000:3
!
vlan 16
vlan 18
vlan 316
vlan 318
!
interface FastEthernet0/13
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface Vlan16
ip vrf forwarding Red
ip address 172.16.0.3 255.255.255.0
!
interface Vlan18
ip vrf forwarding Blue
ip address 172.18.0.3 255.255.255.0
!
interface Vlan316
ip vrf forwarding Red
ip address 172.16.3.1 255.255.255.0
!
interface Vlan318
ip vrf forwarding Blue
ip address 172.18.3.1 255.255.255.0
Configuring OSPF
Next we'll configure OSPF to exchange routes among the three buildings. Three independent OSPF processes are to be run, one per customer VRF. OSPF configuration here is pretty straight forward, as we can simply place all interfaces in area 0 within each VRF.
S1
router ospf 1 vrf Red
network 0.0.0.0 255.255.255.255 area 0
!
router ospf 2 vrf Green
network 0.0.0.0 255.255.255.255 area 0
!
router ospf 3 vrf Blue
network 0.0.0.0 255.255.255.255 area 0
S2
router ospf 1 vrf Red
network 0.0.0.0 255.255.255.255 area 0
passive-interface Vlan216
!
router ospf 2 vrf Green
network 0.0.0.0 255.255.255.255 area 0
passive-interface Vlan217
!
S3
router ospf 1 vrf Red
network 0.0.0.0 255.255.255.255 area 0
passive-interface Vlan316
!
router ospf 3 vrf Blue
network 0.0.0.0 255.255.255.255 area 0
passive-interface Vlan318
We can verify that OSPF adjacencies have been appropriately established with show ip ospf neighbor. Note that this command is not VRF-specific.
S1# show ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface
172.18.3.1 1 FULL/DR 00:00:31 172.18.0.3 Vlan18
172.17.2.1 1 FULL/DR 00:00:32 172.17.0.2 Vlan17
172.16.2.1 1 FULL/BDR 00:00:32 172.16.0.2 Vlan16
172.16.3.1 1 FULL/DR 00:00:31 172.16.0.3 Vlan16
At this point, all customers in all buildings should have full connectivity within their respective VRFs.
S2# show ip route vrf Red
Routing Table: Red
...
172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C 172.16.0.0/24 is directly connected, Vlan16
O 172.16.1.0/30 [110/2] via 172.16.0.1, 00:00:31, Vlan16
C 172.16.2.0/24 is directly connected, Vlan216
O 172.16.3.0/24 [110/2] via 172.16.0.3, 00:00:31, Vlan16
S2# show ip route vrf Green
Routing Table: Green
...
172.17.0.0/16 is variably subnetted, 3 subnets, 2 masks
O 172.17.1.0/30 [110/2] via 172.17.0.1, 00:00:37, Vlan17
C 172.17.0.0/24 is directly connected, Vlan17
C 172.17.2.0/24 is directly connected, Vlan217
S3# show ip route vrf Red
Routing Table: Red
...
172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C 172.16.0.0/24 is directly connected, Vlan16
O 172.16.1.0/30 [110/2] via 172.16.0.1, 00:01:04, Vlan16
O 172.16.2.0/24 [110/2] via 172.16.0.2, 00:01:04, Vlan16
C 172.16.3.0/24 is directly connected, Vlan316
S3# show ip route vrf Blue
Routing Table: Blue
...
172.18.0.0/16 is variably subnetted, 3 subnets, 2 masks
C 172.18.3.0/24 is directly connected, Vlan318
C 172.18.0.0/24 is directly connected, Vlan18
O 172.18.1.0/30 [110/2] via 172.18.0.1, 00:01:05, Vlan18
Configuring Multiprotocol BGP
Multiprotocol BGP (MP-BGP or BGP-MP) will be used only to exchange routes between VRFs, and therefore needs to be configured only on S1. We'll use the private AS number 65000 for the BGP process.
Because all of our routed interfaces on S1 have been assigned to VRFs, we'll create a loopback interface not in a VRF to keep BGP from complaining about not being able to find a router ID. (In the real world, one might consider using the IP of a management interface as the BGP router ID.) A router ID can also be configured per BGP address family.
S1(config)# interface loopback0
S1(config-if)# ip address 192.0.2.1 255.255.255.255
To configure MP-BGP, we configure a separate address family within BGP for each VRF and simply redistribute all connected and OSPF-learned routes within that VRF.
S1
router bgp 65000
no synchronization
bgp log-neighbor-changes
no auto-summary
!
address-family ipv4 vrf Red
redistribute connected
redistribute ospf 1 vrf Red
no synchronization
exit-address-family
!
address-family ipv4 vrf Green
redistribute connected
redistribute ospf 2 vrf Green
no synchronization
exit-address-family
!
address-family ipv4 vrf Blue
redistribute connected
redistribute ospf 3 vrf Blue
no synchronization
exit-address-family
!
address-family ipv4 vrf Shared
redistribute connected
no synchronization
exit-address-family
We can verify that each BGP address family now maintains routes for its respective VRF. (To view the BGP routes for only a specific VRF, use the commandshow ip bgp vpnv4 vrf.)
S1# show ip bgp vpnv4 all
BGP table version is 47, local router ID is 192.0.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65000:1 (default for vrf Red)
*> 172.16.0.0/24 0.0.0.0 0 32768 ?
*> 172.16.1.0/30 0.0.0.0 0 32768 ?
*> 172.16.2.0/24 172.16.0.2 2 32768 ?
*> 172.16.3.0/24 172.16.0.3 2 32768 ?
Route Distinguisher: 65000:2 (default for vrf Green)
*> 172.17.0.0/24 0.0.0.0 0 32768 ?
*> 172.17.1.0/30 0.0.0.0 0 32768 ?
*> 172.17.2.0/24 172.17.0.2 2 32768 ?
Route Distinguisher: 65000:3 (default for vrf Blue)
*> 172.18.0.0/24 0.0.0.0 0 32768 ?
*> 172.18.1.0/30 0.0.0.0 0 32768 ?
*> 172.18.3.0/24 172.18.0.3 2 32768 ?
Route Distinguisher: 65000:99 (default for vrf Shared)
*> 192.168.99.0 0.0.0.0 0 32768 ?
Enabling VRF Route Import and Export
Our final task is to configure route import and export among the Red, Green, and Shared VRFs. VRF route import and export are configured as two explicit, independent functions. Under VRFs Red and Green, we need to:
- Export local routes
- Import routes from VRF Shared
Under VRF Shared:
- Export local routes
- Import routes from VRFs Red and Green
To do this, we configure each VRF with import and export route targets. A route target is similar in format to a route distinguisher, but serves a different purpose. Whereas any given VRF route in our scenario has exactly one route distinguisher to uniquely identify it, a route can have zero or more route targets as arbitrary identifiers for reference by import and export policies. This allows for much greater flexibility than if only the route distinguisher was used. Although used only locally in our example, route targets are communicated with MP-BGP neighbors as extended communities.
Route targets can be defined in the same formats as route distinguishers. Here each route target will reuse the same format and value of its VRF's route distinguisher, just to keep things simple. However, remember that there is no direct relation between the two. Additionally, although each VRF is exporting only one route-target in our example, multiple route-targets can be exported simultaneously (resulting in multiple communities attached to an MP-BGP route).
ip vrf Blue
rd 65000:3
!
ip vrf Green
rd 65000:2
route-target export 65000:2
route-target import 65000:99
!
ip vrf Red
rd 65000:1
route-target export 65000:1
route-target import 65000:99
!
ip vrf Shared
rd 65000:99
route-target export 65000:99
route-target import 65000:1
route-target import 65000:2
Now VRF Shared has routes from VRFs Red and Green, and VRFs Red and Green each have the route from VRF Shared (it may take a minute for the routes to appear in the table):
S1# show ip route vrf Shared
Routing Table: Shared
...
172.17.0.0/16 is variably subnetted, 2 subnets, 2 masks
B 172.17.1.0/30 is directly connected, 00:00:24, FastEthernet0/3
B 172.17.0.0/24 is directly connected, 00:00:24, Vlan17
172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
B 172.16.0.0/24 is directly connected, 00:00:24, Vlan16
B 172.16.1.0/30 is directly connected, 00:00:24, FastEthernet0/1
C 192.168.99.0/24 is directly connected, Loopback99
S1# show ip route vrf Red
Routing Table: Red
...
172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C 172.16.0.0/24 is directly connected, Vlan16
C 172.16.1.0/30 is directly connected, FastEthernet0/1
O 172.16.2.0/24 [110/2] via 172.16.0.2, 00:05:38, Vlan16
O 172.16.3.0/24 [110/2] via 172.16.0.3, 00:05:38, Vlan16
B 192.168.99.0/24 is directly connected, 00:00:44, Loopback99
S1# show ip route vrf Green
Routing Table: Green
...
172.17.0.0/16 is variably subnetted, 3 subnets, 2 masks
C 172.17.1.0/30 is directly connected, FastEthernet0/3
C 172.17.0.0/24 is directly connected, Vlan17
O 172.17.2.0/24 [110/2] via 172.17.0.2, 00:05:43, Vlan17
B 192.168.99.0/24 is directly connected, 00:00:49, Loopback99
VRF Blue, however, still contains only its own routes, as we did not configure import or export for this VRF:
S1# show ip route vrf Blue
Routing Table: Blue
...
172.18.0.0/16 is variably subnetted, 3 subnets, 2 masks
O 172.18.3.0/24 [110/2] via 172.18.0.3, 00:06:02, Vlan18
C 172.18.0.0/24 is directly connected, Vlan18
C 172.18.1.0/30 is directly connected, FastEthernet0/5
Lastly, we need to configure the VRF OSPF processes to redistribute BGP-learned routes to OSPF peers on S1:
S1(config)# router ospf 1 vrf Red
S1(config-router)# redistribute bgp 65000 subnets
S1(config-router)# router ospf 2 vrf Green
S1(config-router)# redistribute bgp 65000 subnets
These routes are now propagated via OSPF to S2 and S3 within their respective VRFs. On S3, we can verify that VLAN 316 (VRF Red) has access to the VOIP network, but VLAN 318 (VRF Blue) does not:
S3# show ip route vrf Red
Routing Table: Red
...
172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C 172.16.0.0/24 is directly connected, Vlan16
O 172.16.1.0/30 [110/2] via 172.16.0.1, 00:10:15, Vlan16
O 172.16.2.0/24 [110/2] via 172.16.0.2, 00:10:15, Vlan16
C 172.16.3.0/24 is directly connected, Vlan316
O E2 192.168.99.0/24 [110/1] via 172.16.0.1, 00:10:15, Vlan16
S3# show ip route vrf Blue
Routing Table: Blue
...
172.18.0.0/16 is variably subnetted, 3 subnets, 2 masks
C 172.18.3.0/24 is directly connected, Vlan318
C 172.18.0.0/24 is directly connected, Vlan18
O 172.18.1.0/30 [110/2] via 172.18.0.1, 00:25:17, Vlan18
S3# ping vrf Red 192.168.99.1 source vlan316
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.99.1, timeout is 2 seconds:
Packet sent with a source address of 172.16.3.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/4 ms
S3# ping vrf Blue 192.168.99.1 source vlan318
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.99.1, timeout is 2 seconds:
Packet sent with a source address of 172.18.3.1
.....
Success rate is 0 percent (0/5)
The final configurations of all three switches are made available here:
Enabling BPDU filtering on an interface is the same as disabling spanning tree on it and can result in spanning-tree loops.
