APRICOT II

Internet Exchange Point

A Case Study and Commentary

(Draft Version 1.0)

By Praveen Akarraju, Cisco Systems

    •

    • Introduction

    The APRICOT Internet eXchange Point (IXP) was conceived as a way to demonstrate how Internet service providers can work with each other while demonstrating some of the basic principles of how to configure an IXP.

    Many thanks to the group of people who put in extra time and effort in the early morning hours to make this a success.

     

    IXP Configuration Case Study :

    Network Description :

    The APRICOT IXP network consists of 7 routers (A - K) each in its own AS connected via an FDDI ring. Router K connects to the Hong Kong Internet Exchange (HKIX) AS4635 and 2 other commercial ISP’s. The traffic from the APRICOT IXP is routed out to the Internet via. A T-1 circuit from Globalone (AS4000).

    The routing configuration is setup such that the central router, Router K and an edge router, Router A carry full routes while the rest of the routers carry local routes and routes from ISP’s directly connected to them. One such setup is Router B which is connected to the commercial ISP HongKong Star AS4515. Most of the 7 routers are grouped together in a Peer-Group which optimizes and allows for easy enforcement of common policies. There are also inbound and outbound policy rules applied to routing updates using Route Maps, Filter Lists and Distribute Lists. Route Flap Dampening is enabled on Router B such that an external flap doesn’t adversely affect the stability of routers in the IXP. Router K also has Netflow switching enabled which allows for very good traffic analysis.

    One interesting case is that of AS4515 which is Multihomed into the IXP. It connects directly to AS7712 (Router B) and via. AS4635, the HKIX to AS7722 (Router K). This case study contains configurations from 2 typical routers and some "show" outputs which give information on the state of the protocols, interfaces etc.

    Some of the router commands displayed are :

    - write term -> Displays the running configurations

    - sh ip bgp summary -> Displays a list of BGP neighbors

    - sh ip bgp neighbor -> Specific information on a neighbor

    - sh ip bgp <network> -> Displays the BGP table entry for the network

    - sh ip bgp regexp _4515$ -> BGP table entries for routes originating

    in AS 4515

    - sh ip route <network> -> Displays routing table entries for the network

    - traceroute <network> -> Performs a traceroute to specified network

     

     

    The configurations and the outputs are explained in detail below.

    K#wr t
    Current configuration:
    !
    version 11.1
    !
    hostname K
    !
    ip subnet-zero
    !
    interface Fddi1/0
    description Apricot Test GIX Backbone
    ip address 169.223.0.15 255.255.255.0
    no ip redirects
    no ip directed-broadcast
    no ip proxy-arp
    ip route-cache same-interface
    ip route-cache flow
    no keepalive
    !
    !
    interface Serial6/0
    description T1 to HKT NetPlus network
    ip address 169.223.1.21 255.255.255.252
    no ip redirects
    no ip directed-broadcast
    no ip proxy-arp
    ip route-cache flow
    !
    interface Serial6/1
    description T1 to HKIX exchange
    ip address 169.223.1.25 255.255.255.252
    no ip redirects
    no ip directed-broadcast
    no ip proxy-arp
    ip route-cache flow
    no fair-queue
    !
    autonomous-system 7722
    !
    router bgp 7722
    no synchronization
    bgp dampening
    aggregate-address 169.222.0.0 255.254.0.0
    aggregate-address 169.222.0.0 255.255.0.0
    aggregate-address 169.223.0.0 255.255.0.0
    aggregate-address 169.223.0.0 255.255.128.0
    aggregate-address 169.223.128.0 255.255.128.0
    redistribute connected route-map connected-to-bgp
    redistribute static route-map connected-to-bgp
    !
  • Routers (A - F) can be combined into the Peer Group "External" since router K has the same outbound routing policy to all of them. Using the Peer Group allows for all the policy statements to be defined just one time. The configuration is hence simpler, the other advantage is the reduction in CPU utilization due to optimization of the routing update process in the router.

    This peer-group basically groups together all the router who are a part of the IXP setup.

    • !
    neighbor external peer-group
    neighbor external send-community
    neighbor external version 4
    neighbor external distribute-list 180 in
    neighbor external distribute-list 180 out
    neighbor external route-map in-peer in
    neighbor external filter-list 198 out
    !
  • Routers in the Peer Group "External" have the Access-List 180 applied to Routing Updates being received from and sent to them. Also, the Route-Map "in-peer" is applied to received updates and filter-list 198 is applied to updates being sent out.

    Distribute-List 180 : This policy rule applied in the inbound and outbound routing updates ensures that the RFC1918 private addresses are no sent out or received in.

    Filter-List 198 : This when applied to updates in the outbound direction ensures that only routes originating in AS4000, AS4635, AS4637 are sent to members of the IXP peer group "external".

    Route-Map In-Peer : This policy rule applied in the inbound direction is used to set the Multi-Exit-Discriminator field on all incoming routes to 10.

    •
    • !
    neighbor oddball peer-group
    neighbor oddball version 4
    neighbor oddball distribute-list 180 in
    neighbor oddball distribute-list 180 out
    neighbor oddball route-map in-peer in
    neighbor full peer-group
    neighbor full version 4
    neighbor full distribute-list 180 in
    neighbor full distribute-list 180 out
    neighbor full route-map in-peer in
    !
  • This part of the configuration lists the BGP neighbors of router K
    •
  • !
  • neighbor 169.223.0.3 remote-as 7717
  • neighbor 169.223.0.3 peer-group oddball
  • neighbor 169.223.0.4 remote-as 7717
  • neighbor 169.223.0.4 peer-group oddball
  • neighbor 169.223.0.10 remote-as 7711
  • neighbor 169.223.0.10 peer-group full
  • neighbor 169.223.0.11 remote-as 7712
  • neighbor 169.223.0.11 peer-group external
  • neighbor 169.223.0.12 remote-as 7713
  • neighbor 169.223.0.12 peer-group external
  • neighbor 169.223.0.13 remote-as 7714
  • neighbor 169.223.0.13 peer-group external
  • neighbor 169.223.0.14 remote-as 7715
  • neighbor 169.223.0.14 peer-group external
  • neighbor 169.223.0.20 remote-as 7716
  • neighbor 169.223.0.20 peer-group external
  • neighbor 202.40.161.1 remote-as 4635
  • neighbor 202.40.161.1 version 4
  • neighbor 202.40.161.1 distribute-list 190 in
  • neighbor 202.40.161.1 distribute-list 191 out
  • neighbor 202.40.161.1 route-map from-transit in
  • neighbor 204.59.88.29 remote-as 4000
  • neighbor 204.59.88.29 version 4
  • neighbor 204.59.88.29 distribute-list 190 in
  • neighbor 204.59.88.29 distribute-list 191 out
  • neighbor 204.59.88.29 route-map from-transit in
  • neighbor 205.252.130.169 remote-as 4637
  • neighbor 205.252.130.169 version 4
  • neighbor 205.252.130.169 distribute-list 190 in
  • neighbor 205.252.130.169 distribute-list 191 out
  • neighbor 205.252.130.169 route-map from-transit in
  • no auto-summary
  • !
  • ip classless
  • ip route 169.222.0.0 255.254.0.0 Null0
  • ip route 169.222.31.0 255.255.255.128 169.222.47.1
  • ip route 169.223.128.0 255.255.128.0 Null0
  • ip route 202.40.161.1 255.255.255.255 169.223.1.26
  • ip route 205.252.130.169 255.255.255.255 169.223.1.22
  • ip as-path access-list 198 permit ^$
  • ip as-path access-list 198 permit ^(4000|4635|4637)$
  • !
  • Listed here are the Access Lists which are applied to the incoming and outgoing routing updates. (The function of each of these lists is described in more detail below)
    •
    • !
    access-list 100 deny ip any any
    access-list 101 deny ip any any
    access-list 180 deny ip host 0.0.0.0 any
    access-list 180 deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    access-list 180 deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    access-list 180 deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255
    access-list 180 deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 180 deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 180 deny ip 128.0.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 180 deny ip 191.255.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 180 deny ip 192.0.0.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 180 deny ip 223.255.255.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 180 deny ip 224.0.0.0 31.255.255.255 224.0.0.0 31.255.255.255
    access-list 180 permit ip any any
    access-list 190 deny ip host 0.0.0.0 any
    access-list 190 deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    access-list 190 deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    access-list 190 deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255
    access-list 190 deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 190 deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 190 deny ip 128.0.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 190 deny ip 191.255.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 190 deny ip 192.0.0.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 190 deny ip 223.255.255.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 190 deny ip 224.0.0.0 31.255.255.255 224.0.0.0 31.255.255.255
    access-list 190 deny ip any 255.255.255.128 0.0.0.127
    access-list 190 permit ip any any
    access-list 191 permit ip host 169.222.0.0 host 255.254.0.0
    access-list 191 permit ip host 169.222.0.0 host 255.255.0.0
    access-list 191 permit ip host 169.223.0.0 host 255.255.0.0
    access-list 191 permit ip host 169.223.0.0 host 255.255.128.0
    access-list 191 permit ip host 169.223.128.0 host 255.255.128.0
    access-list 191 deny ip any any
    !
  • This section contains the Route Maps which enforce policy rules
    •
    • !
    route-map connected-to-bgp permit 10
    set origin igp
    !
    route-map in-peer permit 10
    set metric 10
    set community 7722:6 additive
    !
    Apricot GIX Hong Kong
    K.ix.hk.apricot.net
    !
    end
    !
  • The output of a sh ip bgp summary provides the following information :

    - List of BGP neighbors.

    - Number of entries in the BGP table and the associated memory usage.

    - Number of routes affected by Flap Dampening

    Note the AS number associated with each Neighbor. In this case all the routers in the IXP (Routers A- F) are EBGP peered with each other.

    •
    • !
    K#sh ip bgp summary
    BGP table version is 274609, main routing table version 274609
    42593 network entries (43712/87144 paths) using 7565272 bytes of memory
    7705 BGP path attribute entries using 981356 bytes of memory
    3776 BGP route-map cache entries using 60416 bytes of memory
    3776 BGP filter-list cache entries using 60416 bytes of memory
    Dampening enabled. 167 history paths, 85 dampened paths
    Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State
    169.223.0.3 4 7717 1234 210478 274609 0 0 00:03:39
    169.223.0.4 4 7717 1622 167227 274609 0 0 15:53:43
    169.223.0.10 4 7711 2471 54753 274609 0 0 19:53:21
    169.223.0.11 4 7712 2263 2143 274609 0 0 18:53:43
    169.223.0.12 4 7713 2023 2072 274609 0 0 18:44:11
    169.223.0.13 4 7714 2040 2073 274609 0 0 18:42:55
    169.223.0.14 4 7715 2009 2069 274609 0 0 18:41:31
    169.223.0.20 4 7716 2268 2076 274609 0 0 18:40:01
    202.40.161.1 4 4635 7246 1472 274598 0 0 18:17:11
    204.59.88.29 4 4000 17570 1450 274598 0 0 1d00h
    205.252.130.169 4 4637 96694 2005 274598 0 0 1d07h
  • The output of a "sh ip bgp neighbor" give detailed information on the associated EBGP neighbor. This includes the policy rules applied to updates coming from and going to this neighbor, traffic information as well as information on the BGP state machine.
    •
    • K#sh ip bgp neighbor 169.223.0.11
    BGP neighbor is 169.223.0.11, remote AS 7712, external link
    Index 1, Offset 0, Mask 0x2
    external peer-group member
    Community attribute sent to this neighbor
    BGP version 4, remote router ID 169.223.1.5
    BGP state = Established, table version = 274610, up for 18:54:07
    Last read 00:00:08, hold time is 180, keepalive interval is 60 seconds
    Minimum time between advertisement runs is 5 seconds
    Received 2264 messages, 0 notifications, 0 in queue
    Sent 2144 messages, 0 notifications, 0 in queue
    Inbound path policy configured
    Outbound path policy configured
    Incoming update network filter list is 180
    Outgoing update network filter list is 180
    Outgoing update AS path filter list is 198
    Route map for incoming advertisements is in-peer
    Route map for outgoing advertisements is out-peer
    Connections established 15; dropped 14
    Connection state is ESTAB, I/O status: 1, unread input bytes: 0
    Local host: 169.223.0.15, Local port: 179
    Foreign host: 169.223.0.11, Foreign port: 12808
    Enqueued packets for retransmit: 0, input: 0, saved: 0
    Event Timers (current time is 0x71B0228):
    Timer Starts Wakeups Next
    Retrans 1154 0 0x0
    TimeWait 0 0 0x0
    AckHold 1140 782 0x0
    SendWnd 0 0 0x0
    KeepAlive 0 0 0x0
    GiveUp 0 0 0x0
    PmtuAger 0 0 0x0
    iss: 491726530 snduna: 491749646 sndnxt: 491749646 sndwnd: 14864
    irs: 491733038 rcvnxt: 491754960 rcvwnd: 16042 delrcvwnd: 342
    SRTT: 300 ms, RTTO: 607 ms, RTV: 3 ms, KRTT: 0 ms
    minRTT: 0 ms, maxRTT: 500 ms, ACK hold: 300 ms
    Flags: passive open, nagle, gen tcbs
    Datagrams (max data segment is 4312 bytes):
    Rcvd: 2293 (out of order: 0), with data: 1140, total data bytes: 21921
    Sent: 1942 (retransmit: 0), with data: 1153, total data bytes: 23115
  • Using the command "sh ip bgp regexp _4515$" we can view the list of routes that originated in the AS 4515. The output shown here is the BGP table entries of routes from AS 4515 along with metric, local preference and AS path info.
    •
    • K#sh ip bgp regexp _4515$
    BGP table version is 274639, local router ID is 169.223.1.29
    Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete
    Network Next Hop Metric LocPrf Weight Path
    * 165.202.0.0 202.40.161.1 0 4635 4515 i
    * 169.223.0.11 10 0 7716 7712 4515 i
    *> 169.223.0.11 10 0 7712 4515 i
    * 205.252.130.169 0 4637 4635 4515 i
    * 168.106.0.0 202.40.161.1 0 4635 4515 i
    * 169.223.0.11 10 0 7716 7712 4515 i
    *> 169.223.0.11 10 0 7712 4515 i
    * 205.252.130.169 0 4637 4635 4515 i
    * 202.2.32.0/20 202.40.161.1 0 4635 4515 i
    * 169.223.0.11 10 0 7716 7712 4515 i
    *> 169.223.0.11 10 0 7712 4515 i
    * 205.252.130.169 0 4637 4635 4515 i
    * 202.2.48.0/22 202.40.161.1 0 4635 4515 i
    * 169.223.0.11 10 0 7716 7712 4515 i
    *> 169.223.0.11 10 0 7712 4515 i
    * 205.252.130.169 0 4637 4635 4515 i
    * 202.12.6.0 169.223.0.11 10 0 7716 7712 4515 ?
    *> 169.223.0.11 10 0 7712 4515 ?
    * 202.40.141.0 202.40.161.1 0 4635 4515 i
    * 169.223.0.11 10 0 7716 7712 4515 i
    *> 169.223.0.11 10 0 7712 4515 i
    * 205.252.130.169 0 4637 4635 4515 i
    * 202.40.142.0 202.40.161.1 0 4635 4515 i
    * 169.223.0.11 10 0 7716 7712 4515 i
    *> 169.223.0.11 10 0 7712 4515 i
    * 205.252.130.169 0 4637 4635 4515 i
    *> 202.40.148.0/23 205.252.130.169 0 4637 3561 4515 i
    * 202.40.161.0 169.223.0.11 10 0 7712 4515 ?
    * 202.82.0.0/17 202.40.161.1 0 4635 4515 i
    * 169.223.0.11 10 0 7716 7712 4515 i
    *> 169.223.0.11 10 0 7712 4515 i
    * 205.252.130.169 0 4637 4635 4515 i
    * 206.101.179.0 202.40.161.1 0 4635 4515 i
    * 169.223.0.11 10 0 7716 7712 4515 i
    *> 169.223.0.11 10 0 7712 4515 i
    205.252.130.169 0 4637 4635 4515 i
  • As can be seen from the output of the BGP table for network 202.82.0.0 there are multiple paths to the network. This is due to the fact that the AS 4515 is Multihomed to 7712 as well as the HKIX (AS4635) and HKT NetPlus (AS4637). Based on the BGP route selection policy the path via. 7712 4515 is selected as the best path and is installed in the IP routing table of the router.
    •
    • K#sh ip bgp 202.82.0.0
    BGP routing table entry for 202.82.0.0/17, version 162808
    Paths: (4 available, best #3, advertised over EBGP)
    4635 4515, (aggregated by 4515 202.82.2.34)
    202.40.161.1 from 202.40.161.1
    Origin IGP, metric 10, valid, external, atomic-aggregate
    7716 7712 4515, (aggregated by 4515 202.82.82.101)
    169.223.0.11 from 169.223.0.20 (169.223.2.1)
    Origin IGP, metric 10, valid, external, atomic-aggregate
    7712 4515, (aggregated by 4515 202.82.82.101)
    169.223.0.11 from 169.223.0.11 (169.223.1.5)
    Origin IGP, metric 10, valid, external, atomic-aggregate, best
    4637 4635 4515, (aggregated by 4515 202.82.2.34)
    205.252.130.169 from 205.252.130.169
    Origin IGP, valid, external, atomic-aggregate
    K#sh ip route 202.82.0.0
    Routing entry for 202.82.0.0/17, supernet
    Known via "bgp 7722", distance 20, metric 10
    Tag 7712, type external
    Last update from 169.223.0.11 18:56:44 ago
    Routing Descriptor Blocks:
    * 169.223.0.11, from 169.223.0.11, 18:56:44 ago
    Route metric is 10, traffic share count is 1
    AS Hops 2
  • A traceroute shows the path taken to the destination address 202.82.10.10
    •
    • K#trace 202.82.10.10
    Type escape sequence to abort.
    Tracing the route to 202.82.10.10
    1 169.223.0.11 0 msec 0 msec 0 msec
    2 169.223.1.6 [AS 7712] 4 msec 4 msec 4 msec
    3 202.82.60.65 [AS 4515] 4 msec 4 msec 8 msec
    4 202.82.60.51 [AS 4515] !H !H !H
  • By turning on Netflow switching on the router, you may obtain very good information on the type of IP traffic flowing through the router. As can be seen below, it shows the distribution of the packet sizes, the number of TCP sessions and a list of flows thru’ the router.
    •
    • K#sh ip cache flow optimum
    IP packet size distribution (14739931 total packets):
    1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
    .000 .148 .021 .105 .058 .028 .065 .015 .011 .009 .012 .203 .017 .009 .007
    512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
    .006 .023 .018 .000 .181 .054 .000 .000 .000 .000 .000
    IP Flow Switching Cache, 133 active, 65403 inactive, 49685 added
    0 flows exported, 0 not exported, 0 export msgs sent
    0 cur max hash, 4 worst max hash, 78 valid buckets
    0 flow alloc failures
    statistics cleared 67710 seconds ago
    Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec)
    -------- Flows /Sec /Flow /Pkt /Sec /Flow /Flow
    TCP-Telnet 2443 0.0 152 91 5.5 124.4 43.6
    TCP-FTP 292 0.0 9 68 0.0 17.7 45.9
    TCP-FTPD 216 0.0 183 427 0.5 72.2 47.2
    TCP-WWW 23599 0.3 10 304 3.6 6.6 46.5
    TCP-SMTP 318 0.0 20 136 0.0 13.4 46.7
    TCP-X 172 0.0 1317 299 3.3 112.2 47.3
    TCP-BGP 3192 0.0 19 97 0.9 41.9 45.3
    TCP-Frag 2 0.0 10 1232 0.0 4.9 47.8
    TCP-other 4245 0.0 80 189 5.0 41.9 45.7
    UDP-DNS 5280 0.0 2 139 0.1 2.4 45.9
    UDP-NTP 1560 0.0 2 76 0.0 0.6 47.4
    UDP-TFTP 1 0.0 2 45 0.0 0.0 50.1
    UDP-other 6075 0.0 9 134 0.8 12.1 45.8
    ICMP 1823 0.0 6 108 0.1 10.4 46.3
    IGMP 218 0.0 1293 442 4.1 959.4 23.6
    IPINIP 116 0.0 110283 415 188.9 1520.4 9.4
    Total: 49552 0.7 291 395 213.5 26.4 45.9
  • This shows a list of flows through the router, information includes Source & Destination addresses, number of packets etc. This information can also be exported to management stations.
    •
    SrcIf SrcIPaddress DstIf DstIPaddress Pr DstP SrcP Pkts B/Pk Active
    Et5/6 169.222.38.7 Se6/1 137.189.6.55 06 04D3 0015 5 42 0.1
    Se6/1 137.189.6.55 Et5/6 169.222.38.7 06 0015 04D3 4 47 0.1
    Se6/2 204.162.80.145 Et5/6 169.222.55.6 06 0050 0655 4 69 0.3
    Et5/6 169.222.55.6 Se6/0 204.162.80.145 06 0651 0050 8 90 0.9
    K#sh ip cache flow cbus
    IP packet size distribution (14741399 total packets):
    1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
    .000 .148 .021 .105 .058 .028 .065 .015 .011 .009 .012 .203 .017 .009 .007
    512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
    .006 .023 .018 .000 .181 .054 .000 .000 .000 .000 .000
    IP Flow Switching Cache, 133 active, 65403 inactive, 49685 added
    0 flows exported, 0 not exported, 0 export msgs sent
    1 cur max hash, 4 worst max hash, 87 valid buckets
    0 flow alloc failures
    statistics cleared 67717 seconds ago
    Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec)
    -------- Flows /Sec /Flow /Pkt /Sec /Flow /Flow
    TCP-Telnet 2443 0.0 152 91 5.5 124.4 43.6
    TCP-FTP 292 0.0 9 68 0.0 17.7 45.9
    TCP-FTPD 216 0.0 183 427 0.5 72.2 47.2
    TCP-WWW 23599 0.3 10 304 3.6 6.6 46.5
    TCP-SMTP 318 0.0 20 136 0.0 13.4 46.7
    TCP-X 172 0.0 1317 299 3.3 112.2 47.3
    TCP-BGP 3192 0.0 19 97 0.9 41.9 45.3
    TCP-Frag 2 0.0 10 1232 0.0 4.9 47.8
    TCP-other 4245 0.0 80 189 5.0 41.9 45.7
    UDP-DNS 5280 0.0 2 139 0.1 2.4 45.9
    UDP-NTP 1560 0.0 2 76 0.0 0.6 47.4
    UDP-TFTP 1 0.0 2 45 0.0 0.0 50.1
    UDP-other 6075 0.0 9 134 0.8 12.1 45.8
    ICMP 1823 0.0 6 108 0.1 10.4 46.3
    IGMP 218 0.0 1293 442 4.1 959.4 23.6
    IPINIP 116 0.0 110283 415 188.9 1520.4 9.4
    Total: 49552 0.7 291 395 213.5 26.4 45.9
    SrcIf SrcIPaddress DstIf DstIPaddress Pr DstP SrcP Pkts B/Pk Active
    Et5/6 169.222.38.7 Se6/1 137.189.6.55 06 04D3 0015 5 42 0.1
    Se6/1 137.189.6.55 Et5/6 169.222.38.7 06 0015 04D3 4 47 0.1
    Se6/2 204.162.80.145 Et5/6 169.222.55.6 06 0050 0655 4 69 0.3
    Et5/6 169.222.55.6 Se6/0 204.162.80.145 06 0651 0050 8 90 0.9
    Se6/2 204.162.80.145 Et5/6 169.222.55.6 06 0050 0654 4 69 0.3
  • The following 2 command output display a list of routes which have been dampened due to excessive flapping. Refer to the configuration for the configuration command to enable this important feature.
    •
    K#sh ip bgp flap-statistics
    BGP table version is 581473, local router ID is 169.223.1.29
    Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete
    Network From Flaps Duration Reuse Path
    *d 129.107.0.0 205.252.130.169 4 00:27:07 00:00:20 4637 3561 1239 1791
    3354
    *d 132.150.0.0 205.252.130.169 602 2d01h 00:35:20 4637 3491 1790 1239
    4000 2874 2874 2874 5465 2120
    *> 133.8.0.0 205.252.130.169 1 00:00:42 4637 5683 1794 1239
    1792 2907
    h 133.125.0.0 202.40.161.1 1577 1d02h 4635 4637 2516 2521
    h 133.155.0.0 202.40.161.1 1515 1d01h 4635 4637 2516 2521
    h 133.158.0.0 202.40.161.1 1515 1d01h 4635 4637 2516 2521
    K#sh ip bgp dampened-paths
    BGP table version is 581490, local router ID is 169.223.1.29
    Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete
    Network From Reuse Path
    *d 129.107.0.0 205.252.130.169 00:00:1 4637 3561 1239 1791 3354 ?
    *d 198.151.250.0 205.252.130.169 00:00:2 4637 3561 1239 1791 3354 ?
    *d 192.124.225.0 205.252.130.169 00:00:2 4637 3561 1239 1791 3354 ?
    *d 192.156.4.0/22 205.252.130.169 00:00:2 4637 5683 568 i
    *d 199.249.179.0 205.252.130.169 00:01:3 4637 3561 2828 5097 i
    *d 204.0.0.0/14 205.252.130.169 00:02:3 4637 3561 114 i
    Apricot GIX Hong Kong
    B.ix.hk.apricot.net
    B#wr t
    Current configuration:
    !
    version 11.1
    !
    hostname B
    !
    ip subnet-zero
    !
    interface Fddi0
    description Apricot Test GIX
    ip address 169.223.0.11 255.255.255.0
    no ip redirects
    no ip directed-broadcast
    no ip proxy-arp
    ip route-cache same-interface
    no keepalive
    !
    interface Serial0
    description To Hk Star
    ip address 169.223.1.5 255.255.255.252
    no ip redirects
    no ip directed-broadcast
    no ip proxy-arp
    no fair-queue
    !
    router bgp 7712
    no synchronization
    bgp dampening
    redistribute connected route-map connected-to-bgp
    !
  • Since all the routers in the IXP are direct EBGP peers across the FDDI backbone, this routers BGP Peer Groups configuration is similar to router K’s configuration. The only difference is the Outbound Route Map to the routers in the IXP.

    Distribute-List 180 : This policy rule applied in the inbound and outbound routing updates ensures that the RFC1918 private addresses are no sent out or received in.

    Route-Map In-Peer : This policy rule applied in the inbound direction is used to set the Multi-Exit-Discriminator field on all incoming routes to 10 and also tags a community string on them.

    Route-Map Local_Routes_Only : This policy rule references an AS path filter number 11. The filter ensures that when this router is generating an update to the rest of the IXP routers it only advertises its local routes and routes derived from EBGP peers external to the IXP such as AS4515.

    •
    • !
    neighbor external peer-group
    neighbor external send-community
    neighbor external version 4
    neighbor external distribute-list 180 in
    neighbor external distribute-list 180 out
    neighbor external route-map in-peer in
    neighbor external route-map local_routes_only out
    neighbor oddball peer-group
    neighbor oddball version 4
    neighbor oddball distribute-list 180 in
    neighbor oddball distribute-list 180 out
    neighbor oddball route-map in-peer in
    neighbor 169.223.0.3 remote-as 7717
    neighbor 169.223.0.3 peer-group oddball
    neighbor 169.223.0.4 remote-as 7717
    neighbor 169.223.0.4 peer-group oddball
    neighbor 169.223.0.10 remote-as 7711
    neighbor 169.223.0.10 peer-group external
    neighbor 169.223.0.12 remote-as 7713
    neighbor 169.223.0.12 peer-group external
    neighbor 169.223.0.13 remote-as 7714
    neighbor 169.223.0.13 peer-group external
    neighbor 169.223.0.14 remote-as 7715
    neighbor 169.223.0.14 peer-group external
    neighbor 169.223.0.15 remote-as 7722
    neighbor 169.223.0.15 peer-group external
    neighbor 169.223.0.20 remote-as 7716
    neighbor 169.223.0.20 peer-group external
    !
  • This part shows the BGP configuration to the Multihomed AS 4515. The important part of this configuration is the inbound filter-list of 100 and the outbound distribute-list of 101 which control the incoming & outgoing routing information.

    Distribute-List 101 : Applied on outgoing routing updates to AS 4515, this policy rule ensures that this router advertises routes belonging to the IXP network and nothing else.

    Filter-List 100 : Applied on incoming routing updates from AS 4515, this policy rule ensure that this router accepts all routes except those which passed through AS 3561 or AS 701(These AS’s are upstream providers of AS 4515.

    This router accepts routes that belong to AS 4515 and its customers.

    * VERY IMPORTANT *

    •
    • !
    neighbor 169.223.1.6 remote-as 4515
    neighbor 169.223.1.6 soft-reconfiguration inbound
    neighbor 169.223.1.6 version 4
    neighbor 169.223.1.6 distribute-list 101 out
    neighbor 169.223.1.6 filter-list 100 in
    no auto-summary
    !
    ip classless
    ip bgp-community new-format
    ip community-list 1 deny 7712:6
    ip community-list 1 permit
    ip as-path access-list 11 permit ^4515$
    ip as-path access-list 11 permit ^$
    ip as-path access-list 11 deny .*
    ip as-path access-list 11 permit ^{7712&4515}$
    ip as-path access-list 100 deny ^4515 (701|3561)_
    ip as-path access-list 100 permit .*
    access-list 100 deny ip any any
    access-list 101 permit ip host 169.222.0.0 host 255.254.0.0
    access-list 101 permit ip host 169.222.0.0 host 255.255.0.0
    access-list 101 permit ip host 169.223.0.0 host 255.255.0.0
    access-list 101 permit ip host 169.223.0.0 host 255.255.128.0
    access-list 101 permit ip host 169.223.128.0 host 255.255.128.0
    access-list 101 deny ip any any
    access-list 180 deny ip host 0.0.0.0 any
    access-list 180 deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    access-list 180 deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    access-list 180 deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255
    access-list 180 deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 180 deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 180 deny ip 128.0.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 180 deny ip 191.255.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 180 deny ip 192.0.0.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 180 deny ip 223.255.255.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 180 deny ip 224.0.0.0 31.255.255.255 224.0.0.0 31.255.255.255
    access-list 180 permit ip any any
    access-list 190 deny ip host 0.0.0.0 any
    access-list 190 deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    access-list 190 deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    access-list 190 deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255
    access-list 190 deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 190 deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 190 deny ip 128.0.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 190 deny ip 191.255.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    access-list 190 deny ip 192.0.0.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 190 deny ip 223.255.255.0 0.0.0.255 255.255.255.0 0.0.0.255
    access-list 190 deny ip 224.0.0.0 31.255.255.255 224.0.0.0 31.255.255.255
    access-list 190 deny ip any 255.255.255.128 0.0.0.127
    access-list 190 permit ip any any
    access-list 191 permit ip host 169.222.0.0 host 255.254.0.0
    access-list 191 permit ip host 169.222.0.0 host 255.255.0.0
    access-list 191 permit ip host 169.223.0.0 host 255.255.0.0
    access-list 191 deny ip any any
  • Refer the description of route-maps given above.
    •
    route-map connected-to-bgp permit 10
    set origin igp
    !
    route-map in-peer permit 10
    set metric 10
    !
    route-map out-peer permit 10
    match community 1
    set metric-type internal
    !
    route-map local_routes_only permit 10
    match as-path 11
    !
    Apricot GIX Hong Kong
    B.ix.hk.apricot.net
  • Refer to above description of the commands given below.
    •
    B#sh ip bgp summary
    BGP table version is 942, main routing table version 942
    113 network entries (218/238 paths) using 20732 bytes of memory
    51 BGP path attribute entries using 6148 bytes of memory
    4 BGP route-map cache entries using 64 bytes of memory
    0 BGP filter-list cache entries using 0 bytes of memory
    Dampening enabled. 0 history paths, 0 dampened paths
    Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State
    169.223.0.3 4 7717 1077 1643 942 0 0 00:07:34
    169.223.0.4 4 7717 1259 2374 942 0 0 15:57:46
    169.223.0.10 4 7711 3084 3299 942 0 0 19:07:24
    169.223.0.12 4 7713 2060 2266 942 0 0 18:48:18
    169.223.0.13 4 7714 2077 2271 942 0 0 18:47:09
    169.223.0.14 4 7715 2030 2256 942 0 0 18:45:53
    169.223.0.15 4 7722 2175 2282 942 0 0 18:57:58
    169.223.0.20 4 7716 2276 2236 942 0 0 18:44:09
    169.223.1.6 4 4515 1653 1468 942 0 0 19:06:50
    B#sh ip bgp neigh 169.223.1.6
    BGP neighbor is 169.223.1.6, remote AS 4515, external link
    Index 3, Offset 0, Mask 0x8
    Inbound soft reconfiguration allowed
    BGP version 4, remote router ID 202.82.82.101
    BGP state = Established, table version = 942, up for 19:07:04
    Last read 00:00:07, hold time is 180, keepalive interval is 60 seconds
    Minimum time between advertisement runs is 30 seconds
    Received 1654 messages, 0 notifications, 0 in queue
    Sent 1469 messages, 0 notifications, 0 in queue
    Inbound path policy configured
    Outgoing update network filter list is 101
    Incoming update AS path filter list is 100
    Connections established 9; dropped 8
    Connection state is ESTAB, I/O status: 1, unread input bytes: 0
    Local host: 169.223.1.5, Local port: 12792
    Foreign host: 169.223.1.6, Foreign port: 179
    Enqueued packets for retransmit: 0, input: 0, saved: 0
    Event Timers (current time is 0x73B27E4):
    Timer Starts Wakeups Next
    Retrans 1159 0 0x0
    TimeWait 0 0 0x0
    AckHold 1177 1158 0x0
    SendWnd 0 0 0x0
    KeepAlive 0 0 0x0
    GiveUp 0 0 0x0
    PmtuAger 0 0 0x0
    iss: 4249316589 snduna: 4249338937 sndnxt: 4249338937 sndwnd: 15985
    irs: 4249355398 rcvnxt: 4249378299 rcvwnd: 15486 delrcvwnd: 898
    SRTT: 301 ms, RTTO: 616 ms, RTV: 7 ms, KRTT: 0 ms
    minRTT: 4 ms, maxRTT: 396 ms, ACK hold: 300 ms
    Flags: higher precedence, nagle
    Datagrams (max data segment is 1460 bytes):
    Rcvd: 2160 (out of order: 0), with data: 1177, total data bytes: 22900
    Sent: 2333 (retransmit: 0), with data: 1158, total data bytes: 22347
    B#sh ip bgp regexp _4515$
    BGP table version is 942, local router ID is 169.223.1.5
    Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete
    Network Next Hop Metric LocPrf Weight Path
    *> 165.202.0.0 169.223.1.6 0 0 4515 i
    *> 168.106.0.0 169.223.1.6 0 0 4515 i
    * 169.223.0.0 169.223.1.6 0 0 4515 ?
    *> 192.168.2.0 169.223.1.6 0 0 4515 ?
    *> 192.168.3.0 169.223.1.6 0 0 4515 i
    *> 192.168.4.0 169.223.1.6 0 0 4515 ?
    *> 202.2.32.0/20 169.223.1.6 0 0 4515 i
    *> 202.2.48.0/22 169.223.1.6 0 0 4515 i
    *> 202.12.6.0 169.223.1.6 0 0 4515 ?
    *> 202.40.141.0 169.223.1.6 20 0 4515 i
    *> 202.40.142.0 169.223.1.6 20 0 4515 i
    *> 202.40.161.0 169.223.1.6 0 4515 ?
    *> 202.82.0.0/17 169.223.1.6 0 4515 i
    *> 206.101.179.0 169.223.1.6 20 0 4515 I
  • The following outputs display the BGP Routing table entries for the same network 202.82.0.0 shown in router K above.
    • B#sh ip bgp 202.82.0.0
    BGP routing table entry for 202.82.0.0/17, version 210
    Paths: (1 available, best #1, advertised over EBGP)
    4515, (aggregated by 4515 202.82.82.101), (received & used)
    169.223.1.6 from 169.223.1.6 (202.82.82.101)
    Origin IGP, valid, external, atomic-aggregate, best
    B#sh ip route 202.82.0.0
    Routing entry for 202.82.0.0/17, supernet
    Known via "bgp 7712", distance 20, metric 0
    Tag 4515, type external
    Last update from 169.223.1.6 19:07:42 ago
    Routing Descriptor Blocks:
    * 169.223.1.6, from 169.223.1.6, 19:07:42 ago
    Route metric is 0, traffic share count is 1
    AS Hops 1
    B#trace 202.82.10.10
    Type escape sequence to abort.
    Tracing the route to 202.82.10.10
    1 169.223.1.6 4 msec 4 msec 4 msec
    2 202.82.60.65 [AS 4515] 4 msec 4 msec 8 msec
    3 202.82.60.51 [AS 4515] !H !H !H
  • The following outputs describe the various policy rules being applied to the incoming and outgoing routing updates from BGP neighbors.
    •
    • B#sh route-map local_routes_only
    route-map local_routes_only, permit, sequence 10
    Match clauses:
    as-path (as-path filter): 11
    Set clauses:
    Policy routing matches: 0 packets, 0 bytes
    B#sh ip as
    B#sh ip as-path-access-list 11
    AS path access list 11
    permit ^4515$
    permit ^$
    deny .*
    permit ^{7712&4515}$
    B#sh ip access-lists 180
    Extended IP access list 180
    deny ip host 0.0.0.0 any
    deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255
    deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255 (90 matches)
    deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255
    deny ip 128.0.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    deny ip 191.255.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    deny ip 192.0.0.0 0.0.0.255 255.255.255.0 0.0.0.255
    deny ip 223.255.255.0 0.0.0.255 255.255.255.0 0.0.0.255
    deny ip 224.0.0.0 31.255.255.255 224.0.0.0 31.255.255.255
    permit ip any any (8601 matches)
    B#sh ip access-lists 101
    Extended IP access list 101
    permit ip host 169.222.0.0 host 255.254.0.0 (22 matches)
    permit ip host 169.222.0.0 host 255.255.0.0 (22 matches)
    permit ip host 169.223.0.0 host 255.255.0.0 (22 matches)
    permit ip host 169.223.0.0 host 255.255.128.0 (22 matches)
    permit ip host 169.223.128.0 host 255.255.128.0 (22 matches)
    deny ip any any (4330 matches)
    B#sh ip as-path-access-list 100
    AS path access list 100
    deny ^4515 (701|3561)_
    permit .*
    B#sh ip access-lists 190
    Extended IP access list 190
    deny ip host 0.0.0.0 any
    deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
    deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255
    deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255
    deny ip 128.0.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    deny ip 191.255.0.0 0.0.255.255 255.255.0.0 0.0.255.255
    deny ip 192.0.0.0 0.0.0.255 255.255.255.0 0.0.0.255
    deny ip 223.255.255.0 0.0.0.255 255.255.255.0 0.0.0.255
    deny ip 224.0.0.0 31.255.255.255 224.0.0.0 31.255.255.255
    deny ip any 255.255.255.128 0.0.0.127
    permit ip any any
    B#sh ip access-lists 191
    Extended IP access list 191
    permit ip host 169.222.0.0 host 255.254.0.0
    permit ip host 169.222.0.0 host 255.255.0.0
    permit ip host 169.223.0.0 host 255.255.0.0
    deny ip any any
  • This output shows the state of the interface going out to AS4515, including packet drops and packet rates through the interface.
    •
    • B#sh int s 0
    Serial0 is up, line protocol is up
    Hardware is HD64570
    Description: To Hk Star
    Internet address is 169.223.1.5/30
    MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec, rely 255/255, load 1/255
    Encapsulation HDLC, loopback not set, keepalive set (10 sec)
    Last input 00:00:09, output 00:00:02, output hang never
    Last clearing of "show interface" counters never
    Queueing strategy: fifo
    Output queue 0/40, 0 drops; input queue 0/75, 0 drops, 0 flushes
    5 minute input rate 0 bits/sec, 0 packets/sec
    5 minute output rate 0 bits/sec, 0 packets/sec
    1461791 packets input, 688988222 bytes, 0 no buffer
    Received 13268 broadcasts, 0 runts, 0 giants
    27874 input errors, 6466 CRC, 328 frame, 2 overrun, 179 ignored, 1229 abort
    921043 packets output, 66316056 bytes, 0 underruns
    0 output errors, 0 collisions, 219 interface resets
    0 output buffer failures, 0 output buffers swapped out
    214 carrier transitions
    DCD=up DSR=up DTR=up RTS=up CTS=up