APRICOT II

Internet Exchange Point

Routing Configurations & Design Commentary

(Draft Version 1.0)

By Andrew Partan

Introduction

Cisco Router Configs with Comments from Andrew Partan

 

These are the configs of the cisco routers that were in use atApricot '97 in Hong Kong.The file 'common' is the "master" file for the config of all ofthe routers. I wrote a little script ('fa') that converts this

file into configs for each of the routers.

The file 'common' has all of the comments on how I set up the

routing and why I did it that way.

There are also files (*-confg) that contain the final config of

all of the routers. Various changes to the running config were

made over the course of the week that were not reflected in the

"master" config file (the file 'common').

Also some of the changes to the master config made over the course

of the week were not made to all of the routers.

This is the code that we were running on the routers:

c4500-p-mz.111-8.CA4.6

c7200-j-mz.111-8.CA

Index of the files:

common the starting config of all of the routers

fa script to convert the above into configs per router

B starting (generated) config of router B

C starting (generated) config of router C

D starting (generated) config of router D

E starting (generated) config of router E

F starting (generated) config of router F

K starting (generated) config of router K

b-confg final (actual) config of router B

c-confg final (actual) config of router C

d-confg final (actual) config of router D

e-confg final (actual) config of router E

f-confg final (actual) config of router F

k-confg final (actual) config of router K

 

COMMON

! A few words on the formatting of this file - this file really! contains the configuration of 6 routers (named B-F,K).!! Lines that contain !! are 'special' - the script that converts this! common file into the per router configurations looks at lines that

! contain !! and figures out if they will go into the router's config

! or not.

!

! Note that most of these routers had very little memory - some of the

! odd things that we are doing are to restrict the number of routes sent

! around so as to not run the routers out of memory.

!

! These are all configs for use with cisco version 11.1 code

version 11.1

!

! Turn off some of the services that the router offers by default.

! Also turn on the TCP nagle algorithm - for telnets to/from the router.

no service finger

service nagle

no service slave-log

no service pad

!

! Put in router-generated timestamps on all syslog and debug messages.

service timestamps debug datetime msec

service timestamps log datetime msec

!

! Encrypt the password (note: encryption '7' is reversable).

service password-encryption

!

! Compress the config - this allows for very big configs to fit

! into NVram.

service compress-config

!

! Turn off these - there used to be a denial of service bug (since

! fixed), but I turn them off anyhow - the router should spend its

! time doing routing, not other stuff.

no service udp-small-servers

no service tcp-small-servers

!

! hostname is B-F,K

!!hostname B

!!hostname C

!!hostname D

!!hostname E

!!hostname F

!!hostname K

!

! We use the enable secret instead of the enable password - the enable

! secret is encrypted via a one-way algorithm (the enable password is

! encrypted with a reversible algorithm).

enable secret jellybean

no enable password

!

! The internet is now classless - permit subnet zero.

ip subnet-zero

! Turn on SPD - if the router gets overloaded, it will attepmt to

! toss non-routing packets instead of routing packets.

ip spd enable

! Give the router a ftp user and password.

ip ftp user cisco

ip ftp password cisco

! Turns on incoming rsh connections - but all must come from 'nobody'.

ip rcmd rsh-enable

ip rcmd remote-username nobody

!

! Each routers interface to the Apricot Test GIX

!!interface fddi0 ! B/C

!!interface ethernet0 ! D/F

!!interface fastether5/0 ! E

!!interface fddi1/0 ! K

description Apricot Test GIX

!!ip address 169.223.0.11 255.255.255.0 ! B

!!ip address 169.223.0.12 255.255.255.0 ! C

!!ip address 169.223.0.13 255.255.255.0 ! D

!!ip address 169.223.0.14 255.255.255.0 ! E

!!ip address 169.223.0.15 255.255.255.0 ! K

!!ip address 169.223.0.20 255.255.255.0 ! F

! I always turn these off on all backbone interfaces.

! Sending redirects or doing proxy-arp for other routers does not

! make any sense.

! Permitting someone to send you a packet that you will then send out

! as a broadcast does not make any sense either.

no ip redirects

no ip directed-broadcast

no ip proxy-arp

ip route-cache same-interface

! turn off keepalives on fddi interfaces - default

!!no keepalive ! B/C/E/K

!

! routers F & K also have an ethernet interface.

!!interface ethernet1 ! F

!!interface ethernet5/6 ! K

!!description To Apricot GIX servers ! F

!!description To Shownet ! K

!!ip address 169.223.2.1 255.255.255.0 ! F

!!ip address 169.222.32.253 255.255.224.0 ! K

!!no ip redirects ! F/K

!!no ip directed-broadcast ! F/K

!!no ip proxy-arp ! F/K

!!ip route-cache same-interface ! F/K

!

! router K has another ethernet interface.

!!interface ethernet5/7 ! K

!!description To Mbone ! K

!!ip address 169.222.30.1 255.255.255.0 ! K

!!no ip redirects ! K

!!no ip directed-broadcast ! K

!!no ip proxy-arp ! K

!!ip route-cache same-interface ! K

!

! Routers B/C/D/E/K each have a T1 to an ISP.

! T1 to each ISP, using /30s of 169.223.1/24

!!interface Serial0 ! B/C/D

!!interface Serial1/0 ! E

!!interface Serial6/0 ! K

!!description To Hk Star ! B

!!description To HkT IMS ! C

!!description To Vision? ! D

!!description To Sing Tao ! E

!!description To HKT NetPlus ! K

!!ip address 169.223.1.5 255.255.255.252 ! B

!!ip address 169.223.1.9 255.255.255.252 ! C

!!ip address 169.223.1.13 255.255.255.252 ! D

!!ip address 169.223.1.17 255.255.255.252 ! E

!!ip address 169.223.1.21 255.255.255.252 ! K

!!no ip redirects ! B/C/D/E/K

!!no ip directed-broadcast ! B/C/D/E/K

!!no ip proxy-arp ! B/C/D/E/K

! Fair queue has had a number of bugs. I also believe in not

! trying to reorder or otherwise figure out which packets should

! get better service, so I turn this off. Also, on a backbone

! router, things that unnecessarily add to the router's load should

! be avoided.

!!no fair-queue ! B/C/D/E/K

!

! router K has a T1 to another ISP.

!!interface Serial6/1 ! K

!!description To HKIX ! K

!!ip address 169.223.1.25 255.255.255.252 ! K

!!no ip redirects ! K

!!no ip directed-broadcast ! K

!!no ip proxy-arp ! K

!!no fair-queue ! K

!

! router K has a T1 to another ISP.

!!interface Serial6/2 ! K

!!description To Global One ! K

!!ip address 169.223.1.29 255.255.255.252 ! K

!!no ip redirects ! K

!!no ip directed-broadcast ! K

!!no ip proxy-arp ! K

!!no fair-queue ! K

!

! Each router is running BGP - each has its own AS.

!!router bgp 7712 ! B

!!router bgp 7713 ! C

!!router bgp 7714 ! D

!!router bgp 7715 ! E

!!router bgp 7716 ! F

!!router bgp 7722 ! K

! Router K should announce the Apricot space to the rest of the

! world. Apricot was allocated a /15. In theory we really should

! have announced just the /15. However the 2nd /16 (169.223/16)

! had been allocated to a previous show (long over) and that show's

! transit provider (MCI) had not removed the static route to this

! /16. Until we got MCI to remove this old static route (which

! they did), we also announced a few more routes - both /16s and

! two /17s - to have more specific routes out there to attempt to

! override MCI's static /16.

! Note: These aggregate-address commands do not always put the

! routes into BGP - they will only do so if the route exists in

! the main routing table. A better way of doing this may be to

! install a static route to Null0 and then redistribute static

! routes into BGP (this is what we actually did).

!!aggregate-address 169.222.0.0 255.254.0.0 ! K - GIX/shownet

!!aggregate-address 169.222.0.0 255.255.0.0 ! K - shownet

!!aggregate-address 169.223.0.0 255.255.0.0 ! K - GIX

!!aggregate-address 169.223.0.0 255.255.128.0 ! K - GIX, 1st 1/2

!!aggregate-address 169.223.128.0 255.255.128.0 ! K - GIX, 2nd 1/2

! If all of your routers in your AS are doing BGP, then you turn

! off the IGP/BGP synchronization - this really does not make

! sense here, since we had a pile of single router ASs.

no synchronization

! Turn on bgp dampening - bgp will look for flapping routes and

! dampen (supress) the routes that flap too often.

bgp dampening

! Pick up all of the connected and static routes.

! Note: since we were using parts of 169.223/16 on all of the

! routers (for their interfaces to other ethers and to the T1s),

! we had to make sure that all of the routes got passed around -

! and since these routers using these parts of 169.223/16 are in

! different ASs, we pass these routes around using BGP - even

! though some of them are /30s.

redistribute connected route-map connected-to-bgp

redistribute static route-map connected-to-bgp

! We set up a couple of peer-groups - these are used to classify

! peering into similar groups.

! Use of peer-groups can reduce the load on the router since

! updates are calculated once per peer-group, instead of once per

! router.

!

! Peer groups are great for external peers at a peering point.

! They are also great for all of your iBGP peers (not shown here).

! Note that all members of a peer group must have the same outbound

! policy (must take the same set of routes) and must all be on

! the same LAN. [This 2nd restriction is removed for iBGP peers.]

!

! Normal external peers - these can take communities like all real

! Internet routers.

! Its unclear if you really want to send communities to your peers.

! This is currently an open question in the internet.

! Not all peers can support communities (current gated will reset

! the bgp session if its get a community).

neighbor external peer-group

neighbor external send-community

neighbor external version 4

neighbor external distribute-list 180 in

neighbor external distribute-list 180 out

! K will send some of the routes from its transit providers to the

! rest of the routers. This gives the other routers some routes to

! look at, but not overfill their (limited) memory with full routes.

!!neighbor external filter-list 198 out ! K

neighbor external route-map in-peer in

neighbor external route-map out-peer out

! Send full routes (and communities) to these peers.

neighbor full peer-group

neighbor full send-community

neighbor full version 4

neighbor full distribute-list 180 in

neighbor full distribute-list 180 out

neighbor full route-map in-peer in

neighbor full route-map out-peer out

! Whako external peers - these can not take communities.

! But these do get full routes.

neighbor oddball peer-group

neighbor oddball version 4

neighbor oddball distribute-list 180 in

neighbor oddball distribute-list 180 out

neighbor oddball route-map in-peer in

neighbor oddball route-map out-peer out

! Now set up each of the external peers.

neighbor 169.223.0.3 remote-as 7717 ! DEC

neighbor 169.223.0.3 peer-group oddball

neighbor 169.223.0.4 remote-as 7717 ! DEC

neighbor 169.223.0.4 peer-group oddball

neighbor 169.223.0.10 remote-as 7711 ! A

neighbor 169.223.0.10 peer-group full

!!neighbor 169.223.0.11 remote-as 7712 ! on C/D/E/F/K

!!neighbor 169.223.0.11 peer-group external ! on C/D/E/F/K

!!neighbor 169.223.0.12 remote-as 7713 ! on B/D/E/F/K

!!neighbor 169.223.0.12 peer-group external ! on B/D/E/F/K

!!neighbor 169.223.0.13 remote-as 7714 ! on B/C/E/F/K

!!neighbor 169.223.0.13 peer-group external ! on B/C/E/F/K

!!neighbor 169.223.0.14 remote-as 7715 ! on B/C/D/F/K

!!neighbor 169.223.0.14 peer-group external ! on B/C/D/F/K

!!neighbor 169.223.0.15 remote-as 7722 ! on B/C/D/E/F

!!neighbor 169.223.0.15 peer-group external ! on B/C/D/E/F

!!neighbor 169.223.0.20 remote-as 7716 ! on B/C/D/E/K

!!neighbor 169.223.0.20 peer-group external ! on B/C/D/E/K

!

! These are the bgp session with each ISP.

! We can not use a peer-group here since they are not on a common LAN.

!

!!neighbor 169.223.1.6 remote-as 4515 ! B

!!neighbor 169.223.1.6 version 4 ! B

!!neighbor 169.223.1.6 distribute-list 101 out ! B

!!neighbor 169.223.1.6 filter-list 100 in ! B

!!neighbor 169.223.1.10 remote-as 4760 ! C

!!neighbor 169.223.1.10 version 4 ! C

!!neighbor 169.223.1.10 distribute-list 100 in ! C

!!neighbor 169.223.1.10 distribute-list 101 out ! C

!!neighbor 169.223.1.14 remote-as 4610 ! D

!!neighbor 169.223.1.14 version 4 ! D

!!neighbor 169.223.1.14 distribute-list 100 in ! D

!!neighbor 169.223.1.14 distribute-list 101 out ! D

!!neighbor 169.223.1.18 remote-as 4655 ! E

!!neighbor 169.223.1.18 version 4 ! E

!!neighbor 169.223.1.18 distribute-list 100 in ! E

!!neighbor 169.223.1.18 distribute-list 101 out ! E

!!neighbor 202.40.161.1 remote-as 4635 ! K

!!neighbor 202.40.161.1 ebgp-multihop 3 ! K

!!neighbor 202.40.161.1 version 4 ! K

!!neighbor 202.40.161.1 distribute-list 190 in ! K

!!neighbor 202.40.161.1 distribute-list 191 out ! K

!!neighbor 202.40.161.1 route-map from-transit in ! K

!!neighbor 204.59.88.29 remote-as 4000 ! K

!!neighbor 204.59.88.29 version 4 ! K

!!neighbor 204.59.88.29 distribute-list 190 in ! K

!!neighbor 204.59.88.29 distribute-list 191 out ! K

!!neighbor 204.59.88.29 route-map from-transit in ! K

!!neighbor 205.252.130.169 remote-as 4637 ! K

!!neighbor 205.252.130.169 ebgp-multihop 3 ! K

!!neighbor 205.252.130.169 version 4 ! K

!!neighbor 205.252.130.169 distribute-list 190 in ! K

!!neighbor 205.252.130.169 distribute-list 191 out ! K

!!neighbor 205.252.130.169 route-map from-transit in ! K

!

! The internet is now classless - do not summarize routes to

! the old classfull A/B/C routes.

no auto-summary

!

! Default domain name for when the router does name lookups.

ip domain-name ix.hk.apricot.net

! A couple of nameservers.

ip name-server 169.222.32.1

ip name-server 137.189.6.1

ip name-server 137.189.192.3

! The internet is classless.

ip classless

! Use the new format (AS:NN) for communities.

ip bgp-community new-format

! We use bgp communities fairly extensively.

! One thing that you do not want to do is to send one peer's routes to

! peer - unless you are explicately providing transit between these

! peers.

! We mark all peer routes (when recieved) with a special community -

! the community we use is out-AS:6. Then when we send routes out to

! another peer, we block all of the routes marked with this special

! community.

! Router F is acting as a route server and does not block peer/peer

! routes - it will send out to its peers all of the routes that it

! knows about. It is explicately providing transit for its peers.

! Blocking of communities is done with a community access-list.

! Block the peer community

no ip community-list 1

!!ip community-list 1 deny 7712:6 ! B

!!ip community-list 1 deny 7713:6 ! C

!!ip community-list 1 deny 7714:6 ! D

!!ip community-list 1 deny 7715:6 ! E

!!ip community-list 1 deny 7716:6 ! F

!!ip community-list 1 deny 7722:6 ! K

ip community-list 1 permit

! This list is used to explicately list all of the as-paths from each ISP.

! Until we know what as-paths to recieve, we accept none.

no ip as-path access-list 100

!!ip as-path access-list 100 deny ^4515 (701|3561)_ ! B

!!ip as-path access-list 100 permit .* ! B

!!! K: Match just the internal routes of our transits.

!!! K: These will be the routes we send to our peers that do not get full routes

!!no ip as-path access-list 198 ! K

!!ip as-path access-list 198 permit ^$ ! K

!!ip as-path access-list 198 permit ^(4000|4635|4637)$ ! K

! Keep a local copy of all syslog messages; set the size of this buffer.

logging buffered 16384

! syslog all debug level messages.

logging trap debugging

! Where to send the syslog messages to.

logging 169.222.32.1

! This list is used to explicately list all of the routes from each ISP.

! Until we know what routes to receive, we accept none.

no access-list 100

access-list 100 deny ip any any

! This list is used to list the routes we send to each ISP.

! For now, send nothing.

no access-list 101

!!access-list 101 permit ip host 169.222.0.0 host 255.254.0.0 ! B

!!access-list 101 permit ip host 169.222.0.0 host 255.255.0.0 ! B

!!access-list 101 permit ip host 169.223.0.0 host 255.255.0.0 ! B

!!access-list 101 permit ip host 169.223.0.0 host 255.255.128.0 ! B

!!access-list 101 permit ip host 169.223.128.0 host 255.255.128.0 ! B

access-list 101 deny ip any any

! This list is used to block bogon routes to/from peers.

! Deny martian routes

no access-list 180

! 0/anything

access-list 180 deny ip host 0.0.0.0 any

! 127/8 & longer

access-list 180 deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255

! The private use nets

access-list 180 deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255

access-list 180 deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255

access-list 180 deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255

! Test net

access-list 180 deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255

! 1st and last classical B and C nets (guard nets).

access-list 180 deny ip 128.0.0.0 0.0.255.255 255.255.0.0 0.0.255.255

access-list 180 deny ip 191.255.0.0 0.0.255.255 255.255.0.0 0.0.255.255

access-list 180 deny ip 192.0.0.0 0.0.0.255 255.255.255.0 0.0.0.255

access-list 180 deny ip 223.255.255.0 0.0.0.255 255.255.255.0 0.0.0.255

! All multicast routes - the router now does this itself, but it didn't

! at one point.....

access-list 180 deny ip 224.0.0.0 31.255.255.255 224.0.0.0 31.255.255.255

! I would normally block all routes with a mask longer than /24,

! except that here we are using /30s of 169.223/16 for the serial links

! to each ISP, and we want to pass these between all of the routers at

! this test GIX.

!access-list 180 deny ip any 255.255.255.128 0.0.0.127

access-list 180 permit ip any any

!

! This list is used to block bogon routes from our transits.

! Deny martian routes

no access-list 190

! 0/anything

access-list 190 deny ip host 0.0.0.0 any

! 127/8 & longer

access-list 190 deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255

! The private use nets

access-list 190 deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255

access-list 190 deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255

access-list 190 deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255

! Test net

access-list 190 deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255

! 1st and last classical B and C nets (guard nets).

access-list 190 deny ip 128.0.0.0 0.0.255.255 255.255.0.0 0.0.255.255

access-list 190 deny ip 191.255.0.0 0.0.255.255 255.255.0.0 0.0.255.255

access-list 190 deny ip 192.0.0.0 0.0.0.255 255.255.255.0 0.0.0.255

access-list 190 deny ip 223.255.255.0 0.0.0.255 255.255.255.0 0.0.0.255

! All multicast routes - the router now does this itself, but it didn't

! at one point.....

access-list 190 deny ip 224.0.0.0 31.255.255.255 224.0.0.0 31.255.255.255

! Block all routes > /24

access-list 190 deny ip any 255.255.255.128 0.0.0.127

access-list 190 permit ip any any

!

! This list is used to list (explicately) the routes that we are sending

! to our transit providers.

no access-list 191

access-list 191 permit ip host 169.222.0.0 host 255.254.0.0

access-list 191 permit ip host 169.222.0.0 host 255.255.0.0

access-list 191 permit ip host 169.223.0.0 host 255.255.0.0

access-list 191 permit ip host 169.223.0.0 host 255.255.128.0

access-list 191 permit ip host 169.223.128.0 host 255.255.128.0

access-list 191 deny ip any any

!

! This route-map is used to change the routes that we get from our peers.

! The additive adds this community to any that happen to be there;

! if we did not have the additive, then any incoming communities

! would be removed.

! Override incoming MEDs (if they send any); set the peer community.

no route-map in-peer

route-map in-peer permit

set metric 10

!!set community 7712:6 additive ! B

!!set community 7713:6 additive ! C

!!set community 7714:6 additive ! D

!!set community 7715:6 additive ! E

!!set community 7716:6 additive ! F

!!set community 7722:6 additive ! K

!

! This route-map is used to limit and change the routes that we

! send to our peers.

! Send MEDs.

!!! Only send non-peer routes ! B/C/D/E/K

no route-map out-peer

route-map out-peer permit

!!match community 1 ! B/C/D/E/K

set metric-type internal

!

!!! K: Mark the transit routes with a community.

! We don't do anything in particular with this community, but we could.

!!no route-map from-transit ! K

!!route-map from-transit permit ! K

!!set community 7722:66 additive ! K

!

! Pick up connected routes - we make sure to set the (bgp)

! origin - otherwise it ends up as unknown.

no route-map connected-to-bgp

route-map connected-to-bgp permit

set origin igp

!

! Turn on snmp.

snmp-server community apricot RO

! Send traps if someone uses the wrong snmp community.

snmp-server trap-authentication

! Turn on a pile of snmp traps.

snmp-server enable traps config

snmp-server enable traps envmon

snmp-server enable traps bgp

snmp-server enable traps frame-relay

! Where to send the snmp traps.

snmp-server host 169.223.2.2 apricot

banner motd ^

Apricot GIX Hong Kong

!!A.ix.hk.apricot.net

!!B.ix.hk.apricot.net

!!C.ix.hk.apricot.net

!!D.ix.hk.apricot.net

!!E.ix.hk.apricot.net

!!F.ix.hk.apricot.net

!!K.ix.hk.apricot.net

^

!

line con 0

password jellybean

login

! If you mistype a command, don't turn it into trying to telnet someplace.

transport preferred none

flowcontrol hardware

line aux 0

no exec

login

password jellybean

transport preferred none

transport input telnet

flowcontrol hardware

line vty 0 4

exec-timeout 120 0

login

password jellybean

transport preferred none

!

! When the routers crash, use ftp to send a core dump to this host

! The ftp login/password is at the top of the config.

exception protocol ftp

exception dump 169.222.32.1

! Update the local (hardware) clock with the ntp time.

ntp update-calendar

! Do ntp with the other routers.

! Also do it with an external source.

ntp peer 169.223.0.10

ntp peer 169.223.0.11

ntp peer 169.223.0.12

ntp peer 169.223.0.13

ntp peer 169.223.0.14

ntp peer 169.223.0.15

ntp peer 169.223.0.20

ntp peer 206.111.60.1

end