Note: This column is part of a new semi-regular feature that highlights current events in UIT's network operations group.
By Rosalia Villegas and Jesse Drake
On November 30, 2016, UIT switched off the old Campus Backbone Network (CBN), marking the end of a major undertaking to move all campus and hospital buildings over to new network backbone equipment. This achievement was made possible through the tireless efforts of the network and fiber teams in UIT’s Network Services and Cable Plant teams.
The network backbone links all campus buildings and structures, including the hospital, Downtown Data Center, and the internet. The backbone’s physical network infrastructure includes core switches, routers, outdoor fiber facilities and other networking devices. The project required replacing equipment in more than 200 buildings.
The purpose of the CNB upgrade was three-fold:
- To provide faster data transfer and improved network performance for students, faculty, staff and campus researchers who use computing resources on campus. Tim Urban, senior engineer in Network Services, said the end goal is to increase bandwidth from 1Gb to 10Gb, but all buildings on the new backbone were upgraded to a minimum 2Gb. Internet connection speeds are also expected to gradually increase from 10Gb to 100Gb.
- To take advantage of new, more stable equipment that replaces aging equipment saddled with end-of-support issues.
- To help meet the needs of high-performance, data-intensive research.
If the CNB project could be summed up in one phrase, it would be “pre-planning makes perfect.”
Request for process (RFP), IT Governance approval, and initial design and testing date back to 2013 (see timeline below). Configurations on the switches were built and tested at the University’s Downtown Data Center, then installed at the Eccles Broadcast Center and the Park Building.
Even before mass upgrades could take place early in the project – during which 10 to 15 buildings were cutover in a day with a change window of only a few hours – a considerable amount of communication and coordination took place to ensure an efficient and relatively unobtrusive process.
Given the decentralized nature of the University, simply compiling a concise list of buildings and cognizant IT managers/directors was challenging.
Senior IT Project Manager Earl Lewis and his project team communicated with building managers and placed posters on building entrances alerting people to the planned downtime. Because the cutovers largely took place after regular business hours to minimize user impact, Lewis would also contact the U's Department of Public Safety to make sure the building was accessible to IT techs in the field. Lastly, Lewis had the changes posted on a Network Backbone Upgrade webpage as they were finalized.
“There was a lot of pre-planning by the fiber and network configuration teams before the actual cut,” Urban said. “Assuming that the fiber works and the network is set up properly, the cut goes quick, about 10 minutes, and the building is done. Then we move on to the next building.”
The actual cutover process generally required a team of about 10 UIT employees – some from the fiber team, and some from the network team.
Destry Mulleneaux, technical supervisor of the U’s fiber team, noted that during the first mass moves, the fiber simply had to be re-terminated to a better, cleaner connector.
“There were also cases where the fiber would take a roundabout path around the building, so we would spend some time optimizing the pathway to make it more straight or a shorter distance,” Mulleneaux said.
UIT strives to continuously improve the technology environment at the University. The Campus Network Backbone upgrade is an excellent example of how the organization continues to deliver on its promise of reliable enterprise-grade service.
Campus Backbone Network upgrade timeline
Summary: CBN by the numbers
2013: Design phase/request for proposal (RFP) & IT Governance approval
November 2013 - January 2013: Build and test the new CBN at UIT's Downtown Data Center
February 13, 2014: Migrate first pilot building - Center for High Performance Computing
February 23, 2014: Move new 100G internet connection to new CBN (all traffic going to the internet now goes through new backbone)
March 1, 2014: Move all wireless over to new CBN
March 22, 2014: Move all of student housing node over to new CBN
October 23, 2015: Implement remotely triggered Black Hole Filtering (RFC 5635) on Internet Border Routers in conjunction with UIT's Information Security Office. This security measure is used to deny internet traffic scanning the network for vulnerabilities and known bad hosts at line rate 100G without impacting the network device.
January 2014 - November 2016: Move 200+ buildings to new CBN