HUIT Service Status Dashboard
Network outage at multiple locations
Incident
Report for HUIT
Resolved
As of Friday Aug 9 at 5pm, HUIT Network team were able to restore normal operations at the affected locations and network services have been stable. Work continues to fully restore the Longfellow building.
Update
HUIT Network and Harvard Phone teams continue to implement the staged recovery plan, extending cables to new places and restoring telecom service lines. The work has been running smoothly and temporary provided power seems to be stable.
Update
HUIT Network and Telecom teams continue to implement the staged recovery plan.
Network team put together a map for fiber re-cabling and a bunch of fiber cables are currently being deployed across different locations. Verizon were able to restore the system and at least half of the lines are back in service. HUIT is actively working to restore normal operations in the affected locations.
Network team put together a map for fiber re-cabling and a bunch of fiber cables are currently being deployed across different locations. Verizon were able to restore the system and at least half of the lines are back in service. HUIT is actively working to restore normal operations in the affected locations.
Update
The temporary power in Longfellow building has been working and helped to restore network connections to several affected locations. HUIT network and telecom teams are implementing a staged recovery plan and actively continue to work on restoring normal operations in the affected locations.
Update
HUIT network team managed to provide temporary power in Longfellow building (through a cable pulled from Radcliffe) to get the fiber distribution to other buildings up. This is on a best effort basis and could go down anytime, therefore recommend affected schools to plan around that.
In addition, the network team has a plan of action in place to get everything back up, however in order to implement it, further investigation and permits are required.
Next update at 11:00 am tomorrow should provide better insights into the recovery plan.
In addition, the network team has a plan of action in place to get everything back up, however in order to implement it, further investigation and permits are required.
Next update at 11:00 am tomorrow should provide better insights into the recovery plan.
Update
We were granted access to the facility at Longfellow, techs from Verizon, HUIT Network and WireCable, and HUIT Telecom Field support are on-site assessing the gear. We are expecting potential equipment and cable repair work.
Identified
Crews have been working all night to dewater the Longfellow subbasement. At this point, the water is at a manageable level and Harvard Engineering and Utilities (E&U) is planning to safely power down the Longfellow and Larsen main power feeds so a full assessment of the area can be completed. In order to safely power down Longfellow and Larsen, the crew will need to shut down the power to the GSE campus. The power shutdown is expected to take 1 hour. The Gutman Datacenter will be offline for the duration of the outage window and the following services will be unavailable:
• On-campus network access
• Network Printing
• XDrive ( X: drive)
• Windows network shares (G: and H: drives)
• GSE websites (Maintenance page in place)
• Event reservation system (EMS)
• GSE Mailing Lists
• GRID
• Admissions Tool
• PowerFAIDS/NetPartner
• ScratchED website
• Wireless Network
• On-campus network access
• Network Printing
• XDrive ( X: drive)
• Windows network shares (G: and H: drives)
• GSE websites (Maintenance page in place)
• Event reservation system (EMS)
• GSE Mailing Lists
• GRID
• Admissions Tool
• PowerFAIDS/NetPartner
• ScratchED website
• Wireless Network
Update
The Engineering and Utilities Department estimate to get in the Longfellow building around 12 - 1 pm to assess damage to power and make the building safe. Water is still being pumped out. Further updates on progress expected around 10:00 am.
Update
Work continues at Longfellow, but it appears the pipes are no longer broken. GSE facilities are currently in process of cleaning up the areas. Access is still restricted.
Impact:
All of Radcliffe campus & A.R.T will be without network services until the room can be restored (Estimated another 24 hours). Partial network service outage at Graduate School of Education & Division of Continuing Education. Some buildings in the area have a redundant network path from Gutman to 8 Story Street and those will remain connected. Verizon emergency dial tone to those buildings is offline
Preparations made:
Spectrum (cable vendor) is on standby to come inspect and evaluate the cable infrastructure when the all clear is given. The Network team is performing a large switch software upgrade at the Law School this evening, and are ready to split the maintenance work if the all clear comes before 3 AM or they will pivot from Law to GSE (Longfellow) when the upgrade is completed. They also have some team members planning to arrive early tomorrow morning. Voice team is coordinating with Verizon to have a team onsite as soon as all clear is given
This will be the last update until the morning unless something significant changes.
Impact:
All of Radcliffe campus & A.R.T will be without network services until the room can be restored (Estimated another 24 hours). Partial network service outage at Graduate School of Education & Division of Continuing Education. Some buildings in the area have a redundant network path from Gutman to 8 Story Street and those will remain connected. Verizon emergency dial tone to those buildings is offline
Preparations made:
Spectrum (cable vendor) is on standby to come inspect and evaluate the cable infrastructure when the all clear is given. The Network team is performing a large switch software upgrade at the Law School this evening, and are ready to split the maintenance work if the all clear comes before 3 AM or they will pivot from Law to GSE (Longfellow) when the upgrade is completed. They also have some team members planning to arrive early tomorrow morning. Voice team is coordinating with Verizon to have a team onsite as soon as all clear is given
This will be the last update until the morning unless something significant changes.
Investigating
Due to watermain burst/power cut to the Longfellow building, network services are down affecting users in Radcliffe, GSE, ART, and other buildings.