News: UNMETERED - WebSite Outages

Published: 2016.03.30 12:19:00

Corporate websites and billing portals became unavailable a number of times recently ; during the first rounds of these outages our servers were unaffected but routes serving our primary web pools were diverted ; subsequent outages were caused by maintenance to upgrade our core management infrastructure. Everything has been corrected : no data was at risk during these outages ; design changes for a more seamless experience even during such occurances are being implemented incrementally.

1st Outage - Start/Fix Time: ~9AM Feb 9, 2016 to 11:48AM Feb 9, 2016
Reason : Line Card Failure @ TELUS Communications / Action Taken : Reset Line Card

2nd Outage - Start/Fix Time: ~11PM Feb 9, 2016 to 8:28AM Feb 10, 2016
Reason : SQL Cluster Desyncronization / Action Taken : Reset SQL Cluster

3nd Outage - Start/Fix Time: ~11AM Feb 10, 2016 to 4:20PM Feb 10, 2016
Reason : Line Card Failure @ TELUS Communications / Action Taken : Replaced Line Card

4th Outage - Start/Fix Time: ~11PM Mar 1, 2016 to 7:57AM Mar 2, 2016
Reason: SQL Cluster Migration / Action Taken : Move from Burnaby-Only Cluster to Multi-National Redundant Cluster

5th (hopefully final) Outage - Start/Fix Time : ~10PM Mar 27, 2016 to ~11AM Mar 29, 2016 (some site sections)
Reason : nginx Server Configuration / Action Taken : Contract TechFutures.co to create & maintain Canada.UNMETERED.WebSite
Notes : Monitoring with 2-min failover to USA.UNMETERED.WebSite (via CloudFlare) for all UNMETERED.IO web properties

Time Zone : Pacific Standard Time

Services Affected
- All UNMETERED.IO Web Portals (www.unmetered.io & unmetered.io frontend websites)
- UNMETERED.Chat (Dashboard)

ALL Partner-Delivered Services Not Affected

Long Term Resolution

Combining Server Health Checking with a Failover DNS record & our pre-existing cross-facility SQL cluster : Web Interfaces should remain unaffected during Burnaby outages with the switchover taking ~ 30 seconds. Cross-facility clustering and frontend hosting has been permanently setup ( failover via VPSes for life @ www.CloudAtCost.com ).