We'd like to keep Spreaker up 100% of the time. When that doesn't happen, we write about it here.
|Website||So far so good|
|Api||So far so good|
|Streaming||So far so good|
|Mobile apps||So far so good|
We just resolved an issue with our Tube service that affected many US customers. We’re really sorry for the inconvenience and we’re working hard to avoid the same issue in the future.
Few months ago we switched our traffic to a new high-available and fault-tolerant recording infrastructure. Once of the core components of this infrastructure is the so called “balancer”. The balancer is the entry point for each recording connection and routes the traffic to the nearest available recording server.
Unfortunately, last night one of these balancers stopped to work (after 91 days of uptime) in a bad way, so that the health check was OK, but balancer was unable to route requests.
We’re currently working to investigate the root cause of the issue and detect such condition during the health checks, so that if it happens again in the future we’ll be able to automatically recover it.
Sorry again for the inconveniente. If you have any further question, don’t hesitate to contact us at http://help.spreaker.com
UPDATE (Oct, 3rd)
We investigated the issue and it looks was a bug in a client library we use to connect to Redis. We just pushed a new balancer version to production, that includes two changes: