View Issue Details

IDProjectCategoryView StatusLast Update
0004469Talerdeployment and operationspublic2016-04-28 22:16
ReporterFlorian Dold Assigned ToFlorian Dold  
PrioritynormalSeverityfeatureReproducibilityhave not tried
Status closedResolutionfixed 
Product Version0.0 
Target Version0.0Fixed in Version0.0 
Summary0004469: blue-green deployments for demo.taler.net
DescriptionRight now, when we update demo.taler.net we have two problems:
- downtime during the actual update
- when something goes terribly wrong, we can't go back

Thus we should adopt green-blue deployments, where we have two copies of the demo environment. While updating one of them (e.g. green.demo.taler.net) the other one (e.g. blue.demo.taler.net) still serves traffic. When we're done updating, we switch over to green.demo.taler.net. If something goes horribly wrong, we simply switch back to blue.demo.taler.net while we're fixing the problem (or we could decide to just postpone the upgrade).
TagsNo tags attached.

Activities

Florian Dold

2016-04-24 18:04

manager   ~0010549

(see http://martinfowler.com/bliki/BlueGreenDeployment.html)

Florian Dold

2016-04-24 18:50

manager   ~0010550

This is trivial to implement in theory, but (the free version of) NGINX does not support dynamic upstream reconfiguration. So we'll have to find some other component to do that for us.

Florian Dold

2016-04-24 20:37

manager   ~0010551

Since we're using uWSGI anyway, we could use uWSGI for dynamic upstream configuration, since it has decent support for it:

http://uwsgi-docs.readthedocs.org/en/latest/Fastrouter.html
http://uwsgi-docs.readthedocs.org/en/latest/InternalRouting.html#http

uWSGI has lots of different ways to do dynamic reconfiguration, but the simplest is just a directory with hostname->upstream mappings.

Florian Dold

2016-04-24 20:43

manager   ~0010552

Last edited: 2016-04-24 20:55

Alternatively we could just use a per-user nginx instance (that can be reconfigured without touching the root nxing).

Though this is less convenient, since we always need to restart the nginx process, while uWSGI supports dynamic switching.

Florian Dold

2016-04-27 01:54

manager   ~0010562

There's a /home/{demo,test]/sockets directory now. This links to one of /home/{demo,test}-{blue,green}/sockets.

The socket directory contains the uwsgi/http sockets for merchant, exchange, donations, blog, landing.

Pointing {test,demo}.taler.net to either blue or green is done by simply updating this symlink.

Florian Dold

2016-04-27 01:55

manager   ~0010563

Note that we're not using any active switching / load balancing, nginx simply points to the unix-domain sockets in the symlinked directory.

Issue History

Date Modified Username Field Change
2016-04-24 18:01 Florian Dold New Issue
2016-04-24 18:03 Florian Dold Assigned To => Marcello Stanisci
2016-04-24 18:03 Florian Dold Severity minor => feature
2016-04-24 18:03 Florian Dold Status new => assigned
2016-04-24 18:04 Florian Dold Note Added: 0010549
2016-04-24 18:50 Florian Dold Note Added: 0010550
2016-04-24 20:37 Florian Dold Note Added: 0010551
2016-04-24 20:43 Florian Dold Note Added: 0010552
2016-04-24 20:55 Florian Dold Note Edited: 0010552
2016-04-24 23:38 Florian Dold Description Updated
2016-04-27 01:54 Florian Dold Note Added: 0010562
2016-04-27 01:54 Florian Dold Status assigned => resolved
2016-04-27 01:54 Florian Dold Resolution open => fixed
2016-04-27 01:54 Florian Dold Assigned To Marcello Stanisci => Florian Dold
2016-04-27 01:55 Florian Dold Note Added: 0010563
2016-04-28 22:16 Christian Grothoff Status resolved => closed
2016-04-28 22:16 Christian Grothoff Product Version => 0.0
2016-04-28 22:16 Christian Grothoff Fixed in Version => 0.0
2016-04-28 22:16 Christian Grothoff Target Version => 0.0