View Issue Details

IDProjectCategoryView StatusLast Update
0004469Talerdeployment and operationspublic2016-04-28 20:16
ReporterFlorian Dold Assigned ToFlorian Dold Resolution deadline
ImplementerReviewer 
Release signed byDeployed by 
VerifierCommunicator 
Data protection authority notification requiredNo  
CSIRT notification requiredNo  
Estimated effort (hours)Actual effort (hours)0 
Fast-trackNo
Work packagenone
PrioritynormalSeverityfeatureReproducibilityhave not tried
Status closedResolutionfixed 
Product Version0.0 
Target Version0.0Fixed in Version0.0 
Summary0004469: blue-green deployments for demo.taler.net
DescriptionRight now, when we update demo.taler.net we have two problems:
- downtime during the actual update
- when something goes terribly wrong, we can't go back

Thus we should adopt green-blue deployments, where we have two copies of the demo environment. While updating one of them (e.g. green.demo.taler.net) the other one (e.g. blue.demo.taler.net) still serves traffic. When we're done updating, we switch over to green.demo.taler.net. If something goes horribly wrong, we simply switch back to blue.demo.taler.net while we're fixing the problem (or we could decide to just postpone the upgrade).
TagsNo tags attached.

Required notifications

Address or linkRemarkAdded byNotifiedActions

Activities

Florian Dold

2016-04-24 16:04

manager   ~0010549

(see http://martinfowler.com/bliki/BlueGreenDeployment.html)

Florian Dold

2016-04-24 16:50

manager   ~0010550

This is trivial to implement in theory, but (the free version of) NGINX does not support dynamic upstream reconfiguration. So we'll have to find some other component to do that for us.

Florian Dold

2016-04-24 18:37

manager   ~0010551

Since we're using uWSGI anyway, we could use uWSGI for dynamic upstream configuration, since it has decent support for it:

http://uwsgi-docs.readthedocs.org/en/latest/Fastrouter.html
http://uwsgi-docs.readthedocs.org/en/latest/InternalRouting.html#http

uWSGI has lots of different ways to do dynamic reconfiguration, but the simplest is just a directory with hostname->upstream mappings.

Florian Dold

2016-04-24 18:43

manager   ~0010552

Last edited: 2016-04-24 18:55

Alternatively we could just use a per-user nginx instance (that can be reconfigured without touching the root nxing).

Though this is less convenient, since we always need to restart the nginx process, while uWSGI supports dynamic switching.

Florian Dold

2016-04-26 23:54

manager   ~0010562

There's a /home/{demo,test]/sockets directory now. This links to one of /home/{demo,test}-{blue,green}/sockets.

The socket directory contains the uwsgi/http sockets for merchant, exchange, donations, blog, landing.

Pointing {test,demo}.taler.net to either blue or green is done by simply updating this symlink.

Florian Dold

2016-04-26 23:55

manager   ~0010563

Note that we're not using any active switching / load balancing, nginx simply points to the unix-domain sockets in the symlinked directory.

Issue History

Date ModifiedUsernameFieldChange
2016-04-24 16:01Florian DoldNew Issue
2016-04-24 16:03Florian DoldAssigned To => Marcello Stanisci
2016-04-24 16:03Florian DoldSeverityminor => feature
2016-04-24 16:03Florian DoldStatusnew => assigned
2016-04-24 16:04Florian DoldNote Added: 0010549
2016-04-24 16:50Florian DoldNote Added: 0010550
2016-04-24 18:37Florian DoldNote Added: 0010551
2016-04-24 18:43Florian DoldNote Added: 0010552
2016-04-24 18:55Florian DoldNote Edited: 0010552
2016-04-24 21:38Florian DoldDescription Updated
2016-04-26 23:54Florian DoldNote Added: 0010562
2016-04-26 23:54Florian DoldStatusassigned => informing
2016-04-26 23:54Florian DoldResolutionopen => fixed
2016-04-26 23:54Florian DoldAssigned ToMarcello Stanisci => Florian Dold
2016-04-26 23:55Florian DoldNote Added: 0010563
2016-04-28 20:16Christian GrothoffStatusinforming => closed
2016-04-28 20:16Christian GrothoffProduct Version => 0.0
2016-04-28 20:16Christian GrothoffFixed in Version => 0.0
2016-04-28 20:16Christian GrothoffTarget Version => 0.0