View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0009607 | Taler | deployment and operations | public | 2025-03-10 22:10 | 2025-05-10 02:11 |
| Reporter | Florian Dold | Assigned To | Christian Grothoff | Resolution deadline | |
| Implementer | Reviewer | ||||
| Release signed by | Deployed by | ||||
| Verifier | Communicator | ||||
| Data protection authority notification required | No | ||||
| CSIRT notification required | No | ||||
| Estimated effort (hours) | Actual effort (hours) | 0 | |||
| Fast-track | No | ||||
| Work package | none | ||||
| Priority | normal | Severity | minor | Reproducibility | have not tried |
| Status | closed | Resolution | fixed | ||
| Target Version | 1.0 | Fixed in Version | 1.0 | ||
| Summary | 0009607: taler-ansible-exchange should support backup+restore of DBs | ||||
| Description | * When a backup (and possibly some file to flag "please do import") is present, the exchange playbook should import the backup * Exporting a backup should probably be its own playbook * Backups should be done via pg_dump * The playbook should refuse to import a backup if the database already exists (by looking at the _v schema) | ||||
| Tags | No tags attached. | ||||
| child of | 0008567 | closed | Christian Grothoff | Write Ansible playbook to deploy - Taler exchange (towards the taler-ops server) |
| dvn: any progress here? | |
| Christian: please take a look at the latest changes, and if you can, give it a try. Any feedback welcome. | |
| Well, the backup is Ok for a one-off, but *usually* we'd want to rely on the borg backup that should already be installed and deployed and run automatically (instead of a one-off manual backup). So I'd have expected to at least additionally see some shell script to extract taler-exchange-backup.sql.xz from the borg backup (plus I think there we currently use a different compression algo, but I'm OK with switching -- but if we switch, we should do so consistently and probably NOT use xz (see https://www.nongnu.org/lzip/xz_inadequate.html). I'd be happy with bzip2). On the restore-from-backup logic, I'm a bit confused: you delete that 'ansible_local.taler_backup_import' fact, but where does it come from in the first place? I see some logic that tests the database (ansible_local.taler_backup_import is defined and database_info.databases == "taler-exchange", or when: ansible_local.taler_backup_import is defined and DATABASE_EXISTS.statusmessage is defined and DATABASE_EXISTS.statusmessage == "SELECT 1") which seems related, but I had expected to see the fact being set by checking if the taler-exchange database has a schema "_v" with say "exchange-0001" in patches.patch_name. So SELECT 1 FROM _v.patches WHERE patch_name='exchange-0001'; would seem like a reasonable check to detect that we should NOT restore a backup. But I don't see that, so confusing. Similarly, I didn't see a test where you'd check that /tmp/taler-exchange-backup.sql.xz actually exists (is this a file that must exist on the server, or locally?), so I'm not sure the error is handled cleanly when the file does not exist. Also, is "target:" here a file locally on the machine of the uploader, or already on the target host? If on the target host, where is the ansible logic to upload the backup to the server? | |
| Okay, so you're thinking that the administrator should be uploading the backup that they have already retrieved from the backup server, is that right? My thought was that the administrator upload the backup to the target server at `/tmp/taler-exchange-backup.sql.xz`, and create an empty file at `/etc/ansible/facts.d/taler_backup_import.fact`. Creating this empty file will signal the backup jobs to run. The next job will check if `_v` exists and if it has any value, and if it does then it will bail out with a failure. If the backup succeeds then the empty "fact" file is removed, so that the server is in "normal" state so that subsequent runs of the ansible-playbook will not trigger the import jobs. By the way, AFAICT we are using xz as the compression format in all of the backup scripts. | |
| Yes, plus the fact that the admin has shell-scripts at-the-ready to do the extraction of the backup. The administrator should not "magically" create backup files or facts, that should IMO all be done by the playbook. I should only have to tell the playbook *here* is the backup, and it should then do the upload. xz: well, AFAIK not the best choice... | |
| Oh, and we might actually want to prevent the admin from accidenttaly NOT providing a backup to restore if the server doesn't already have a DB. Because it should be super-rare that the admin sets up a fresh server, we might want to expect a special ansible variable "do-fresh-setup" or something like this to be set to *allow* ansible to proceed in this situation. After all, common cases are either 'upgrade running system', or 'restore running system from backup', and very rarely (in production) 'deploy from scratch'. | |
| Fixed & tested now. | |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2025-03-10 22:10 | Florian Dold | New Issue | |
| 2025-03-10 22:10 | Florian Dold | Status | new => assigned |
| 2025-03-10 22:10 | Florian Dold | Assigned To | => Florian Dold |
| 2025-03-21 13:34 | Florian Dold | Assigned To | Florian Dold => dvn |
| 2025-04-04 05:29 | Christian Grothoff | Relationship added | child of 0008567 |
| 2025-04-04 05:29 | Christian Grothoff | Note Added: 0024397 | |
| 2025-04-10 23:42 | dvn | Note Added: 0024505 | |
| 2025-04-14 14:57 | Christian Grothoff | Note Added: 0024533 | |
| 2025-04-14 15:13 | dvn | Note Added: 0024536 | |
| 2025-04-14 15:22 | Christian Grothoff | Note Added: 0024538 | |
| 2025-04-14 15:25 | Christian Grothoff | Note Added: 0024539 | |
| 2025-04-20 09:44 | Christian Grothoff | Assigned To | dvn => Christian Grothoff |
| 2025-04-20 13:37 | Christian Grothoff | Status | assigned => informing |
| 2025-04-20 13:37 | Christian Grothoff | Resolution | open => fixed |
| 2025-04-20 13:37 | Christian Grothoff | Fixed in Version | => 1.0 |
| 2025-04-20 13:37 | Christian Grothoff | Note Added: 0024623 | |
| 2025-05-10 02:11 | Christian Grothoff | Status | informing => closed |