View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0008302 | Taler | deployment and operations | public | 2024-02-01 09:17 | 2026-09-19 14:40 |
| Reporter | javier.sepulveda | Assigned To | Florian Dold | Resolution deadline | |
| Implementer | Reviewer | ||||
| Release signed by | Deployed by | ||||
| Verifier | Communicator | ||||
| Data protection authority notification required | No | ||||
| CSIRT notification required | No | ||||
| Estimated effort (hours) | Actual effort (hours) | 0 | |||
| Fast-track | No | ||||
| Work package | none | ||||
| Priority | high | Severity | text | Reproducibility | have not tried |
| Status | informing | Resolution | reopened | ||
| Product Version | git (master) | ||||
| Target Version | 1.6 | Fixed in Version | 1.6 | ||
| Summary | 0008302: Current monitoring tools - Write documentation | ||||
| Description | At this point we have currently up and running: Netdata, Grafana and Kuma. So write documentation about what exactly is being monitored by each program. | ||||
| Tags | No tags attached. | ||||
| Agreed to create a completely new .rst file under docs.git | |
| Doing quite good progress with drafting of this document. Right now I do have quite a nice skeleton of it, and are getting into the detail of each section. | |
| Pushed to git the reviewed Draft. Still need to be linked somewhere in the --menu. Even though I refer in this document to Loki and Promotail, yet, we don't have them installed on taler.net (loki+promtail). Installing those packages is easy, but configuring the logs is another story. I need to --learn first how to manage Loki+Promtail through the Grafana admin panel, and after that learn how to track specific logs. Nowadays, there is different and opened bug for that 0008303. | |
| Did you forget to Git add images? /research/taler/docs/taler-monitoring-infrastructure.rst:24: WARNING: image file not readable: images/taler-monitoring-infrastructure.png /research/taler/docs/taler-monitoring-infrastructure.rst:91: WARNING: image file not readable: images/grafana-postgres-exporter.png /research/taler/docs/taler-monitoring-infrastructure.rst:108: WARNING: image file not readable: images/uptime-kuma-from-grafana.png /research/taler/docs/taler-monitoring-infrastructure.rst:170: WARNING: image file not readable: images/kuma.png /research/taler/docs/taler-monitoring-infrastructure.rst:197: WARNING: image file not readable: images/uptime-kuma-edit.png | |
| Yes. I realized that yesterday and added to my caldav tasks. I will do that in not time. Thank you. | |
| I have checked this, as I was sure I had copied those images to the --images folder beforehand. So I thought doing "git add ." would have include them. Now I can see, the .gitignore is avoiding to track these .png files. # generated images *.png So, no idea what's the plan now. As until I can't change the .gitignore file, I won't able to commit to add these images files. I can find my way to upload them manually but I am not sure if this what we want to do forever, when adding image files on the rst files. | |
| Added missing image files by using: git add -f I will add that to my notes, to take that into account for the next time. | |
| Reopening now to give feedback on both documentation and the setup. 1) In the main figure on top, it would be good to consistently have the real hostnames (not: TALER OPS, not: "external server") in the architecture diagram. Also, we have 6 internal machines: firefly.gnunet.org (BFH), sam.gnunet.org (TUM), gv.taler.net (BFH), $tbd.taler.net (TUE), anastasis.lu (contabo) and taler-ops.ch (hosttech). They are at 5 different hosters -- that should be *plenty* to run the uptime kuma servers _within_ our own infrastructure instead of depending on any external hosts. I propose we run one at TUE, and a second one at contabo. Also, we should monitor _all_ of our hosts, not just taler.net and taler-ops.ch. In fact, we should now *also* start monitoring exchange.e.netzbon-basel.ch ;-). The diagram should also say where grafana is hosted. 2) Grafana dashboards: Marco Boss had developed some taler-specific Grafana dashboards as part of this BS thesis. You can find them in the grid5k.git, and probably should unearth and deploy them (and then document that). 3) The Node exporter, Postgres exporter and Prometheus seem perfect targets for an Ansible playbook for taler-ops.ch (and in the future other hosts). 4) Managing logs: it is totally unclear from what you write *where* this is done. | |
| Javier, are you working on this? Please try to address this for 0.13! | |
| There's now basic documentation for the monitoring setup in taler-docs.git and more extensive documentation in tsys-infra.git/monitoring. | |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2024-02-01 09:17 | javier.sepulveda | New Issue | |
| 2024-02-01 09:17 | javier.sepulveda | Status | new => assigned |
| 2024-02-01 09:17 | javier.sepulveda | Assigned To | => javier.sepulveda |
| 2024-02-07 13:18 | javier.sepulveda | Note Added: 0021193 | |
| 2024-02-14 13:16 | javier.sepulveda | Note Added: 0021290 | |
| 2024-02-14 13:28 | Christian Grothoff | Severity | minor => text |
| 2024-02-14 13:28 | Christian Grothoff | Target Version | => 0.11 |
| 2024-03-05 12:56 | javier.sepulveda | Note Added: 0021672 | |
| 2024-03-05 12:56 | javier.sepulveda | Assigned To | javier.sepulveda => Christian Grothoff |
| 2024-03-05 12:56 | javier.sepulveda | Status | assigned => feedback |
| 2024-03-07 08:29 | Christian Grothoff | Note Added: 0021746 | |
| 2024-03-07 08:30 | Christian Grothoff | Assigned To | Christian Grothoff => javier.sepulveda |
| 2024-03-07 08:32 | javier.sepulveda | Note Added: 0021747 | |
| 2024-03-07 11:53 | javier.sepulveda | Note Added: 0021755 | |
| 2024-03-11 08:08 | javier.sepulveda | Note Added: 0021832 | |
| 2024-03-12 08:03 | javier.sepulveda | Status | feedback => informing |
| 2024-03-12 08:03 | javier.sepulveda | Resolution | open => fixed |
| 2024-03-12 12:46 | Christian Grothoff | Product Version | => git (master) |
| 2024-03-12 12:46 | Christian Grothoff | Fixed in Version | => 0.10 |
| 2024-03-12 12:46 | Christian Grothoff | Target Version | 0.11 => 0.10 |
| 2024-03-12 12:48 | Christian Grothoff | Assigned To | javier.sepulveda => Christian Grothoff |
| 2024-03-12 12:48 | Christian Grothoff | Status | informing => feedback |
| 2024-03-12 12:48 | Christian Grothoff | Resolution | fixed => reopened |
| 2024-03-12 12:59 | Christian Grothoff | Note Added: 0021864 | |
| 2024-03-13 16:19 | Christian Grothoff | View Status | private => public |
| 2024-03-18 11:19 | Christian Grothoff | Note Edited: 0021864 | |
| 2024-03-18 11:19 | Christian Grothoff | Assigned To | Christian Grothoff => javier.sepulveda |
| 2024-03-18 11:20 | Christian Grothoff | Status | feedback => assigned |
| 2024-04-12 16:10 | Christian Grothoff | Priority | normal => high |
| 2024-04-12 16:10 | Christian Grothoff | Fixed in Version | 0.10 => |
| 2024-04-12 16:10 | Christian Grothoff | Target Version | 0.10 => 0.11 |
| 2024-05-13 19:22 | Christian Grothoff | Target Version | 0.11 => 0.12 |
| 2024-06-26 16:08 | Christian Grothoff | Target Version | 0.12 => 0.13 |
| 2024-06-26 16:08 | Christian Grothoff | Note Added: 0022730 | |
| 2024-07-25 22:35 | Christian Grothoff | Target Version | 0.13 => 0.14 |
| 2024-09-07 15:24 | Christian Grothoff | Assigned To | javier.sepulveda => |
| 2024-09-07 15:24 | Christian Grothoff | Status | assigned => implementing |
| 2024-09-07 15:24 | Christian Grothoff | Target Version | 0.14 => 1.0 |
| 2024-12-29 16:22 | dvn | Assigned To | => dvn |
| 2024-12-29 16:22 | dvn | Status | implementing => assigned |
| 2025-02-02 23:00 | Christian Grothoff | Target Version | 1.0 => 1.0 stretch goals |
| 2025-04-25 05:17 | Christian Grothoff | Target Version | 1.0 stretch goals => post-1.0 |
| 2025-06-21 21:00 | Christian Grothoff | Assigned To | dvn => |
| 2025-06-21 21:00 | Christian Grothoff | Status | assigned => implementing |
| 2026-09-11 13:34 | Florian Dold | Assigned To | => Florian Dold |
| 2026-09-11 13:34 | Florian Dold | Status | implementing => informing |
| 2026-09-11 13:34 | Florian Dold | Note Added: 0029952 | |
| 2026-09-19 14:38 | Christian Grothoff | Target Version | post-1.0 => 1.6 |
| 2026-09-19 14:40 | Christian Grothoff | Fixed in Version | => 1.6 |