View Issue Details

IDProjectCategoryView StatusLast Update
0010827Talerdeployment and operationspublic2025-12-23 04:19
Reportervecirex Assigned Tovecirex  
PriorityimmediateSeveritycrashReproducibilityalways
Status closedResolutionfixed 
Summary0010827: TOPS/CHF test instance taler.hacktivism.ch down
Description...just realized a few hours ago; checking what's going on.

While I'm monitoring my.taler-ops.ch to be certainly online, I didn't do the same for taler.hacktivism.ch, which is not highly problematic, as it's test and not prod, but still not nice if someone wants to test.
TagsNo tags attached.

Activities

vecirex

2025-12-23 00:55

manager   ~0027172

On it.

vecirex

2025-12-23 01:48

manager   ~0027173

Last edited: 2025-12-23 02:04

Oups, container intentionally closed for maintenance (base sys upgrade), neither upgrading base sys by then nor starting the container... ehm, yeah; as seen in screen session -- sorry for downtime, sleep helps, also now! :)

"hernani@koopa:~> podman stop taler-hacktivism
WARN[0010] StopSignal SIGTERM failed to stop container taler-hacktivism in 10 seconds, resorting to SIGKILL
taler-hacktivism
hernani@koopa:~> podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
hernani@koopa:~>"

vecirex

2025-12-23 04:18

manager   ~0027175

Last edited: 2025-12-23 04:19

0.
taler-merchant DB copies avail by pg_dump:

"taler-merchant-httpd@bcead3ae4f5e:/usr/local/bin$ ls -l /var/taler-backups/*
-rw-r--r-- 1 taler-merchant-httpd www-data 2141326 Nov 18 00:07 /var/taler-backups/20251118--taler-merchant-db
.sql |
-rw-r--r-- 1 taler-merchant-httpd www-data 2246099 Dec 3 11:40 /var/taler-backups/20251203--taler-merchant-db
.sql
-rw-r--r-- 1 taler-merchant-httpd www-data 2283635 Dec 13 04:59 /var/taler-backups/20251213--taler-merchant-db
.sql
-rw-r--r-- 1 taler-merchant-httpd www-data 2296238 Dec 15 14:35 /var/taler-backups/20251215--taler-merchant-db
.sql
-rw-r--r-- 1 taler-merchant-httpd www-data 2306938 Dec 16 20:26 /var/taler-backups/20251216--taler-merchant-db
.sql
-rw-r--r-- 1 taler-merchant-httpd www-data 2381065 Dec 23 02:23 /var/taler-backups/20251223--taler-merchant-db
.sql "

1a. As root:
zypper update # many packages
reboot

1b. Troubleshooting
No networked services were reachable whatsoever first; leading to some rabbit holes first, in any case having to access server locally, finally becoming clear firewall system got changed [restored] to all things restrictive; this was needed, applying to both IPv4/IPv6 inet:

firewall-cmd --add-service=ssh --permanent # SSH
firewall-cmd --add-port=tcp/443 --permanent # Taler running unprivileged on tcp/8081: unexposed; internal port fwd w/ systemd from tcp 443 to 8081
firewall-cmd --add-port=tcp/8080 --permanent # Tor relay: entry guard / middle node
firewall-cmd --reload

2a.
podman start taler-hacktivism

Site available again.

2b.
Important test of SMS and email sending + check reception (importat for login: one additional factor needed despite credentials; important for sign-up / change PW as onboarded merchant: both additional factors needed);

testing email:

"taler-merchant-httpd@bcead3ae4f5e:/usr/local/bin$ echo "Test for TOPS/CHF testing infra" | ./taler-hacktivism-
email-helper.sh hernani+test@vecirex.net
[...]
<~ 235 2.0.0 OK
 ~> MAIL FROM:<taler-merchant@hacktivism.ch>
<~ 250 OK
 ~> RCPT TO:<hernani+test@vecirex.net>
<~ 250 Accepted
 ~> DATA
<~ 354 Enter message, ending with "." on a line by itself
 ~> Date: Tue, 23 Dec 2025 03:12:47 +0000
 ~> To: hernani+test@vecirex.net
 ~> From: taler-merchant@hacktivism.ch
 ~> Subject: Taler Merchant Auth Code
 ~> Message-Id: <20251223031247.000698@bcead3ae4f5e>
 ~> X-Mailer: swaks v20240103.0 jetmore.org/john/code/swaks/
 ~>
 ~> Test for TOPS/CHF testing infra
 ~>
 ~>
 ~> .
<~ 250 OK id=1vXspL-0000000H2qd-2zq2
 ~> QUIT
<~ 221 s016.cyon.net closing connection"

testing SMS:

taler-merchant-httpd@bcead3ae4f5e:/usr/local/bin$ echo "Test for TOPS/CHF testing infra" | ./taler-hacktivism-
sms-helper-wrapper.sh +41791912370 # public phone no.
Polling message status (reference_id: 366D24CEE8E809689197910CF9CF4F32)...
Delivered to headset
taler-merchant-httpd@bcead3ae4f5e:/usr/local/bin$

3.
AFK (sleep++)

vecirex

2025-12-23 04:18

manager   ~0027176

Close it!

Issue History

Date Modified Username Field Change
2025-12-23 00:54 vecirex New Issue
2025-12-23 00:55 vecirex Assigned To => vecirex
2025-12-23 00:55 vecirex Status new => assigned
2025-12-23 00:55 vecirex Note Added: 0027172
2025-12-23 01:48 vecirex Note Added: 0027173
2025-12-23 01:48 vecirex Note Edited: 0027173
2025-12-23 02:04 vecirex Note Edited: 0027173
2025-12-23 04:18 vecirex Note Added: 0027175
2025-12-23 04:18 vecirex Status assigned => closed
2025-12-23 04:18 vecirex Resolution open => fixed
2025-12-23 04:18 vecirex Note Added: 0027176
2025-12-23 04:19 vecirex Note Edited: 0027175