View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0011646Talerlibeufin-bankpublic2026-07-19 15:182026-07-19 15:29
Reportervecirex Assigned To 
PrioritynoneSeverityminorReproducibilitysometimes
Status newResolutionopen 
Summary0011646: Running dozens of withdraws w/ very high amounts can make the bank throw 500 / SQL errors
DescriptionExample, cf. for more https://taler.hacktivism.ch/monitoring-max-ladder/

╔══════════════════════════════════╗
║ ladder · rung 7  GOA:37 · 37 GOA ║
╚══════════════════════════════════╝
┌ 63%┐ █████████████████░░░░░░░░░░░  76/120
┌ OK     ┐ #076 ladder.withdraw-48  mint GOA:37 (e9edaca8-2af7-41f5-be11-2ff6ca899b9e) 238ms
┌ OK     ┐ #077 ladder.withdraw-49  accept-uri GOA:37 2033ms
┌ INFO   ┐ #078 ladder.withdraw-50  force-select · HTTP 200 rpub=ZZJSAQGF6BBQ… (ok for WID e9edaca8)
┌ INFO   ┐ #079 ladder.withdraw-51  confirm · HTTP 204 after selected (poll 2)
┌ OK     ┐ #080 ladder.withdraw-52  confirm GOA:37 2551ms (client, on selected)
┌ OK     ┐ #081 ladder.withdraw-53  settle GOA:37 bank transfer_done (wallet avail=GOA:0, 877ms)
┌ INFO   ┐ #082 ladder.withdraw-54  coins ladder-r7-after-GOA_37 · n=0  in_circulation=0  spent=0  amount_circ=0 GOA (GOA:0)
┌ INFO   ┐ #083 ladder.withdraw-55  coins ladder-r7-after-GOA_37 · denoms_in_circ (none)
┌ 70%┐ ███████████████████░░░░░░░░░  84/120
┌ INFO   ┐ #084 ladder.withdraw-56  coins ladder-r7-after-GOA_37 Δ · new=+0  total_now=0  |  (no denom change)

╔══════════════════════════════════╗
║ ladder · rung 8  GOA:61 · 61 GOA ║
╚══════════════════════════════════╝
┌ OK     ┐ #085 ladder.withdraw-57  mint GOA:61 (cfa99aa3-15e3-47b1-bc62-40c5de8489e4) 276ms
┌ OK     ┐ #086 ladder.withdraw-58  accept-uri GOA:61 3602ms
┌ INFO   ┐ #087 ladder.withdraw-59  force-select · HTTP 200 rpub=TKE70G18CRJB… (ok for WID cfa99aa3)
┌ INFO   ┐ #088 ladder.withdraw-60  confirm · HTTP 204 after selected (poll 2)
┌ OK     ┐ #089 ladder.withdraw-61  confirm GOA:61 2630ms (client, on selected)
┌ OK     ┐ #090 ladder.withdraw-62  settle GOA:61 bank transfer_done (wallet avail=GOA:0, 935ms)
┌ INFO   ┐ #091 ladder.withdraw-63  coins ladder-r8-after-GOA_61 · n=0  in_circulation=0  spent=0  amount_circ=0 GOA (GOA:0)
┌ 76%┐ █████████████████████░░░░░░░  92/120
┌ INFO   ┐ #092 ladder.withdraw-64  coins ladder-r8-after-GOA_61 · denoms_in_circ (none)
┌ INFO   ┐ #093 ladder.withdraw-65  coins ladder-r8-after-GOA_61 Δ · new=+0  total_now=0  |  (no denom change)

╔════════════════════════════════════╗
║ ladder · rung 9  GOA:204 · 204 GOA ║
╚════════════════════════════════════╝
┌ OK     ┐ #094 ladder.withdraw-66  mint GOA:204 (a1d6cb30-40ad-4c7d-872c-dfc3f56ba3d7) 297ms
┌ OK     ┐ #095 ladder.withdraw-67  accept-uri GOA:204 5367ms
┌ INFO   ┐ #096 ladder.withdraw-68  force-select · HTTP 200 rpub=89Y8RB7BY357… (ok for WID a1d6cb30)
┌ WARN   ┐ #097 ladder.withdraw-69  bank: confirm GOA:204 skipped · problem: bank status='selected' after 40 polls (want selected/confirmed). Usually reserve_pub mismatch (5114), not out-of-money — mint/accept already OK. detail: confirm HTTP 500

╔═════════════════════════════════════╗
║ ladder · rung 10  GOA:286 · 286 GOA ║
╚═════════════════════════════════════╝
┌ OK     ┐ #098 ladder.withdraw-70  mint GOA:286 (472dd0d4-473b-4cde-a497-54082353bd15) 330ms
┌ OK     ┐ #099 ladder.withdraw-71  accept-uri GOA:286 5625ms
┌ 83%┐ ███████████████████████░░░░░  100/120
┌ INFO   ┐ #100 ladder.withdraw-72  force-select · HTTP 200 rpub=GY6NRXXZPCAK… (ok for WID 472dd0d4)
┌ WARN   ┐ #101 ladder.withdraw-73  bank: confirm GOA:286 skipped · problem: bank status='selected' after 40 polls (want selected/confirmed). Usually reserve_pub mismatch (5114), not out-of-money — mint/accept already OK. detail: confirm HTTP 500

╔══════════════════════════════════════════════════════════╗
║ ladder · rung 11  GOA:47061 · 47.061 Kilo-GOA (GOA:47061) ║
╚══════════════════════════════════════════════════════════╝
┌ OK     ┐ #102 ladder.withdraw-74  mint GOA:47061 (4240dd18-a122-4dcf-9fec-b9502784e5b1) 261ms
┌ OK     ┐ #103 ladder.withdraw-75  accept-uri GOA:47061 5467ms
┌ INFO   ┐ #104 ladder.withdraw-76  force-select · HTTP 200 rpub=6S1V7A3024NP… (ok for WID 4240dd18)
┌ WARN   ┐ #105 ladder.withdraw-77  bank: confirm GOA:47061 skipped · problem: bank status='selected' after 40 polls (want selected/confirmed). Usually reserve_pub mismatch (5114), not out-of-money — mint/accept already OK. detail: confirm HTTP 500

╔══════════════════════════════════════════════════════════╗
║ ladder · rung 12  GOA:2987987 · 2.987987 Mega-GOA (GOA:2987987) ║
╚══════════════════════════════════════════════════════════╝
┌ WARN   ┐ #106 ladder.withdraw-78  bank: mint GOA:2987987 rejected (ceiling) · problem: bank mint HTTP 500/5110 SQL P0001 on high amount (not only pin:max). Ladder continues; use max-ladder to bound hi. detail: amount above live mint ceiling (5110/P0001): mint HTTP 500 { code :5110, hint : Unexpected sql error with state P0001 }

╔══════════════════════════════════════════════════════════╗
║ ladder · rung 13  GOA:21008392 · 21.008392 Mega-GOA (GOA:21008392) ║
╚══════════════════════════════════════════════════════════╝
┌ WARN   ┐ #107 ladder.withdraw-79  bank: mint GOA:21008392 rejected (ceiling) · problem: bank mint HTTP 500/5110 SQL P0001 on high amount (not only pin:max). Ladder continues; use max-ladder to bound hi. detail: amount above live mint ceiling (5110/P0001): mint HTTP 500 { code :5110, hint : Unexpected sql error with state P0001 }

╔══════════════════════════════════════════════════════════╗
║ ladder · rung 14  GOA:638490396 · 638.490396 Mega-GOA (GOA:638490396) ║
╚══════════════════════════════════════════════════════════╝
┌ 90%┐ █████████████████████████░░░  108/120
┌ WARN   ┐ #108 ladder.withdraw-80  bank: mint GOA:638490396 rejected (ceiling) · problem: bank mint HTTP 500/5110 SQL P0001 on high amount (not only pin:max). Ladder continues; use max-ladder to bound hi. detail: amount above live mint ceiling (5110/P0001): mint HTTP 500 { code :5110, hint : Unexpected sql error with state P0001 }

╔══════════════════════════════════════════════════════════╗
║ ladder · rung 15  GOA:1144771370 · 1.14477137 Giga-GOA (GOA:1144771370) ║
╚══════════════════════════════════════════════════════════╝
┌ WARN   ┐ #109 ladder.withdraw-81  bank: mint GOA:1144771370 rejected (ceiling) · problem: bank mint HTTP 500/5110 SQL P0001 on high amount (not only pin:max). Ladder continues; use max-ladder to bound hi. detail: amount above live mint ceiling (5110/P0001): mint HTTP 500 { code :5110, hint : Unexpected sql error with state P0001 }

╔══════════════════════════════════════════════════════════╗
║ ladder · rung 16  GOA:1932790550 · 1.93279055 Giga-GOA (GOA:1932790550) ║
╚══════════════════════════════════════════════════════════╝
┌ WARN   ┐ #110 ladder.withdraw-82  bank: mint GOA:1932790550 rejected (ceiling) · problem: bank mint HTTP 500/5110 SQL P0001 on high amount (not only pin:max). Ladder continues; use max-ladder to bound hi. detail: amount above live mint ceiling (5110/P0001): mint HTTP 500 { code :5110, hint : Unexpected sql error with state P0001 }


See also for CLI-AUTOMATIONES-NOTES in the taler-monitoring repo for a more detailed analysis / possible hints: https://git.hacktivism.ch/hernani/taler-monitoring/src/branch/main/CLI-AUTOMATION-NOTES.md#3b-we-do-not-manage-to-withdraw-in-time-goa-bank-path-unreliable-not-load
Additional InformationSoftware stack: libeufin-bank 1.6.6 / libeufin-common 1.6.6 on Debian Trixie (Stable)

See also rough bank stats here: https://bank.hacktivism.ch/intro/

Tagsperformance, quality

Activities

vecirex

2026-07-19 15:20

administrator   ~0029195

Tell me if it's better to try w/ new libeufin-bank in rust, if handling changed there.

vecirex

2026-07-19 15:24

administrator   ~0029196

Last edited: 2026-07-19 15:28

 Also it's taking at least over 180s (cut_off; maybe many more minutes) to see any spendable money in the wallet, such that additional payment ladder tests cannot even launch (currently, e.g., not at all for anyone trying to collect money from this eco sys: I assume minting is going strong...); example extract:

┌ INFO   ┐ #179 ladder.pay-50  pay · withdraw board (start order) · timer 13s left · available=GOA:0 · pendingIncoming=GOA:309997

┌100%┐ ████████████████████████████  180/180

┌ INFO   ┐ #180 ladder.pay-51  pay ·   (no OK/OK_BANK rows in TSV yet)

┌ INFO   ┐ #181 ladder.pay-52  pay · withdraw board (start order) · timer 5s left · available=GOA:0 · pendingIncoming=GOA:309997

┌ INFO   ┐ #182 ladder.pay-53  pay ·   (no OK/OK_BANK rows in TSV yet)

┌ INFO   ┐ #183 ladder.pay-54  pay · withdraw board (start order) · timer 0s left · available=GOA:0 · pendingIncoming=GOA:309997

┌ INFO   ┐ #184 ladder.pay-55  pay ·   (no OK/OK_BANK rows in TSV yet)

┌ INFO   ┐ #185 ladder.pay-56  pay · timer 0s left — stop waiting (available still GOA:0)

┌ WARN   ┐ #186 ladder.pay-57  pay: no spendable coins after 180s · problem: available still GOA:0 (pendingIncoming=GOA:309997). Board above shows which withdraws still pendingIncoming. Pay will SKIP_BALANCE / skip max-pay until wirewatch+wallet finish. Same wallet as withdraw.

0 — never order more than this

vecirex

2026-07-19 15:29

administrator   ~0029197

This site is made to generate all 24hrs, let's see: https://taler.hacktivism.ch/monitoring-max-ladder

Issue History

Date Modified Username Field Change
2026-07-19 15:18 vecirex New Issue
2026-07-19 15:20 vecirex Note Added: 0029195
2026-07-19 15:21 vecirex Tag Attached: quality
2026-07-19 15:21 vecirex Tag Attached: performance
2026-07-19 15:22 vecirex Additional Information Updated
2026-07-19 15:24 vecirex Note Added: 0029196
2026-07-19 15:25 vecirex Note Edited: 0029196
2026-07-19 15:25 vecirex Note Edited: 0029196
2026-07-19 15:28 vecirex Note Edited: 0029196
2026-07-19 15:29 vecirex Note Added: 0029197