View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0011876Talerlibeufin (general)public2026-10-20 05:172026-10-20 05:17
Reporterapteryx Assigned To 
PrioritynormalSeverityminorReproducibilitysometimes
Status newResolutionopen 
Platformx86-64-linuxOSGuix SystemOS Versionlatest
Product Version1.5 
Summary0011876: libeufin test suite fails non-deterministically (looks like there is a race)
DescriptionThe test suite of libeufin often fails, but not always. It appears there is a non-deterministic race that can be triggered, causing a cascade of failures.
Steps To ReproduceCreate this script, named `init-taler-db-for-tests.sh`:
```
#!/bin/sh

dbdir=$PWD/test-db
initdb -D "$dbdir"
pg_ctl --options="-k \"$dbdir\"" -D "$dbdir" -l "$dbdir/db.log" start
createdb -h "$dbdir" taler_rust_check
```

Then, checkout libeufin at commit fd1ee57ead51220f86088619550ff62e477d5902 (current latest), and build/test it with:

```
 ./bootstrap
./configure --prefix=$PWD/install
make install -j32
./init-taler-db-for-tests.sh
PATH=$PWD/install/bin:$PATH make check -j32
```

I'm building using rust-1.96 provided by Guix and postgresql at version 14.23.
Additional InformationHere's what the failures look like, after the problem started:
```
2026-10-20T02:50:21.609749+00:00 DEBUG (RZD18I5JNT)api: GET /accounts/exchange/taler-revenue/history?limit=-9223372036854775808
2026-10-20T02:50:21.610169+00:00 WARN (RZD18I5JNT)api: 400 Bad Request GET /accounts/exchange/taler-revenue/history?limit=-9223372036854775808 0ms: GENERIC_PARAMETER_MALFORMED Param 'limit' must be <= 1024 for -9223372036854775808
2026-10-20T02:50:21.610268+00:00 DEBUG (MB44RWTO7U)api: GET /accounts/exchange/taler-revenue/history?limit=invalid
2026-10-20T02:50:21.610672+00:00 WARN (MB44RWTO7U)api: 400 Bad Request GET /accounts/exchange/taler-revenue/history?limit=invalid 0ms: GENERIC_PARAMETER_MALFORMED invalid digit found in string
2026-10-20T02:50:21.610771+00:00 DEBUG (DIAMYZ7JW8)api: GET /accounts/exchange/taler-revenue/history?offset=-1
2026-10-20T02:50:21.661651+00:00 ERROR (EZWHA2JWKN)api: 502 Bad Gateway POST /accounts/admin/token 31060ms: GENERIC_DB_FETCH_FAILED pool timed out while waiting for an open connection
2026-10-20T02:50:21.662163+00:00 WARN (DIAMYZ7JW8)api: 400 Bad Request GET /accounts/exchange/taler-revenue/history?offset=-1 51ms: GENERIC_PARAMETER_MALFORMED Param 'offset' must be positive got -1
2026-10-20T02:50:21.668092+00:00 DEBUG (HCC99GUIXF)api: GET /accounts/exchange/taler-revenue/history?offset=invalid
test api::wire::test::incoming_history ... FAILED
2026-10-20T02:50:21.680257+00:00 WARN (HCC99GUIXF)api: 400 Bad Request GET /accounts/exchange/taler-revenue/history?offset=invalid 12ms: GENERIC_PARAMETER_MALFORMED invalid digit found in string
test api::wire::test::account_check ... FAILED
test api::revenue::test::revenue ... ok
2026-10-20T02:50:21.695082+00:00 TRACE (CJI286U6CJ)api: req {"duration":{"d_us":"forever"},"scope":"readwrite"}
2026-10-20T02:50:21.703050+00:00 INFO (CJI286U6CJ)api: 200 OK POST /accounts/merchant/token 30525ms
test api::tan::test::send ... FAILED
2026-10-20T02:50:21.967982+00:00 ERROR (5W3UTXSBYJ)api: 502 Bad Gateway POST /accounts/exchange/token 30084ms: GENERIC_DB_FETCH_FAILED pool timed out while waiting for an open connection
test api::account::test::list ... FAILED
test bench::bench_db ... FAILED
test db::test::tan_challenge ... FAILED

failures:

---- api::prepared::test::registration stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::prepared::test::registration' (32741) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- api::conversion::test::classes stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::conversion::test::classes' (32738) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/merchant/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::cashout::test::cashout stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::cashout::test::cashout' (32737) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::tx::test::history stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::tx::test::history' (32761) panicked at libeufin-bank/src/api.rs:720:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- db::test::amount_computation stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'db::test::amount_computation' (32773) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::account::test::reconfig stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::account::test::reconfig' (32736) panicked at libeufin-bank/src/api.rs:720:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::withdrawal::test::withdrawal stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::withdrawal::test::withdrawal' (32771) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::account::test::password stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::account::test::password' (32735) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::token::test::create_and_delete stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::token::test::create_and_delete' (32757) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::account::test::create stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::account::test::create' (32732) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::conversion::test::rates stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::conversion::test::rates' (32739) panicked at libeufin-bank/src/api.rs:747:18:
called `Result::unwrap()` on an `Err` value: pool timed out while waiting for an open connection

---- api::test::corebank stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::test::corebank' (32755) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::wire::test::outgoing_history stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::wire::test::outgoing_history' (32769) panicked at libeufin-bank/src/api.rs:747:18:
called `Result::unwrap()` on an `Err` value: pool timed out while waiting for an open connection

---- api::tan::test::rate_limited stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::tan::test::rate_limited' (32744) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::wire::test::transfer stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::wire::test::transfer' (32770) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::wire::test::admin_add_incoming stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::wire::test::admin_add_incoming' (32767) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::observability::test::observability stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::observability::test::observability' (32740) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::tx::test::tx_with_fee stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::tx::test::tx_with_fee' (32765) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::tan::test::confirm stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::tan::test::confirm' (32743) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::tx::test::tx stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::tx::test::tx' (32762) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::tan::test::tan_err stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::tan::test::tan_err' (32754) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- db::test::statistics stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'db::test::statistics' (32774) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::token::test::get stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::token::test::get' (32760) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::account::test::delete stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::account::test::delete' (32733) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/merchant/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::wire::test::incoming_history stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::wire::test::incoming_history' (32768) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::wire::test::account_check stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::wire::test::account_check' (32766) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/admin/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- api::tan::test::send stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::tan::test::send' (32750) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- api::account::test::list stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'api::account::test::list' (32734) panicked at libeufin-bank/src/api.rs:426:26:
POST https://example/accounts/exchange/token expected 200 OK got 502 Bad Gateway: GENERIC_DB_FETCH_FAILED

---- bench::bench_db stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'bench::bench_db' (32772) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut

---- db::test::tan_challenge stdout ----
could not fetch system time zone, default to UTC: failed to get system time zone: operation failed because Jiff crate feature `tz-system` is not enabled

thread 'db::test::tan_challenge' (32775) panicked at libeufin-bank/src/api.rs:698:14:
called `Result::unwrap()` on an `Err` value: PoolTimedOut


failures:
    api::account::test::create
    api::account::test::delete
    api::account::test::list
    api::account::test::password
    api::account::test::reconfig
    api::cashout::test::cashout
    api::conversion::test::classes
    api::conversion::test::rates
    api::observability::test::observability
    api::prepared::test::registration
    api::tan::test::confirm
    api::tan::test::rate_limited
    api::tan::test::send
    api::tan::test::tan_err
    api::test::corebank
    api::token::test::create_and_delete
    api::token::test::get
    api::tx::test::history
    api::tx::test::tx
    api::tx::test::tx_with_fee
    api::wire::test::account_check
    api::wire::test::admin_add_incoming
    api::wire::test::incoming_history
    api::wire::test::outgoing_history
    api::wire::test::transfer
    api::withdrawal::test::withdrawal
    bench::bench_db
    db::test::amount_computation
    db::test::statistics
    db::test::tan_challenge

test result: FAILED. 2 passed; 30 failed; 0 ignored; 0 measured; 0 filtered out; finished in 35.59s

error: test failed, to rerun pass `-p libeufin-bank --lib`
```

The complete test log is attached.
TagsNo tags attached.
Attached Files
libeufin-test-suite-failures.log.txt (364,514 bytes)

Activities

Issue History

Date Modified Username Field Change
2026-10-20 05:17 apteryx New Issue
2026-10-20 05:17 apteryx File Added: libeufin-test-suite-failures.log.txt