View Issue Details

IDProjectCategoryView StatusLast Update
0005010Talerexchangepublic2017-10-18 15:42
ReporterChristian GrothoffAssigned ToChristian Grothoff 
PrioritynormalSeveritymajorReproducibilityrandom
Status closedResolutionfixed 
Platformi7OSDebian GNU/LinuxOS Versionsqueeze
Product VersionSVN HEAD 
Target Version0.4Fixed in Version0.4 
Summary0005010: DB serializability violations not always handled gracefully
DescriptionSometimes the DB plugin returns a "hard" error code for a serializabiltiy violation, causing the exchange to not retry the request, but go all 500 on the user.

We need to go over the code and properly identify all places where serializability violations can occur and double-check their handling.
Steps To ReproduceRun multiple instances of

taler-merchant-generate-payments -n 1000 -e -m

against one merchant/exchange backend produces errors like:

May 14 15:31:46-114418 taler-exchange-httpd-12796 ERROR Query failed at plugin_exchangedb_postgres.c:2065: could not serialize access due to read/write dependencies among transactions/Reason code: Canceled on identification as a pivot, during write./ERROR: could not serialize access due to read/write dependencies among transactions
DETAIL: Reason code: Canceled on identification as a pivot, during write.
HINT: The transaction might succeed if retried.
/PGRES_FATAL_ERROR/ERROR: could not serialize access due to read/write dependencies among transactions
DETAIL: Reason code: Canceled on identification as a pivot, during write.
HINT: The transaction might succeed if retried.

May 14 15:31:46-114486 taler-exchange-httpd-12796 ERROR Assertion failed at plugin_exchangedb_postgres.c:2483.
May 14 15:31:46-114502 taler-exchange-httpd-12796 ERROR Assertion failed at taler-exchange-httpd_db.c:872.

and

May 14 14:10:53-171686 taler-merchant-generate-payments-4409 ERROR Unexpected response code 500 to command withdraw-coin-3
{"error": "internal error", "code": 1108, "hint": "Failure in database interaction"}May 14 14:10:53-171790 taler-merchant-generate-payments-4409 ERROR Assertion failed at taler-merchant-generate-payments.c:680.


Configuration file in use for this against ~0.3 is attached.
TagsNo tags attached.

Activities

Christian Grothoff

2017-05-14 15:40

manager  

taler.conf (4,053 bytes)

Christian Grothoff

2017-05-25 02:07

manager   ~0012159

6bb2631..6029ed7 contains some initial work in this direction, adding a "state" to the sessions to track serializability violations, logic to test the PQ return code against this situation, and modifies some (!) of the functions to return #GNUNET_NO if such a violation is detected.

TODO:
* broaden the logic to all functions in the API
* actually handle the return code in all clients using the API
* make the same changes also in the auditor and merchant backend DB logic

Christian Grothoff

2017-06-25 13:05

manager   ~0012268

Exchange and auditor have been updated, merchant backend is still TBD.

Christian Grothoff

2017-06-25 13:11

manager   ~0012269

For the merchant, the payment generators (sometimes, after a while) fail IF one runs 2 merchant instances and 4 payment generators. While the merchant backend fails to log a proper error message, I suspect this is due to this issue not being fixed yet in the merchant backend.

Christian Grothoff

2017-06-29 19:25

manager   ~0012282

Fixed merchantdb handling of soft errors (by visual inspection); found that generate-payments test failed in part because of timeout given large values of '-n'. Adjusted timeout based on value given to '-n'.

Issue History

Date Modified Username Field Change
2017-05-14 15:40 Christian Grothoff New Issue
2017-05-14 15:40 Christian Grothoff Status new => assigned
2017-05-14 15:40 Christian Grothoff Assigned To => Christian Grothoff
2017-05-14 15:40 Christian Grothoff File Added: taler.conf
2017-05-25 02:07 Christian Grothoff Note Added: 0012159
2017-06-25 13:05 Christian Grothoff Note Added: 0012268
2017-06-25 13:11 Christian Grothoff Note Added: 0012269
2017-06-29 19:25 Christian Grothoff Note Added: 0012282
2017-06-29 20:01 Christian Grothoff Status assigned => resolved
2017-06-29 20:01 Christian Grothoff Resolution open => fixed
2017-06-29 20:01 Christian Grothoff Fixed in Version => 0.4
2017-06-29 20:01 Christian Grothoff Target Version 0.5 => 0.4
2017-10-18 15:42 Christian Grothoff Status resolved => closed