View Issue Details

IDProjectCategoryView StatusLast Update
0005136Talerexchange API (C)public2019-12-20 19:12
ReporterChristian Grothoff Assigned ToMarcello Stanisci  
Status closedResolutionfixed 
Platformi7OSDebian GNU/LinuxOS Versionsqueeze
Product Versiongit (master) 
Target Version0.6Fixed in Version0.6 
Summary0005136: exchange API should implement and export functions to persist /keys data
DescriptionEventually wallets will want to avoid re-downloading /keys across process restarts. In this case, we should (1) have an API to store the /keys data on disk, and (2) have a way to garbage-collect expired keys. While the wallet needs this "today" (0004188), the C API is only used for testcases and merchants today, so this is not urgent at all.
TagsNo tags attached.


Christian Grothoff

2018-10-13 08:15

manager   ~0013266

eb1b6fbc..c09c9009 implements the deserialization logic.
Serialization and testing are still TODO.

Christian Grothoff

2018-10-13 19:48

manager   ~0013267

API implemented in c09c9009..16b0c654. Needs testing.

Marcello, can you please devise a test? It should be possible to extend the existing keys cherry picking testcase, i.e. by adding a command that calls the serialize function, then disconnects from the exchange and then immediately reconnects (with the serialized state). Afterwards, doing 'something' that involves the data from /keys ought to suffice to test most of the logic (plus maybe looking at the result with wireshark to make sure we didn't do a full /keys transfer again).

Marcello Stanisci

2018-12-21 10:46

viewer   ~0013432

Last edited: 2018-12-21 10:47

By looking at the current code, the full /keys download is not delayed when the client reconnects with a serialized state. So in order to test it properly, the test case should be "fast enough";
in other words, it should run before the /keys are downloaded.

Is that right?

Shall we delay the download of /keys when the client reconnects with a serialized state?

Also, without any delay in place, it is quite likely that the next task that runs is the /keys download itself ('request_keys()'), making the client *double* the work needed to recover the keys state: deserialization + download.


Christian Grothoff

2018-12-21 10:56

manager   ~0013433

Eh, I don't understand why you'd need to be "fast enough" or delay anything.

The idea was that you connect, serialize the /keys with the existing API, disconnect, reconnect with the serialized /keys data, and then only a (tiny) download happens because most of the state we already have, but still _all_ /keys data is correctly available.

There is two cases to test: (1) /keys did not change at all, and (2) there are a few new keys and the state thus needs to be merged. So in (2) you additionally need to trigger new key generation and _there_ you might need a bit of a delay.

Marcello Stanisci

2018-12-21 11:14

viewer   ~0013434

Okay, I didn't investigate deeply enough to figure that the URL parameter 'last_issue_date=XXX' makes the /keys download *tiny*. This answers my question!

Marcello Stanisci

2018-12-24 17:11

viewer   ~0013438

Last edited: 2018-12-24 17:11

Current problem: the deserialized JSON object has more auditor-signed keys than denominations. This suggests there is some issue in the way new downloaded keys get merged to the old one.

As a result, the 'break_op' command fails - for some keys - at exchange_api_handle.c:459 (as of bc771beb23629f9251).

Christian Grothoff

2019-01-02 11:55

manager   ~0013442

Hmm. My first guess would be that this could happen if we removed DKs due to expiration in one place of the logic, but failed to remove the matching signatures elsewhere. How do I reproduce this one best? (which test?)

Christian Grothoff

2019-01-02 12:10

manager   ~0013443

I've checked, and the code did indeed kick out expired denominatino keys but kept their auditor signatures, which would explain the assertion failure.

So this should be fixed in a6331c8e..5c71c3e7, but I did not manage to reproduce the issue, so Marcello please confirm that this is indeed fixed.

Marcello Stanisci

2019-01-07 15:58

viewer   ~0013445

I confirm. And BTW, the test being run was 'test_exchange_api_keys_cherry_picking_new', just in case something will happen again.

Note, the issue is not yet fixed, I need to *use* the serialized keys now.

Marcello Stanisci

2019-01-22 15:47

viewer   ~0013458

4877cb07 closes this bug.

Issue History

Date Modified Username Field Change
2017-09-12 15:30 Christian Grothoff New Issue
2017-09-12 15:30 Christian Grothoff Status new => assigned
2017-09-12 15:30 Christian Grothoff Assigned To => Christian Grothoff
2017-09-12 15:33 Christian Grothoff Assigned To Christian Grothoff =>
2017-09-12 15:33 Christian Grothoff Status assigned => confirmed
2017-09-12 15:33 Christian Grothoff Summary exchange API should have a way to persist /keys data => exchange API should implement and export functions to persist /keys data
2017-11-12 16:12 Christian Grothoff Assigned To => Christian Grothoff
2017-11-12 16:12 Christian Grothoff Status confirmed => assigned
2018-10-13 08:15 Christian Grothoff Note Added: 0013266
2018-10-13 19:48 Christian Grothoff Note Added: 0013267
2018-10-13 19:48 Christian Grothoff Assigned To Christian Grothoff => Marcello Stanisci
2018-11-18 00:36 Christian Grothoff Target Version => 0.8
2018-12-21 10:46 Marcello Stanisci Note Added: 0013432
2018-12-21 10:47 Marcello Stanisci Note Edited: 0013432
2018-12-21 10:56 Christian Grothoff Note Added: 0013433
2018-12-21 11:14 Marcello Stanisci Note Added: 0013434
2018-12-24 17:11 Marcello Stanisci Note Added: 0013438
2018-12-24 17:11 Marcello Stanisci Note Edited: 0013438
2019-01-02 11:55 Christian Grothoff Note Added: 0013442
2019-01-02 11:55 Christian Grothoff Assigned To Marcello Stanisci => Christian Grothoff
2019-01-02 12:10 Christian Grothoff Note Added: 0013443
2019-01-02 12:11 Christian Grothoff Assigned To Christian Grothoff => Marcello Stanisci
2019-01-07 15:58 Marcello Stanisci Note Added: 0013445
2019-01-22 15:47 Marcello Stanisci Note Added: 0013458
2019-01-22 15:48 Marcello Stanisci Status assigned => resolved
2019-01-22 15:48 Marcello Stanisci Resolution open => fixed
2019-01-31 19:33 Christian Grothoff Fixed in Version => 0.6
2019-01-31 19:33 Christian Grothoff Target Version 0.8 => 0.6
2019-12-20 19:12 Christian Grothoff Status resolved => closed