View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0004965 | GNUnet | datastore service | public | 2017-03-19 22:38 | 2018-06-07 00:24 |
Reporter | amatus | Assigned To | amatus | ||
Priority | normal | Severity | minor | Reproducibility | N/A |
Status | closed | Resolution | fixed | ||
Product Version | Git master | ||||
Target Version | 0.11.0pre66 | Fixed in Version | 0.11.0pre66 | ||
Summary | 0004965: Push the logic for updating an existing record if it already exists down into the plugins | ||||
Description | The only use of the `update` plugin API call is if a record already exists in the database and we need to update the priority, replication, or expiration. On every put we do a get then an update or put, this logic can be pushed down into the plugins where it can be accomplished in one transaction on the database (if supported by the database). | ||||
Tags | No tags attached. | ||||
|
I pushed the update logic into the plugins and got a nice performance increase without any fancy SQL. The code is smaller too. amatus@nuc:~/gnunet/src/datastore$ cat before_test_run_* | grep heap perf_datastore_api_heap.log:PUT performance: 989 ms for 2712 operations perf_datastore_api_heap.log:PUT performance: 0 ms/operation perf_datastore_api_heap.log:PUT performance: 1149 ms for 2712 operations perf_datastore_api_heap.log:PUT performance: 0 ms/operation perf_datastore_api_heap.log:PUT performance: 1085 ms for 2708 operations perf_datastore_api_heap.log:PUT performance: 0 ms/operation amatus@nuc:~/gnunet/src/datastore$ cat after_test_run_* | grep heap perf_datastore_api_heap.log:PUT performance: 630 ms for 2704 operations perf_datastore_api_heap.log:PUT performance: 0 ms/operation perf_datastore_api_heap.log:PUT performance: 616 ms for 2718 operations perf_datastore_api_heap.log:PUT performance: 0 ms/operation perf_datastore_api_heap.log:PUT performance: 630 ms for 2712 operations perf_datastore_api_heap.log:PUT performance: 0 ms/operation amatus@nuc:~/gnunet/src/datastore$ cat before_test_run_* | grep sqlite perf_datastore_api_sqlite.log:PUT performance: 1739 ms for 2706 operations perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation perf_datastore_api_sqlite.log:PUT performance: 1802 ms for 2712 operations perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation perf_datastore_api_sqlite.log:PUT performance: 1761 ms for 2712 operations perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation amatus@nuc:~/gnunet/src/datastore$ cat after_test_run_* | grep sqlite perf_datastore_api_sqlite.log:PUT performance: 1160 ms for 2714 operations perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation perf_datastore_api_sqlite.log:PUT performance: 1164 ms for 2710 operations perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation perf_datastore_api_sqlite.log:PUT performance: 1167 ms for 2714 operations perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation amatus@nuc:~/gnunet/src/datastore$ cat before_test_run_* | grep mysql perf_datastore_api_mysql.log:PUT performance: 61 s for 2708 operations perf_datastore_api_mysql.log:PUT performance: 22 ms/operation perf_datastore_api_mysql.log:PUT performance: 61 s for 2712 operations perf_datastore_api_mysql.log:PUT performance: 22 ms/operation perf_datastore_api_mysql.log:PUT performance: 62 s for 2714 operations perf_datastore_api_mysql.log:PUT performance: 22 ms/operation amatus@nuc:~/gnunet/src/datastore$ cat after_test_run_* | grep mysql perf_datastore_api_mysql.log:PUT performance: 47 s for 2708 operations perf_datastore_api_mysql.log:PUT performance: 17 ms/operation perf_datastore_api_mysql.log:PUT performance: 45 s for 2706 operations perf_datastore_api_mysql.log:PUT performance: 16 ms/operation perf_datastore_api_mysql.log:PUT performance: 45 s for 2710 operations perf_datastore_api_mysql.log:PUT performance: 16 ms/operation amatus@nuc:~/gnunet/src/datastore$ cat before_test_run_* | grep postgres perf_datastore_api_postgres.log:PUT performance: 42 s for 2714 operations perf_datastore_api_postgres.log:PUT performance: 15 ms/operation perf_datastore_api_postgres.log:PUT performance: 42 s for 2710 operations perf_datastore_api_postgres.log:PUT performance: 15 ms/operation perf_datastore_api_postgres.log:PUT performance: 42 s for 2714 operations perf_datastore_api_postgres.log:PUT performance: 15 ms/operation amatus@nuc:~/gnunet/src/datastore$ cat after_test_run_* | grep postgres perf_datastore_api_postgres.log:PUT performance: 30 s for 2714 operations perf_datastore_api_postgres.log:PUT performance: 11 ms/operation perf_datastore_api_postgres.log:PUT performance: 30 s for 2706 operations perf_datastore_api_postgres.log:PUT performance: 11 ms/operation perf_datastore_api_postgres.log:PUT performance: 31 s for 2710 operations perf_datastore_api_postgres.log:PUT performance: 11 ms/operation |
|
Fixed in 4907330f51ffd48af1f7bac6f43c7c7f78c37818 |
Date Modified | Username | Field | Change |
---|---|---|---|
2017-03-19 22:38 | amatus | New Issue | |
2017-03-19 22:38 | amatus | Status | new => assigned |
2017-03-19 22:38 | amatus | Assigned To | => amatus |
2017-04-16 19:29 | amatus | Note Added: 0012043 | |
2017-04-16 19:47 | amatus | Status | assigned => resolved |
2017-04-16 19:47 | amatus | Resolution | open => fixed |
2017-04-16 19:47 | amatus | Fixed in Version | => 0.11.0pre66 |
2017-04-16 19:47 | amatus | Note Added: 0012044 | |
2017-04-16 19:54 | Christian Grothoff | Target Version | => 0.11.0pre66 |
2018-06-07 00:24 | Christian Grothoff | Status | resolved => closed |