View Issue Details

IDProjectCategoryView StatusLast Update
0004965GNUnetdatastore servicepublic2018-06-07 00:24
ReporteramatusAssigned Toamatus 
PrioritynormalSeverityminorReproducibilityN/A
Status closedResolutionfixed 
Product VersionSVN HEAD 
Target Version0.11.0pre66Fixed in Version0.11.0pre66 
Summary0004965: Push the logic for updating an existing record if it already exists down into the plugins
DescriptionThe only use of the `update` plugin API call is if a record already exists in the database and we need to update the priority, replication, or expiration.
On every put we do a get then an update or put, this logic can be pushed down into the plugins where it can be accomplished in one transaction on the database (if supported by the database).
TagsNo tags attached.

Activities

amatus

2017-04-16 19:29

developer   ~0012043

I pushed the update logic into the plugins and got a nice performance increase without any fancy SQL. The code is smaller too.

amatus@nuc:~/gnunet/src/datastore$ cat before_test_run_* | grep heap
perf_datastore_api_heap.log:PUT performance: 989 ms for 2712 operations
perf_datastore_api_heap.log:PUT performance: 0 ms/operation
perf_datastore_api_heap.log:PUT performance: 1149 ms for 2712 operations
perf_datastore_api_heap.log:PUT performance: 0 ms/operation
perf_datastore_api_heap.log:PUT performance: 1085 ms for 2708 operations
perf_datastore_api_heap.log:PUT performance: 0 ms/operation
amatus@nuc:~/gnunet/src/datastore$ cat after_test_run_* | grep heap
perf_datastore_api_heap.log:PUT performance: 630 ms for 2704 operations
perf_datastore_api_heap.log:PUT performance: 0 ms/operation
perf_datastore_api_heap.log:PUT performance: 616 ms for 2718 operations
perf_datastore_api_heap.log:PUT performance: 0 ms/operation
perf_datastore_api_heap.log:PUT performance: 630 ms for 2712 operations
perf_datastore_api_heap.log:PUT performance: 0 ms/operation

amatus@nuc:~/gnunet/src/datastore$ cat before_test_run_* | grep sqlite
perf_datastore_api_sqlite.log:PUT performance: 1739 ms for 2706 operations
perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation
perf_datastore_api_sqlite.log:PUT performance: 1802 ms for 2712 operations
perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation
perf_datastore_api_sqlite.log:PUT performance: 1761 ms for 2712 operations
perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation
amatus@nuc:~/gnunet/src/datastore$ cat after_test_run_* | grep sqlite
perf_datastore_api_sqlite.log:PUT performance: 1160 ms for 2714 operations
perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation
perf_datastore_api_sqlite.log:PUT performance: 1164 ms for 2710 operations
perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation
perf_datastore_api_sqlite.log:PUT performance: 1167 ms for 2714 operations
perf_datastore_api_sqlite.log:PUT performance: 0 ms/operation

amatus@nuc:~/gnunet/src/datastore$ cat before_test_run_* | grep mysql
perf_datastore_api_mysql.log:PUT performance: 61 s for 2708 operations
perf_datastore_api_mysql.log:PUT performance: 22 ms/operation
perf_datastore_api_mysql.log:PUT performance: 61 s for 2712 operations
perf_datastore_api_mysql.log:PUT performance: 22 ms/operation
perf_datastore_api_mysql.log:PUT performance: 62 s for 2714 operations
perf_datastore_api_mysql.log:PUT performance: 22 ms/operation
amatus@nuc:~/gnunet/src/datastore$ cat after_test_run_* | grep mysql
perf_datastore_api_mysql.log:PUT performance: 47 s for 2708 operations
perf_datastore_api_mysql.log:PUT performance: 17 ms/operation
perf_datastore_api_mysql.log:PUT performance: 45 s for 2706 operations
perf_datastore_api_mysql.log:PUT performance: 16 ms/operation
perf_datastore_api_mysql.log:PUT performance: 45 s for 2710 operations
perf_datastore_api_mysql.log:PUT performance: 16 ms/operation

amatus@nuc:~/gnunet/src/datastore$ cat before_test_run_* | grep postgres
perf_datastore_api_postgres.log:PUT performance: 42 s for 2714 operations
perf_datastore_api_postgres.log:PUT performance: 15 ms/operation
perf_datastore_api_postgres.log:PUT performance: 42 s for 2710 operations
perf_datastore_api_postgres.log:PUT performance: 15 ms/operation
perf_datastore_api_postgres.log:PUT performance: 42 s for 2714 operations
perf_datastore_api_postgres.log:PUT performance: 15 ms/operation
amatus@nuc:~/gnunet/src/datastore$ cat after_test_run_* | grep postgres
perf_datastore_api_postgres.log:PUT performance: 30 s for 2714 operations
perf_datastore_api_postgres.log:PUT performance: 11 ms/operation
perf_datastore_api_postgres.log:PUT performance: 30 s for 2706 operations
perf_datastore_api_postgres.log:PUT performance: 11 ms/operation
perf_datastore_api_postgres.log:PUT performance: 31 s for 2710 operations
perf_datastore_api_postgres.log:PUT performance: 11 ms/operation

amatus

2017-04-16 19:47

developer   ~0012044

Fixed in 4907330f51ffd48af1f7bac6f43c7c7f78c37818

Issue History

Date Modified Username Field Change
2017-03-19 22:38 amatus New Issue
2017-03-19 22:38 amatus Status new => assigned
2017-03-19 22:38 amatus Assigned To => amatus
2017-04-16 19:29 amatus Note Added: 0012043
2017-04-16 19:47 amatus Status assigned => resolved
2017-04-16 19:47 amatus Resolution open => fixed
2017-04-16 19:47 amatus Fixed in Version => 0.11.0pre66
2017-04-16 19:47 amatus Note Added: 0012044
2017-04-16 19:54 Christian Grothoff Target Version => 0.11.0pre66
2018-06-07 00:24 Christian Grothoff Status resolved => closed