View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6475 [GNUnet] util library minor have not tried 2020-08-11 17:36 2020-08-12 09:43
Reporter: Florian Dold Platform:  
Assigned To: schanzen OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.13.2  
    Target Version: 0.13.2  
Summary: GNUNET_is_zero is a dangerously bad name
Description: A function that has "is" in the name would usually be expected to return a boolean. However,

struct GNUNET_HashCode hc = {0};

if (GNUNET_is_zero (&hc))
  printf("good");
else
  printf("gotcha!");

actually prints "gotacha!".

I get where this is coming from (0 == memcmp (...)), but it's still bad.

Can we at least rename it to GNUNET_cmp_zero?

This has bitten me twice in unrelated places now.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016588)
schanzen   
2020-08-11 20:53   
IMO it should return "enum GNUNET_GenericReturnValue" and not int.
I also would have assumed that it would return != 0 on success. But it does not.
To me, either swapping the return values or using the enum is a viable solution.
(0016590)
schanzen   
2020-08-12 09:43   
I changed the API in be3bbfb47..99f820453


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6468 [Taler] merchant backend API (HTTP specification) feature N/A 2020-08-09 14:20 2020-08-11 21:36
Reporter: Christian Grothoff Platform: i7  
Assigned To: Florian Dold OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: Use 410 Gone if token is provided for claimed order in public GET /orders/{order_id} page
Description: Right now, we give a 403 Forbidden. However, we don't actually know if the token is invalid, as we may have forgotten it (once the order was paid). This may result in a very confusing error for users that may have bookmarked the page with the token in the URL. Hence, we should change the logic to detect that a token was provided for a claimed order and return 410 Gone instead of 403. Additionally, we _eventually_ should provide a nice HTML reply (if requested content-type is HTML), and there detail what went wrong (order already claimed), instead of always returning JSON.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016589)
Florian Dold   
2020-08-11 21:36   
The description of this is outdated, as the suggested approach doesn't work with NoJS, and also breaks when reloading the page at inopportune times.

We should first agree on a clearer specification for what /orders/{order_id} does, depending on the state of the order and and authentication given.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6477 [Taler] merchant backend API (C) minor have not tried 2020-08-11 21:12 2020-08-11 21:12
Reporter: Florian Dold Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: already_paid_fulfillment_url is required by the public version of /orders/{order_id}
Description: Only the storefront can really use the already_paid_order_id.

For the /orders/{order_id} HTML page (which in turn is making the /orders/{order_id} JSON request), the already_paid_order_id is not useful, as it can't really do anything with it. Instead, the backend should return the fulfillment URL corresponding to the already paid order ID.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6476 [Taler] merchant backend API (HTTP specification) minor have not tried 2020-08-11 21:02 2020-08-11 21:02
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: provide concrete examples for the paywall flow
Description: We have a spec now, but we need to validate it with some concrete examples.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5530 [GNUnet] TCP transport feature N/A 2019-01-28 19:24 2020-08-11 14:39
Reporter: Christian Grothoff Platform: i7  
Assigned To: t3sserakt OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.13.2  
Summary: add replay protection to TCP communicator
Description: As TCP is always bi-redirectional, we can easily add some weak form of replay protection by simply adding a nonce to the KX and requiring that the nonce is sent back. To avoid increasing latency on the initial handshake (and knowing that the first bytes sent will be CORE/CADET KX in all likelihood anyway) we would then simply require that after N bytes the nonce is played back to us.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6465 [GNUnet] build process trivial always 2020-08-07 15:57 2020-08-11 11:53
Reporter: ahasenack Platform: Linux  
Assigned To: schanzen OS: Ubuntu  
Priority: normal OS Version: 20.10  
Status: resolved Product Version: Git master  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.13.2  
    Target Version: 0.13.2  
Summary: Fails to build with MySQL8: my_bool is gone
Description: Since mysql 8, the my_bool type was removed. See "Compilation Notes" in https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-1.html:
"""
Incompatible Change: The my_bool type is no longer used in MySQL source code. Any third-party code that used this type to represent C boolean variables should use the bool or int C type instead.
"""

In Ubuntu we have been carrying this workaround patch:
diff --git a/src/mysql/mysql.c b/src/mysql/mysql.c
index 22804d4..4933c44 100644
--- a/src/mysql/mysql.c
+++ b/src/mysql/mysql.c
@@ -24,6 +24,7 @@
  */
 #include "platform.h"
 #include <mysql/mysql.h>
+typedef bool my_bool;
 #include "gnunet_mysql_lib.h"
 
 /**

But that is just a workaround.
Tags: patch
Steps To Reproduce: Build gnunet with mysql8
Additional Information:
Attached Files:
Notes
(0016571)
ahasenack   
2020-08-07 16:36   
Update on the patch:
--- a/src/include/gnunet_mysql_lib.h
+++ b/src/include/gnunet_mysql_lib.h
@@ -32,6 +32,7 @@

 #include "gnunet_util_lib.h"
 #include <mysql/mysql.h>
+typedef bool my_bool;

 #ifdef __cplusplus
 extern "C"
(0016587)
schanzen   
2020-08-11 11:43   
Fixed in 05004fd89..286759692


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6474 [Taler] mechant backend major always 2020-08-11 09:03 2020-08-11 09:03
Reporter: jonathanbuchanan Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: high OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: increase refund does not work after first time
Description: If an order is refunded, and then the refund is increased again after the first refund, the request to increase the refund succeeds, but the income is not shown when querying the merchant for the order.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6450 [Taler] merchant backend API (C) minor have not tried 2020-07-30 11:27 2020-08-10 22:18
Reporter: Florian Dold Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: merchant lib should parse taler_payto_uri, should be used in test cases
Description: It looks like right now the taler_payto_uri is only used by the wallet, not by the merchant's own test cases. This makes it hard to tell if changes break anything.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016560)
jonathanbuchanan   
2020-08-04 22:04   
How should the merchant parse `taler_pay_uri`? Should I add a function that extracts all the parameters from the URI?
(0016563)
Florian Dold   
2020-08-06 15:37   
I think this has two parts:

(1) Implementing a function that parses a taler://pay URI into its components (same will also be needed for taler://refund)
(2) Make the test cases use the taler://pay URI to execute the payment. TALER_TESTING_cmd_merchant_pay_order currently uses the order_id directly. Instead it should take the order_id (and the claim token and session id) from the taler://pay URI, just like the wallet does!

(Right now the test cases would pass even if the merchant screws up the taler://pay URI generation.)
(0016569)
jonathanbuchanan   
2020-08-07 03:35   
I added a spec for these methods in b791a14..5560527.
(0016572)
jonathanbuchanan   
2020-08-08 09:58   
I'm working on implementing it and I noticed that there can be certain situations where there is no way to distinguish between order_id and session_id according to the spec in
https://docs.taler.net/core/taler-uri.html

Suppose there is no claim token or ssid. If there is a segment seperated by '/' from the last segment, then the last segment could be a session_id or an order_id with the segments preceding it being part of merchant_prefix_path.

On a related note, what happens to 'c' and 'ssid' if there isn't a 'session_id'? It would make sense to omit the slash and append them to the order id.
(0016573)
Florian Dold   
2020-08-08 11:39   
Let's look at the definition:

  taler://pay/{merchant_host}{/merchant_prefix_path*}/{order_id}/{session_id}{?c}{#ssid}

Effectively, the session_id is always present. But it can be the empty string, and in that case the path component of the URI ends with a slash! To get the session ID and order ID, you need to count from the back, as the merchant_prefix_path can consist of an arbitrary number of path segments.

The syntax follows the generic URI syntax: https://tools.ietf.org/html/rfc3986 (see Section 3. Syntax Components). You can get the claim token (c) from the query string, which is simply between the first "?" and the fragment delimiter "#". Same for the ssid, which is just the whole fragment (if present)

Here's how the wallet does this parsing: https://git.taler.net/wallet-core.git/tree/packages/taler-wallet-core/src/util/taleruri.ts#n138
(0016574)
jonathanbuchanan   
2020-08-08 21:51   
Ok, thanks for explaining!
(0016578)
jonathanbuchanan   
2020-08-10 05:36   
Parser for pay URIs implemented in ef6fed1..119de8d (merchant.git).
(0016585)
jonathanbuchanan   
2020-08-10 20:52   
Should the claim token always be returned with the pay_uri, or only when it the order is unclaimed? It currently does the latter.

On similar note, would it make sense to define a 'taler://claim' action and return that URI when the order is unclaimed?
(0016586)
jonathanbuchanan   
2020-08-10 22:18   
Parser for refund URIs implemented in 119de8d..280bbb5 (merchant.git).


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6466 [Taler] merchant backend API (HTTP specification) feature N/A 2020-08-09 14:15 2020-08-10 19:48
Reporter: Christian Grothoff Platform: i7  
Assigned To: jonathanbuchanan OS: Debian GNU/Linux  
Priority: high OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: enable long polling for refund pending in GET /orders/{order_id} page
Description: The page currently returns us a 'refund_pending' flag. We should have a wait to long-poll for this flag to change to 'false'. Basically: extra query parameter "await_refund_obtained" that, if 'true' means we wait until timeout_ms _or_ the wallet obtained the refund.
Tags:
Steps To Reproduce: This is for the design document 007 specification, where the HTML page showing the refund URI should long-poll and stop showing the QR code once the wallet obtained the refund.
Additional Information:
System Description
Attached Files:
Notes
(0016584)
jonathanbuchanan   
2020-08-10 19:48   
Spec added in 8a3b86d..3f9f0bb (docs.git).


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6469 [Taler] merchant backend API (HTTP specification) feature N/A 2020-08-09 14:22 2020-08-10 19:44
Reporter: Christian Grothoff Platform: i7  
Assigned To: jonathanbuchanan OS: Debian GNU/Linux  
Priority: high OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: ignore timeout_ms when generating HTML
Description: Right now, the long-polling logic remains in effect even if the backend generates an HTML response. This is bad, as pointing a browser to a URL with the timeout_ms would simply result in the browser showing a blank page waiting for the HTML. While we would never usually do this, it is still bad behavior. We should update the spec and implementation to ignore timeout_ms (treat any value as zero) if the Accept header specifies that we should generate HTML.

(This applies to the GET /orders/{order_id} handler.)
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016576)
jonathanbuchanan   
2020-08-10 03:29   
Spec changed in b9f1061..e225ee2 (docs.git).
(0016577)
jonathanbuchanan   
2020-08-10 03:37   
Implemented in ac7d956..ef6fed1 (merchant.git).


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6457 [Taler] mechant backend feature N/A 2020-08-02 23:27 2020-08-10 16:30
Reporter: Christian Grothoff Platform: i7  
Assigned To: Florian Dold OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8.1  
Summary: HTML version of /orders/$ID should long-poll on refund QR code
Description: The HTML version of /orders/$ID should have three versions:
* unpaid => show pay QR code, long poll for 'paid'
* paid => redirect to fulfillment page
* refund available => show refund QR code, long poll for refund being grabbed by the wallet
* refund performed => show that the order was refunded
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016555)
Christian Grothoff   
2020-08-03 14:20   
This is done, except for the JS-based URI to long-poll URL conversion. See FIXME-6457 in bug. Assigning to Florian.
(0016565)
Florian Dold   
2020-08-06 19:46   
Fixed in 5698d609ef.
(0016566)
Florian Dold   
2020-08-06 19:58   
Refunds etc. aren't done yet
(0016567)
Christian Grothoff   
2020-08-06 19:59   
You missed some:
contrib/offer_tip.en.must: let checkUrl = FIXME-#6457_dold_tip_uri_to_URL("{{taler_tip_uri}}");
contrib/offer_refund.en.must: let checkUrl = FIXME-#6457_dold_refund_uri_to_URL("{{taler_refund_uri}}");
(0016568)
Florian Dold   
2020-08-06 21:09   
Should be fully addressed in b791a14c
(0016583)
Florian Dold   
2020-08-10 16:30   
Doesn't actually do long polling yet ...


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
4096 [Taler] documentation text have not tried 2015-12-17 11:53 2020-08-10 15:16
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version: 0.0  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: Wallet database schema needs to be documented.
Description: It should be documented in some human readable form and preferable also a schema language (JSON Schema?).
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016582)
Florian Dold   
2020-08-10 15:16   
What's more important than the normal database schema (which is already documented by the TypeScript declarations) is the "backup&sync schema", which has to contain versioning information, as well as information about other wallets in the sync group.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6473 [Anastasis] General feature N/A 2020-08-10 14:42 2020-08-10 14:42
Reporter: dennis Platform:  
Assigned To: OS:  
Priority: immediate OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: BRIDGE proposal
Description: Work on BRIDGE proposal.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6472 [Anastasis] command line tools crash always 2020-08-10 14:40 2020-08-10 14:40
Reporter: dennis Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version: Git master  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: Git master  
Summary: Fix client CLI to work with multiple servers
Description: When more than 2 servers are added by "server add"-command, the program crashes. Maybe a loop does not work correctly. Fix "server add"-command.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6471 [Anastasis] backend minor N/A 2020-08-10 14:34 2020-08-10 14:34
Reporter: dennis Platform:  
Assigned To: OS:  
Priority: urgent OS Version:  
Status: new Product Version: Git master  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: Git master  
Summary: fix dependencies (Taler exchange, merchant, gnunet etc.)
Description: Anastasis does not build with current git repositories of dependencies (Taler exchange, Taler merchant, gnunet etc.). Anastasis has to be updated to work with current dependencies.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6470 [Taler] merchant backend API (C) minor have not tried 2020-08-10 13:21 2020-08-10 14:32
Reporter: Florian Dold Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Merchant logs "Finished handling request ... with status 0" despite HTTP 200 response
Description: Can be reproduced by running the API test case (test_merchant_api).
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6452 [Taler] exchange major random 2020-07-31 19:36 2020-08-10 09:17
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: feedback Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: exchange livelocks with only small number of concurrent requests
Description: See the attached logs, specifically async scope IDs

DVGVWZGJMQSMTKBZ2PM0AS0ZHR

and

QVBSQHWV58JKX98G9DS6966KCG.

During 150 milliseconds (!), pretty much the only thing that happens is these two transactions trampling on each other's feet!
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: logs-exchange-2020-07-31.tar.gz (370,347 bytes) 2020-07-31 19:36
https://bugs.gnunet.org/file_download.php?file_id=920&type=bug
Notes
(0016543)
Christian Grothoff   
2020-08-02 23:31   
I still think this is because the exchange's database is so tiny, that page-level locks are too coarse. Disabling parallelism in Postgres seems to fix it pretty well for now, and might be a good answer as parallelism right now won't help anyway -- and once it would, the database will be large enough for the page-level locks to not conflict.
(0016562)
Florian Dold   
2020-08-06 12:33   
Can be reproduced easily now:

cd wallet-core.git
./bootstrap && ./configure --prefix=$HOME/local && make install
cd packages/taler-integrationtests/
./testrunner 'test-payment-multiple'
(0016570)
Christian Grothoff   
2020-08-07 15:56   
(Last edited: 2020-08-10 07:04)
Findings:
- Bug is not 100% reproducible on every run on my system. But happens 'often'.
- Postgres does NOT show any database accesses without an index in the exchange => not the cause.
- Postgres documentation for serializable urges clients to limit concurrency. Reducing thread pool size to 2-4 (from 32) seems to make the problem largely disappear
- Deploying pgpool2 to 'merge' connections from the exchange to Postgres causes all kinds of other havoc (=> abandoned).
- Playing with various postgres configuration settings seems to have no impact

What to do:
- limit #threads for now (workaround)
- investigate if the issue truly persists for 'large' databases where page-level locks should be less of an issue
- medium term: consider reducing #DB connections to less than one per thread (i.e. introduce DB connection pool
  into the exchange logic, protect with semaphore, grab, run, release to pool, etc.)
- long term: investigate what happens if we shard the database

(0016579)
Christian Grothoff   
2020-08-10 07:03   
(Last edited: 2020-08-10 07:38)
So here are some interesting first results.

I am running

$ TALER_TEST_REPEAT=10000 ./scenario rerun-payment-multiple

and analyzing the logs. First:

$ for n in 23 00 01 02 03 04 05 06; do cat /tmp/taler-integrationtest-Hy7qum/exchange-httpd-testexchange-1-stderr.log | grep " $n:..:..-" | grep "Trying to withdraw" | wc -l; done
50678
50988
51012
50291
50465
51096
50782
51523

This query shows: we have a relatively stable number of # withdraw operations/hour. The key point we need from this is that the number is NOT somehow dropping over time.

Now, let's look at the number of serialization failures per hour in the logs, excluding (!) the query "reserve_update":

$ for n in 23 00 01 02 03 04 05 06; do cat /tmp/taler-integrationtest-Hy7qum/exchange-httpd-testexchange-1-stderr.log | grep serializ | grep " $n:..:..-" | grep Query | grep -v reserve_update | wc -l; done
458
299
242
190
174
166
134
119

So here there is a very clear drop. As hoped, with a growing database, serialization failures go down! Great.

Now what about that 'reserve_update' query?

$ for n in 23 00 01 02 03 04 05; do cat /tmp/taler-integrationtest-Hy7qum/exchange-httpd-testexchange-1-stderr.log | grep serializ | grep " $n:..:..-" | grep Query | grep reserve_update | wc -l; done
10656
10936
10928
10608
10863
11110
10952

so here I don't see a drop, it might even be going up. (Why could it go up? Well, maybe the # transactions isn't perfectly stable (see first query), and we actually do get a slight increase here because we do more work per hour, say because the other transactions have fewer serialization failures.)

Big question: why is this query not behaving as hoped? I see three possibilities:
1) Are we re-using the reserve_pub? So the DB is not growing the size of this table?
2) Do we have a missing index problem, so the DB does not benefit from conflict reduction from the growth?
3) Something else we don't understand yet.

Anyway, the other queries make me very hopeful that this entire issue can be solved by:
1) running with low concurrency initially
2) increasing concurrency as the system scales and the DB has naturally fewer serialization conflicts.

(0016580)
Christian Grothoff   
2020-08-10 07:18   
More findings:
(1) we're not re-using the reserve_pub, but even now there are only 0001588:0004080 reserves in the database (but 330884 known coins). So the growth there is MUCH slower, so we may still simply be way too slow.
(2) We definitively have an index, the only item the query selects upon is the PRIMARY KEY of the table.
(3) The reserve_update statement is a key action we do during every withdraw inside of "insert_withdraw_info". So this is likely _the_ transactional bottleneck and maybe a good candidate for replacing the current "SELECT + UPDATE" with a maybe more performant statement.
(0016581)
Christian Grothoff   
2020-08-10 08:27   
(Last edited: 2020-08-10 09:17)
First (preliminary) data shows that merging a SELECT and an UPDATE statement for the reserve balance in 1a37d9d8..6503a9fe (creating a new 'reserve_reduce' SQL statement instead of 'reserve_update') reduces conflicts by about 3%. Again, this is preliminary data.

That said, it seems unavoidable to get conflicts (and larger DBs won't help) *IF* the wallet makes withdraw operations from the same reserve (!) in parallel (!). My guess right now is that this is what is actually going on: the wallet creates coins and runs withdrawals in parallel, and since we only have one wallet and that wallet probably only uses one reserve, we get massive serialization issues _because_ during the withdraw phase we DO really hit the same reserve from multiple threads with massively conflicting updates.

Assuming my analysis above is correct, the solution is clear: change the wallet to ensure it never withdraws in parallel from the same reserve (or equivalently, from operating in parallel on the same coin, say deposit + refresh), at least as far as the network is concerned. Sure, that'll (minimally) reduce parallelism (crypto can be parallel!), but it'll massively help the exchange.



View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6467 [Taler] merchant backend API (HTTP specification) feature N/A 2020-08-09 14:17 2020-08-09 15:08
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: urgent OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: allow ${ORDER_ID} replacment by backend in fulfillment URL
Description: If the backend generates the order ID *and* the store logic requires it in the fulfillment URL, we currently use a hack where the wallet _always_ appends the order ID. We should not do this. Instead, the backend should support replacing ${ORDER_ID} with the actual order ID upon POST to /private/orders.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016575)
Christian Grothoff   
2020-08-09 15:08   
Implemented in a864161..ac7d956


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6464 [Taler] mechant backend minor always 2020-08-05 03:35 2020-08-07 01:44
Reporter: jonathanbuchanan Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: orders can be deleted even after being paid
Description: DELETE /private/orders/$ORDER_ID returns 204 when deleting a paid order, but it should return a 409.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016561)
jonathanbuchanan   
2020-08-05 21:57   
fixed in afc52d5..36e616c.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6454 [Taler] merchant backend API (C) minor have not tried 2020-07-31 19:42 2020-08-07 01:44
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: generate SVG for QR code in merchant backend
Description: RIght now, the /orders/{order_id} HTML page doesn't show a QR code, but instead displays a unicode approximation that doesn't render nicely on some browsers.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: qr.png (97,622 bytes) 2020-08-03 09:23
https://bugs.gnunet.org/file_download.php?file_id=921&type=bug
qr3.png (139,703 bytes) 2020-08-03 12:22
https://bugs.gnunet.org/file_download.php?file_id=922&type=bug
Notes
(0016541)
Christian Grothoff   
2020-08-02 23:28   
Do we still want to do this? I think the ASCII-based QR code is good enough (or even better) than SVG.
(0016545)
Florian Dold   
2020-08-03 09:23   
It doesn't display well in my browser at all, see the attached screenshot.

(Even worse, the jumbled QR code *pulsates* due to some size adjustments made by the browser when the page is reloaded.)
(0016546)
Christian Grothoff   
2020-08-03 09:28   
Hmm. That looks like a font issue. As for pulsating, I did put in some code to 'zoom' the QR code if you go over it with the mouse. That is not what you mean?
(0016547)
Florian Dold   
2020-08-03 09:30   
Yes, that actually explains the pulsating.

These font issues will be unavoidable. Are there any arguments against using SVGs?
(0016548)
Christian Grothoff   
2020-08-03 09:37   
If we can find a way to fix the font issue, SVGs just make the page larger and add avoidable complexity to the C logic. The current code is very readable. Not to mention you have already plenty on your plate. Which browser are you using?
(0016549)
Florian Dold   
2020-08-03 11:06   
I'm using Chromium 84.0.4147.

I'm not sure if we can fix the font issues reliably, as there are other variables, such as the browser's zoom setting or accessibility settings to make all fonts larger. It's much easier to precisely control the rendering of SVGs.
(0016550)
Christian Grothoff   
2020-08-03 11:54   
It works fine with my Chromium 83.0.4103.116. And I've tried with various zoom settings / font enlargements, that works fine (FF and Chromium).

I suspect your browser has trouble with:
.qrtext {
    width: max-content;
    margin: auto;
    transition: font-size 0.2s;
    font-family: 'Lucida Console', Monaco, monospace;
    font-size: 0.5em;
}
and there specifically the font-family. I certainly never installed extra fonts, but this is still what I think we should investigate first. Maybe there is an easy fix that makes this work virtually everywhere?
(0016551)
Christian Grothoff   
2020-08-03 12:04   
Hmm. The font-family is a "Web safe font" (aka: should be pre-installed everywhere). Also, I get your image ONLY if while I mess up the CSS syntax in that line and end up with a non-monospace font. Could it be that somehow you are not grabbing our CSS in full? Note that we (currently) "inline" the pure-min.css from a 3rd party site, which is likely broken on the backend.taler.net installation (this is on my list of things to fix).
(0016552)
Florian Dold   
2020-08-03 12:22   
When I manually remove the "text-align: center" on the "<div class="qr">" (why is that there?), I still get a broken result. See the dev tools for the actual CSS properties the browser loaded. I even manually changed the font selection to *only* monospace, still won't display properly.

I suspect the width of a non-breaking white space character is simply different from the unicode black box character.

And I was actually be able to confirm this: When I ask the dev tools, it tells me that the QR code div is using a combination of two "physical" fonts:

Droid Sans Mono—Local file(1988 glyphs)
Tinos—Local file(1702 glyphs)

=> We should use SVGs, as the merchant must work on systems that have *way* more broken font configurations than mine ;-)
(0016553)
Christian Grothoff   
2020-08-03 12:26   
Ok. Well, the code is (now) in taler-merchant-httpd_qr.c now, should be trivial to find for you.
(0016564)
Florian Dold   
2020-08-06 18:55   
Implemented in b4fd96d2.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5958 [Taler] bank (demonstrator) feature always 2019-11-01 22:50 2020-08-07 01:44
Reporter: Florian Dold Platform:  
Assigned To: Marcello Stanisci OS:  
Priority: high OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: won't fix  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: some accounts should not have a spending limit
Description: Accounts like "survey" shouldn't run out of funds, since this disrupts the tipping demo!

Maybe these can be marked as "service accounts".
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016528)
Florian Dold   
2020-07-27 17:34   
This is IMHO already solved in a better way right now.

We do normal "double entry book-keeping" where money is never created, but always deducted from the bank's "root" account, which can go negative.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6455 [Taler] mechant backend minor always 2020-08-01 08:40 2020-08-07 01:44
Reporter: jonathanbuchanan Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: high OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: merchant fails to lookup all orders in GET /private/orders
Description: When including orders that have been paid, the request fails with an internal server error.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016539)
jonathanbuchanan   
2020-08-02 02:33   
Fixed by ab90c57..dc4901c. The backend was not taking into account that paid orders are deleted in the db.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6446 [Taler] merchant backend API (C) feature N/A 2020-07-27 10:46 2020-08-07 01:44
Reporter: Christian Grothoff Platform: i7  
Assigned To: jonathanbuchanan OS: Debian GNU/Linux  
Priority: urgent OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: merchant backend should support claim tokens
Description: 22eadc2..0ad320c provides a draft specification.
We need to:
1) extend merchantdb to store a 128-bit claim token in the orders table
  (use non-NULL even though optional: all zeros means "no claim")
2) extend merchantdb API to store/retrieve the token where needed
3) extend server-side REST API to generate claim token if specified upon request
4) extend server-side REST API to accept claim tokens as alternative to h_contract for authorization
5) extend client-side REST APIs to match new server-side API
6) update tests.

High-ish priority as this is (a) a protocol change for v0.8 and (b) blocks me on WooCommerce. I'll likely do (1) immediately myself and may notify here if I pick up other sub-tasks.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016525)
Christian Grothoff   
2020-07-27 11:03   
3cbbe7b..01f8cd4 adds the required logic to the GET /orders/$ID request, alas without the DB support.
(0016526)
Christian Grothoff   
2020-07-27 11:08   
01f8cd4..801da06 adds basic support to POST /orders/$ID/claim, alas again without DB API/logic update.
(0016527)
Christian Grothoff   
2020-07-27 11:59   
06825b7..838f605 adds claim generation to POST /private/orders, again without DB API / logic update.
Jonathan: I'll leave it like this for now, please try to prioritize this issue over your other tasks.
(0016532)
jonathanbuchanan   
2020-07-30 00:32   
fully implemented by 783520c..f706ada.
(0016533)
Florian Dold   
2020-07-30 11:20   
Jonathan: Could you make sure that there are test cases for payments both with and without the claim token mechanism enabled?
(0016534)
jonathanbuchanan   
2020-07-31 10:01   
test for no token added in ce38476..9a13696.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6445 [Taler] merchant backend API (C) major always 2020-07-25 21:17 2020-08-07 01:44
Reporter: Christian Grothoff Platform: i7  
Assigned To: jonathanbuchanan OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: POST /private/orders is not idempotent
Description: If a client posts _exactly_ the same request to /private/orders *with* an order_id specified (!), we currently generate an error.
Instead, we should create exactly the same response a 2nd time.

(Note that this ONLY applies if the client includes the order_id; if there is no order_id, we create one at random so the 2nd response would be different.)

Fixing this requires additional logic to check that the request has an order_id and that we have exactly the same order (with the same terms) in the database. This is a bit tricky, as we modify/patch the order (adding merchant details, etc.), so really we can only do the idempotentcy check _after_ patching the incoming order.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016540)
jonathanbuchanan   
2020-08-02 04:01   
implemented in 859675f..b90d7d2.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6442 [Taler] mechant backend minor always 2020-07-24 23:34 2020-08-07 01:44
Reporter: jonathanbuchanan Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Merchant gives 409 conflct when deleting an unpaid, unclaimed order
Description: If DELETE /private/orders/$ORDER_ID is used on an order that is unclaimed and unpaid, then the merchant gives a 409 response and claims that the order is locked.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016520)
jonathanbuchanan   
2020-07-25 00:35   
fixed and tested in 31b17ff..8631026.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6435 [Taler] merchant backend API (C) minor have not tried 2020-07-21 09:28 2020-08-07 01:44
Reporter: Florian Dold Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: refund_delay parameter of POST /private/orders should go in request body
Description: Since the request body of POST /private/orders is an instruction for constructing an order, the refund_delay parameter shall go in the body of the request.

Having it as a request parameter doesn't make much sense for a RESTful API, as

   /private/orders?refund_delay=...

does not really represent a resource that will process the enclosed body of the POST request.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016524)
jonathanbuchanan   
2020-07-25 22:31   
fixed in 469ea4f..d78afeb.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6463 [GNUnet] reclaim minor have not tried 2020-08-04 18:02 2020-08-04 21:16
Reporter: schanzen Platform:  
Assigned To: schanzen OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.13.2  
    Target Version: 0.13.2  
Summary: Add well-known/openid-configuration
Description: Add a well-known/openid-configuration endpoint
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016559)
schanzen   
2020-08-04 21:16   
Fixed in ade9b5e52


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6462 [GNUnet] rest service minor have not tried 2020-08-04 18:01 2020-08-04 21:16
Reporter: schanzen Platform:  
Assigned To: schanzen OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.13.2  
    Target Version: 0.13.2  
Summary: Allow rest plugins to handle any namespace
Description: Allow, for example, the reclaim OIDC plugin to handle "/.well-known/openid-configuration" instead of only "/reclaim/*" or "/openid/*".
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016558)
schanzen   
2020-08-04 21:16   
Fixed in 815ded19f


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6461 [Taler] other minor have not tried 2020-08-03 17:24 2020-08-03 17:24
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version: 0.8  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: demo frontends should use the merchant backend HTML page for tips, refunds and payments
Description: See summary.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6456 [Taler] merchant frontend (blog) minor have not tried 2020-08-02 08:51 2020-08-03 16:59
Reporter: MS Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Automatic refresh of page not working
Description: After a payment is sent to the shop, the page should automatically refresh and show the article.
(At this moment, the refresh is done manually by the user.)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6459 [Taler] mechant backend feature N/A 2020-08-03 14:17 2020-08-03 14:26
Reporter: Christian Grothoff Platform: i7  
Assigned To: Florian Dold OS: Debian GNU/Linux  
Priority: high OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: show full contract wallet-style in merchant HTML page
Description: ASSUMING this is easily done based on what the wallet already has, we should modify the "show_order_details.en.must" template to include (existing!) JavaScript logic to properly render the JSON contract.

If such JS does not exist, and/or also in the long run, we should instead enhance our mustach template expansion capabilities so we can iterate over all contract fields and do everything mustach-template style so we can avoid needing any JavaScript.

Assigning to Florian so he can eval if existing Wallet code can easily plug this. If not, bump to 0.8.1 and we'll take care of it then via mustach.

Location in the code is marked with a FIXME and the bug number.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016556)
Florian Dold   
2020-08-03 14:21   
Contracts are currently rendered via React, i.e. by client-side DOM manipulation. That's necessary in the WebExtension wallet, as in a web extension we don't have any server to render stuff for us ;-)

We could of course also use this code in the the merchant backend HTML page, but that drags in a lot of dependencies (well, mainly React.JS) that we might not want. It also wouldn't work without JavaScript anymore.
(0016557)
Christian Grothoff   
2020-08-03 14:26   
I understand that this won't work with JS anymore, that's why I suggested this as a temporary / quick fix (which means additional dependencies can also be OK). I don't think we'll have the time to do a proper (!) and fully equivalent MUST-based rendering for 0.8, and I don't think we want to show the raw JSON or _nothing_ for 0.8. So temporarily dropping in a JS-based solution seemed like an acceptable trade-off.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6460 [Taler] merchant backend API (C) feature N/A 2020-08-03 14:22 2020-08-03 14:23
Reporter: Christian Grothoff Platform: i7  
Assigned To: OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: confirmed Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: enable serving static files from merchant backend
Description: We should enable serving certain static resources (favicon, pure.css, etc.) from the Taler backend.

Right now the purecss is loaded from unpkg.com, which is not good for privacy. (See FIXME in contrib/*.must).
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6453 [Taler] merchant backend API (HTTP specification) minor have not tried 2020-07-31 19:40 2020-08-03 13:54
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: feedback Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: come to consensus on /orders/{order_id} page and semantics of refunds
Description: Related e-mail discussion:

On 7/31/20 2:09 PM, Florian Dold wrote:

In my understanding, refunds can be
partial and incremental. That means the fulfillment page will *always*
still be shown, but it might be "degraded". So it might just show some
text "You've bought this video but have gotten a refund. Click here to
buy access again". Or "You can't download the high-res version of this
picture, as you've gotten a refund. Here's the low-res thumbnail.".

Thus I always thought we should indeed go to the fulfillment page even
after a refund. Where else should we go? Smuggling in some other URL
is not really clean. The shop should handle showing refunded
fulfillment pages nicely anyway, because the fulfillment URL might be
bookmarked.

My main problem with your suggestion (pay/paid/refund below) is that is
does not map well at all to the intention of the end user.

Generally, the end user will want to use the wallet with three different
intentions when currently interacting with a merchant:

1. Giving money to the merchant, because the merchant is offering to
give them something for it. That's the initial "taler://pay" entry
point. Here, I completely object to making the user manually click the
fulfillment URL. What if they click it too early? They'll confusingly
will land on the same page again, giving a "something's not working
correctly" feel to the whole experience. It should really
auto-redirect, let's not make users click when there is no need to.

2. Navigating to some page on the merchant's Website (article, video,
purchase receipt) that the user wants to view, because they have paid
for it. Your "paid" doesn't map naturally to this use case, since the
merchant *might not know* that the user's wallet already paid for it!
As a user, I don't really care at this point if there's a refund, I just
want to view the fulfillment page (even if it might then tell me I can't
view it anymore)

3. The user interacted with the merchant, and obtained a refund. They
now want to make sure that they actually received the refund. It is
IMHO natural to go to the fulfillment page after the refund was applied.
 But here the "taler://refund" URI has a problem: The browser might
have a fresh session ID for the shop page, and thus the fulfillment page
will prompt payment again. So it is IMHO natural that the wallet *also*
proves payment under the current session ID during/after a refund.

Now, I can live with having a separate "taler://refund" to make (3) more
visually distinct, but I think it should really just be an alias for
"taler://pay", maybe with the tiny difference that "taler://refund"
can't be used to trigger the *initial* payment.

So in summary, my suggestion is:

* keep both taler://pay and taler://refund URIs
* make *almost* be aliases for one another
* allow the shop to navigate the user to the "order HTML page" in either
the "payment" mode or the "refund" mode (via /orders/{id}?refund=<0,1>).
  * the payment mode will have a taler://pay URI
  * the payment mode will auto-redirect
  * the refund mode will have a taler://refund URI and slightly
different text
  * the refund mode won't auto-redirect, but only show a button/link
"view product page" that is *only* visible after the paid session ID
matches the browser's session ID

----------------------------

On 7/31/20 11:45 AM, Christian Grothoff wrote:
> On 7/30/20 11:24 PM, Florian Dold wrote:
>> Okay, I think I can work with a variation of that!
>>
>> But unless we use JS or put two QR codes next to each other (let's never
>> do this!), we still need some flag that tells the backend what to show
>> after the "entry point".
>
> Eh, I'm confused. I agree we should never show two QR codes. But
> _either_ the contract is paid, *or* there is a refund. It can never be
> both (no refunds for unpaid contracts), so I don't see the problem.
>
>> A link to the fulfillment page is not enough, as if you don't have the
>> cookie, going to the fulfillment URL will just bring you back to the
>> order HTML page
>
> Well, that just sounds like a(nother) good reason for not automatically
> redirecting to the fulfillment URL .
>
> We could:
> - unpaid: show taler://pay/ URI
> - paid: show a NEW taler://paid/ URI which includes the _current_
> session ID of the visitor and triggers the repurchase logic if scanned
> by a mobile wallet
> - refunded: show taler://refund/ URI

Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016538)
Florian Dold   
2020-07-31 19:41   
(We might want to discuss this outside the bug tracker, just putting it here so it doesn't get lost)
(0016542)
Christian Grothoff   
2020-08-02 23:29   
I think the consensus is described in 0006457. Also, we keep both taler://pay and taler://refund/ URIs.
(0016554)
Christian Grothoff   
2020-08-03 13:54   
I've implemented my understanding of the consensus now - modulo nice HTML.

Florian: Please check the code in taler-merchant-httpd_get-orders-ID.c to see if this is OK with you. Then resolve this one.

I'll keep 0006457 open until the HTML output is "nice".


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6458 [Taler] mechant backend feature N/A 2020-08-03 13:51 2020-08-03 13:51
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: low OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: support summary_i18n in HTML generation of merchant backend
Description: Right now, we render just the summary, even if we have language preferences and a summary_i18n in the contract.

Location in the code is marked. We should write a generic function to extract an i18n-string based on HTTP-style accept-language preferences.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6044 [Taler] documentation text N/A 2020-01-12 22:39 2020-08-02 23:33
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: merchant backend configuration unclear
Description: Florian writes:

I don't understand the configuration.

We have HONOR_instance set for the euro accounts. But the documentation
(taler.conf man page) says this (and the code also uses it):

"""
ACTIVE_instance
Must be set to YES for the instances (where “instance” is the section
name of the instance) of the merchant backend that should use this bank
account in new offers/contracts. Setting ACTIVE_instance to YES requires
also setting ENABLE_instance to YES.
"""

So the ACTIVE_foo *does* seem to matter?!

But ACTIVE_foo isn't set anywhere on euro ...
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0015644)
Christian Grothoff   
2020-04-13 21:40   
The v1 API will change how the configuration is to be done, and hopefully simplify it. Hence we should update the documentation once v1 is implemented.
(0016544)
Christian Grothoff   
2020-08-02 23:33   
Resolved, as these options are now history. The accounts are now fully configured via the REST API to POST to /private/instances.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6066 [Taler] bank API (C) feature N/A 2020-01-26 10:35 2020-07-31 18:01
Reporter: Christian Grothoff Platform: i7  
Assigned To: MS OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: reopened  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: bank should have an API to ask it for the currency it uses
Description: This will be needed for Apps where the user configures a bank connection via URL, username and password and we then need to use the withdraw APIs with a currency. Right now, the app has to magically know the currency or ask the user, both of which is inconvenient. We likely should keep the API endpoint name generic, as there might be other things (especially protocol version) to include in the future as well.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0015306)
Christian Grothoff   
2020-02-04 21:49   
bank API specification was updated.
(0015307)
Christian Grothoff   
2020-02-04 22:04   
taler_bank_lib.h and the fakebank support /config as of e6d6987e..42bc3174
(0015308)
Christian Grothoff   
2020-02-04 22:05   
Marcello: could you please implement the new "/config" endpoint in nexus and the Python bank? Specification is already done, you only need to return {"version":"0:0:0","currency":"$CURRENCY"} for now.
(0016492)
MS   
2020-07-21 12:17   
Implemented.

Python bank: 7b85b8731693216f967668e5444cbc85e1f6d995
Nexus: 71201a801ddc58a06392941f808c3ca5148edeea

Note that Nexus does not read configuration values from any files yet,
therefore the currency is hard-coded to "EUR".
(0016535)
grote   
2020-07-31 14:30   
The test bank currently gives this response:

% curl "https://bank.test.taler.net/config"
{"version": "0.0.0", "currency": "TESTKUDOS"}

which fails parsing, because "0.0.0" != "0:0:0"
(0016537)
MS   
2020-07-31 18:01   
Fixed here: 32652cc335b140c1e5aaa2f2b13089cc53c3932c


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6427 [libeufin] sandbox minor have not tried 2020-07-09 19:41 2020-07-29 15:41
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: low OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: EBICS errors need XML
Description: In some (all?) error cases related to EBICS, the sandbox responds with plain text.
Instead it should respond with the XML document that complies to the EBICS specifications.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016507)
MS   
2020-07-23 13:42   
(Last edited: 2020-07-23 13:44)
Mostly addressed here: c716d23073be8846d0765fa766de341dbc953a88

(0016508)
MS   
2020-07-23 13:48   
The EbicsError global catcher needs a private key to be included in the exception object,
and if that's not found, it responds with JSON (and status 500). (The private key is needed
to sign the response object.)

The current problem is that not in all the parts of the code such private key is included in
the exception object, resulting still in JSON+500 being returned.
(0016509)
MS   
2020-07-23 17:13   
After private discussion, the private key will be moved away from the exception object. But at the same time, the exception handler
must be moved in a place where it is easier to get more context (including the private key) from the current request.
(0016531)
MS   
2020-07-29 15:41   
Fixed, but this test now fails: integration-tests/test-ebics-double-payment-submission.py


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6448 [Taler] wallet (JS core) minor have not tried 2020-07-28 20:25 2020-07-29 13:42
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9  
Summary: auditors/exchanges UX; wallet should have better support for regional currencies
Description: Regional currencies should be handled in a special way by the wallet. Balances for regional currencies should always be specified per exchange.

We should consider currency codes that are >3 letters a regional currency.

For non-regional currencies, we should always require a trusted auditor.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016530)
grote   
2020-07-29 13:42   
Added documentation notes:

https://docs.taler.net/design-documents/002-wallet-exchange-management.html#trust


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6449 [Taler] Merchant frontends (Python3) minor have not tried 2020-07-28 20:38 2020-07-28 20:38
Reporter: Florian Dold Platform:  
Assigned To: Marcello Stanisci OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: merchant frontend demos should use the backend's order display page
Description: The merchant backend now implements all the logic to display the page that (1) triggers the wallet and (2) shows a QR code as fallback.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6447 [Taler] other minor have not tried 2020-07-28 16:45 2020-07-28 16:45
Reporter: MS Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: i18n for demo sites needed.
Description: Using the same technology used for the main Taler site.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6443 [Taler] merchant backend API (HTTP specification) minor N/A 2020-07-24 23:43 2020-07-28 03:52
Reporter: jonathanbuchanan Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Expand information returned in GET /private/orders.
Description: Currently, this method is only specified to return the id of each order, but the implementation also returns the row number and timestamp. The spec should reflect this, and also should return the order amount and summary.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016521)
jonathanbuchanan   
2020-07-25 03:07   
Spec changed in 471480d..b2deb4d.
(0016529)
jonathanbuchanan   
2020-07-27 22:09   
Implemented in 608d276..5907099.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6444 [Taler] merchant backend API (C) major always 2020-07-25 12:55 2020-07-25 20:23
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: not fixable  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: merchant backend returns 100 continue before 404 on POST /private/orders
Description: Instead of failing immediately, the backend first 404s and then fails shortly thereafter with a 404.
Somehow we don't do the right thing (TM) and check early on for the 404 failure case.

Also odd: we still seem to fail before the actual upload succeeds, at least with my current WooCommerce test.

Tags:
Steps To Reproduce: Post to backend where no instance is configured. Transcript from wireshark:

POST //private/orders HTTP/1.1
Host: backend.test.taler.net
Accept: */*
Authorization: ApiKey sandbox
Content-Type: application/json
Expect: 100-continue

HTTP/1.1 100 Continue

HTTP/1.1 404 Not Found
Server: nginx
Date: Sat, 25 Jul 2020 10:46:15 GMT
Content-Type: application/json
Content-Length: 57
Connection: keep-alive
Vary: Accept-Encoding
Access-Control-Allow-Origin: *

{
  "code": 2000,
  "hint": "merchant instance unknown"
}
Additional Information:
System Description
Attached Files:
Notes
(0016522)
Christian Grothoff   
2020-07-25 20:19   
Turns out this is the nginx reverse proxy's 100 continue. It just doesn't wait for the backend to react :-(.
(0016523)
Christian Grothoff   
2020-07-25 20:21   
And they wontfix'ed the bug 7 years ago: https://trac.nginx.org/nginx/ticket/493


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5210 [Taler] merchant backend API (HTTP specification) feature have not tried 2017-12-10 20:57 2020-07-24 22:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: merchant's /pay should also accept own signature for replay to save traffic and wallet/merchant storage
Description: Currently even if coins are expired, we need to keep (at least parts of) them around, so we can to a /pay replay when we don't have the session cookie for e.g. an article.

Instead /pay should also accept the signature that the merchant previously gave.
 This way the merchant backend and wallet can garbage collect coins after they expired.

If we don't implement this, we either
a) must accept that after coins expired we can't do a replay and if cookies are lost we can't view digital goods anymore
b) need to store coins forever in the merchant and wallet
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015642)
Christian Grothoff   
2020-04-13 21:36   
Can we define another endpoint for this? Anything except /pay, to avoid overloading it?
(0016489)
Christian Grothoff   
2020-07-18 21:49   
c95e975..ece354e (docs.git) specifies /paid as a new endpoint for this.
(0016519)
jonathanbuchanan   
2020-07-24 22:56   
Implemented in the merchant in 26c6bb9..b6cb1cc.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6441 [libmicrohttpd] libmicrohttpd API minor N/A 2020-07-24 18:21 2020-07-24 18:21
Reporter: vmatare Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version: 0.9.71  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: MHD_queue_basic_auth_fail_response should also return MHD_Result
Description: microhttpd.h in v0.9.71 still has:

_MHD_EXTERN int
MHD_queue_basic_auth_fail_response (struct MHD_Connection *connection,
                                    const char *realm,
                                    struct MHD_Response *response);

which is inconsistent with the use of MHD_Result throughout the rest of the API. The doxy also says "@return #MHD_YES on success, #MHD_NO otherwise", so it looks like a simple oversight to me.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6174 [Taler] Web site(s) feature have not tried 2020-04-11 08:20 2020-07-24 17:33
Reporter: buckE Platform:  
Assigned To: Christian Grothoff OS:  
Priority: low OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: no change required  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Improvement: add wp-cli support to GNU Taler WordPress plugin
Description: If the GNU Taler Wordpress/WooCommerce plugin will be used programmatically (ex. in scripts) it will be necessary to support wp-cli.

Doc on subject: https://wpreset.com/add-wp-cli-support-wordpress-plugin/

Tags:
Steps To Reproduce:
Additional Information: Recommended items to automate:

Title (--title)
Description (--description)
GNU Taler Back-end URL (--back-end)
GNU Taler Backend API Key (--api-key)
GNU Taler Payment URL (--payment-url)
Summary Text of the Order (--summary-text)
Send Merchant Information? (--send-merchant-info)
Name of This Webshop (--webshop-name)

Example:

$ wp taler set --title="GNU Taler Patments" --description="Pay with GNU Taler" --back-end="http://backend.demo.taler.net" --api-key="asdfasdf" --payment-url="https://backend.demo.taler.net" --summary-text="Using GNU Taler will make you more attractive." --webshop-name="My Store" --send-merchant-info
  
Argument against: WooCommerce requires GUI intervention anyway. So full automation is not possible for https://bugs.gnunet.org/view.php?id=6144. However this is a good feature for large/bulk/geek deployments.
Attached Files:
Notes
(0015620)
Christian Grothoff   
2020-04-11 11:07   
One argument for this aside from full automation: once this exists, we can also in our own Documentation tell people to run 'command X/Y/Z' instead of (or in addition to) 'click here, then there', and it is likely that giving explicit shell commands to run is more reliable and faster for site administrators than finding things in the (more likely changing, possibly translated) menus.
(0015633)
buckE   
2020-04-13 03:34   
I do think this is a good thing to add, especially because it is easy. However I disagree with this statement:

> it is likely that giving explicit shell commands to run is more reliable and faster for site administrators than finding things in the (more likely changing, possibly translated) menus.

Actually, it's very unlikely. It would be true of a good/robust/unix-thinking system, and if wp-cli were build by WP. But WordPress is a GUI-first system, wp-cli is not part of any default WP install, and I'm not even sure what the relationship is between the WP team and the WP-cli team. These add up to making wp-cli a real pain, unreliable, really not smart to use in a script, and it's why I would have preferred to use standard tools to automate the WooTaler site instead of spending so much more time on wp-cli. I've built big, automated WP installs before. I've never played with rinky-dink wp-cli until it was specified as a late-entry requirement in the WooTaler build.

It should never (until wp-cli is part of a default WP install) be recommended to do things with a wp-cli-first approach, particularly as long as WP specifically recommends a GUI-first approach and even requires it for basic things like WooComerce first-run. Oh, and until wp-cli has a good API because Jesus! *But* by supporting it in our plugin, we make it more useful and more likely that WP will someday take over wp-cli.

My recommendation is that the current plugin devs take a quick look at the above link, just to keep a vague mental map of what hooks might be needed as they develop future versions. Then when they release the next version if you want me to add support, I'll do that. (But let's not hold up the next version for this.)
(0015637)
Christian Grothoff   
2020-04-13 10:47   
Oh, I didn't know WP-cli was *not* part of a core Wordpress installation. I thought it was (see, I don't know much about Wordpress!). And yes, this is not release critical (hence no target version in our system). Oh, and we don't currently have developers taking care of the Taler plugin...
(0016517)
Christian Grothoff   
2020-07-24 17:32   
Actually, this whole bug was bogus: the plugin already can be fully configured, simply providing the 'settings' in JSON using:

 wp wc --user=admin payment_gateway update gnutaler --enabled=true --settings='{"title":{"id":"title","label":"Title","description":"This is what the customer will see when choosing payment methods.","type":"text","value":"GNU Taler","default":"GNU Taler","tip":"This is what the customer will see when choosing payment methods.","placeholder":""},"GNU_Taler_Backend_URL":{"id":"GNU_Taler_Backend_URL","label":"GNU Taler Back-end URL","description":"Set the URL of the GNU Taler back-end. (Ex: https:\/\/backend.demo.taler.net)","type":"text","value":"https:\/\/backend.demo.taler.net\/","default":"","tip":"Set the URL of the GNU Taler back-end. (Ex: https:\/\/backend.demo.taler.net)","placeholder":""},"GNU_Taler_Backend_API_Key":{"id":"GNU_Taler_Backend_API_Key","label":"GNU Taler Backend API Key","description":"Enter your API key to authenticate with the GNU Taler back-end.","type":"text","value":"Sandbox","default":"","tip":"Enter your API key to authenticate with the GNU Taler back-end.","placeholder":""},"Payment_url":{"id":"Payment_url","label":"GNU Taler Payment URL","description":"Enter the URL for your (merchant) GNU Taler Wallet, where customer coins will be sent.","type":"text","value":"payto:\/\/x-taler-bank\/bank.demo.taler.net\/Merchant","default":"","tip":"Enter the URL for your (merchant) GNU Taler Wallet, where customer coins will be sent.","placeholder":""},"Order_text":{"id":"Order_text","label":"Summary Text of the Order","description":"Set the text the customer will see when confirming payment.","type":"text","value":"Please Confirm Order","default":"Please Confirm Order","tip":"Set the text the customer will see when confirming payment.","placeholder":""},"merchant_information":{"id":"merchant_information","label":"Enable sending your merchant information to the GNU Taler back-end","description":"Do you want to send your merchant information to the GNU Taler back-end with this transaction?","type":"checkbox","value":"yes","default":"yes","tip":"Do you want to send your merchant information to the GNU Taler back-end with this transaction?","placeholder":""},"merchant_name":{"id":"merchant_name","label":"Name of This Webshop","description":"Set the name of this webshop that the customer will see during the payment transaction.","type":"text","value":"GNU Taler WooCommerce Demonstrator","default":"","tip":"Set the name of this webshop that the customer will see during the payment transaction.","placeholder":""}}'
(0016518)
Christian Grothoff   
2020-07-24 17:33   
Deployed as this in buildWebstore.sh.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6107 [Taler] deployment and operations major always 2020-02-24 17:03 2020-07-24 17:18
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: low OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: taler-backoffice fails to start in test-deployment
Description: taler-test@gv:~$ taler-deployment-arm -I
Services (excluding stopped services):
(started: 10 / failed: 1 / stopped: 4)
taler-exchange (binary='taler-exchange-httpd', status=started)
taler-backoffice (binary='taler-log-adapter', status=failed, exit_status=255, restart_delay='46 s')
taler-auditor (binary='taler-auditor-httpd', status=started)
taler-demobank (binary='taler-log-adapter', status=started)
taler-donations (binary='taler-log-adapter', status=started)
taler-survey (binary='taler-log-adapter', status=started)
taler-sync (binary='sync-httpd', status=started)
taler-aggregator (binary='taler-exchange-aggregator', status=started)
taler-exchange-wirewatch (binary='taler-exchange-wirewatch', status=started)
taler-blog (binary='taler-log-adapter', status=started)
taler-merchant (binary='taler-merchant-httpd', status=started)
Tags:
Steps To Reproduce: Look at taler-test account.
Additional Information:
System Description
Attached Files:
Notes
(0015406)
Christian Grothoff   
2020-02-25 12:47   
After building ('make install') from hand, it now launches. Still, very strange as the BB should have done the exact same instructions. To be investigated further...


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5810 [Taler] deployment and operations minor have not tried 2019-07-20 12:19 2020-07-24 16:53
Reporter: Marcello Stanisci Platform:  
Assigned To: Marcello Stanisci OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Survey reserve from 'demo' needs automation
Description: The tip reserve from demo suffers from the following two problems:

1) it is not automatically topped up.

2) it is not automatically "refreshed". In particular, even if the reserve did
  not expired and was replenished with money, the merchant backend still
  needs to receive a "/tip-query" call in order to update its database and flag
  the reserve as active. If we miss this step, we get that 2702 error.

I have manually patched the situation by:

1) log in as the active demo user (demo-blue now)
2) source activate && taler-deployment-top-reserve
3) visit survey.demo.taler.net, and *click* "survey stats"
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015189)
davidak   
2019-12-19 04:16   
Is this the reason for the demo beeing broken?

Taler Survey Demo
An Error Occurred
Backend returned error status

The backend returned status code 404.

Backend Response:

{'code': 1151, 'hint': 'Reserve unknown at exchange'}


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6440 [Taler] wallet (JS core) minor have not tried 2020-07-24 16:04 2020-07-24 16:04
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: wallet should support aborted withdrawal operations
Description: The bank and cashier already support this feature, the wallet currently ignores the abort status.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6431 [Taler] exchange trivial have not tried 2020-07-16 20:06 2020-07-24 12:50
Reporter: Florian Dold Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8.1  
Summary: consider committing the generated error codes header
Description: Currently a "git pull" doesn't pick up new error codes, since only "./bootstrap" generates the taler_error_codes.h.

Additionally, we now have an (undocumented) dependency on recutils, which is not really necessary (and recutils is less widely packaged than our other dependencies).

Is there any good reason to not just commit the error codes file?
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016479)
Christian Grothoff   
2020-07-16 21:12   
Well, mostly lots of additional unnecessary stuff in Git. recutils is then really a developer-only dependency, and developers are likely to need to frequently add error codes, so that dependency just won't go away.

And everyone else won't be affected. So I actually like the current situation.
(0016490)
Florian Dold   
2020-07-20 14:47   
(I started to think about how to do this because wallet-core.git also needs some way to handle the generated error codes. So this is more a discussion of how our project handles this type of thing in general, not just for this one file.)

Every time we update error codes, we need a git commit anyway. Either with the contents of the file or a new submodule commit hash. Having the generated source code "inline" in the commit makes it easier to audit the change.

I tend to be against having source files that are auto-generated but not directly committed to the repository:

1. It makes it harder for people to compile the code base
2. It kills "jump to definition" in online code browsers (cgit doesn't have this, but GitHub/GitLab) and local tools, unless you at least bootstrap the repository (
3. It likely makes it harder for static analysis tools that don't run a full build

Furthermore, the current approach totally breaks incremental compilation. Every single change to the error codes requires a full rebuild (due to having to do ./bootstrap), which is annoying during development.
(0016495)
Christian Grothoff   
2020-07-22 15:08   
Current discussion/compromise: modify build to avoid re-generation of files only via bootstrap, instead make it part of the regular build (Makefile[.am]).
(0016501)
Christian Grothoff   
2020-07-22 21:35   
I've changed the scripting to minimize the impact of GANA regeneration on incremental compilation:
1) bootstrap still triggers contrib/gana.sh
2) manually running contrib/gana.sh can be used to update 'on the spot' without re-running bootstrap OR configure
3) running contrib/gana.sh is harmless if nothing changed (no change to timestamp, no full recompile!)
4) GANA-internal update was optimized to run a smaller set of the recutils steps if possible.

(0016511)
Florian Dold   
2020-07-24 09:40   
This doesn't solve the issue. One still needs to remember to do "./contrib/gana.sh" after doing a pull. My idea would've been to add a make rule that automatically re-builds the file.

Or, you know, just add the generated file in git and let the developer that adds an error code figure out these issues, not the poor person/CI who just wants to build the code.
(0016513)
Christian Grothoff   
2020-07-24 11:53   
Eh, you need to remember to contrib/gana.sh after changing GANA. Not after every pull, right? At least for me it seems to work fine assuming the committer did the GANA change properly.

If 'make' were to automatically pull GANA, that would be terrible if we ever remove an error code from GANA, as then older sources would start to FTBFS, as they would try to pull in a more recent GANA. Also, it must be possible to run 'make' while offline.

Not having the generated file in Git saved me already a ton of work with codespell, as it didn't yell at me for the GANA spelling mistakes multiple times.
(0016515)
Florian Dold   
2020-07-24 12:18   
I'm not saying that "make" should *pull*, but that make should at least update src/include/taler_error_codes_h if the file in the registry.rec file in the submodule changed because of a "git pull".

My main concern with the current state is that it has lead to about half a dozen issues so far, mostly on the side of Marcello's CI stuff. All because of the gana code generation.
(0016516)
Christian Grothoff   
2020-07-24 12:50   
Ah, got your point. Should be better now.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6439 [Taler] Anastasis feature N/A 2020-07-24 12:09 2020-07-24 12:09
Reporter: Christian Grothoff Platform: i7  
Assigned To: OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: new Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9  
Summary: wallet should support Anastasis
Description: This is kind-of a meta-bug, as it is about both wallet-core support and support in the various wallet user interfaces.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6045 [Taler] documentation text N/A 2020-01-12 22:42 2020-07-24 12:08
Reporter: Christian Grothoff Platform: i7  
Assigned To: Stefan OS: Debian GNU/Linux  
Priority: low OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9.1  
Summary: user confusion: peer-to-peer payments in bank demonstrator
Description: someone wrote:

I tried the taler demo. While I'm not very happy about needing a browser
extension to do the demo, I'm not clear on how Taler provides payment
without registration as a result of it (as the main page states).

For example, if I want to send a kudo directly to another friend with a
taler wallet, how would I do so? From the looks of it, I have to
register for a (demo) bank account and send it to someone else who also
has a (demo) bank account.
Tags:
Steps To Reproduce:
Additional Information: User is confused by the transfer between bank accounts 'feature' of the bank, which isn't really for *Taler* payments. Maybe we should remove it/hide it better / disable on demo and/or make it clear that this is NOT Taler? (UX question!).

Also highlights that we need to plan for the peer-to-peer wallet-to-wallet payments UX, *and* document better that *today* Taler does not support peer-to-peer payments.
System Description
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5019 [Taler] bank (demonstrator) tweak always 2017-05-20 20:34 2020-07-24 12:08
Reporter: Christian Grothoff Platform: i7  
Assigned To: Marcello Stanisci OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9.1  
Summary: taler bank should follow PEP 484 and 526 where applicable
Description: PEP 484 specifies a syntax for annotation arguments with types.

Wherever possible, we should add such type annotations.
Tags:
Steps To Reproduce:
Additional Information: https://www.python.org/dev/peps/pep-0484/
https://www.python.org/dev/peps/pep-0526/
System Description
Attached Files:
Notes
(0012144)
Christian Grothoff   
2017-05-22 14:40   
Once this approaches completion, the mypy tool should be integrated with the buildbot:

http://mypy-lang.org/
(0012628)
Marcello Stanisci   
2017-12-06 17:55   
arguments like '*args' and '**kwargs' will be type-annotated at last, as this requires special care.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6366 [Taler] deployment and operations feature N/A 2020-06-08 11:40 2020-07-24 12:08
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9.1  
Summary: Build Taler exchange + GNUnet-Taler Debian package
Description: We eventually want to create a Debian installer which provides a minimum Taler exchange setup (minimum in that there is no unnecessary code/service on the system, to minimize the security footprint). Our setup should pull packages from our own Debian archive (possibly in combination with the standard Debian archive), as we must be able to deploy urgent security updates even if Debian has not yet packaged them.

As a first step towards this setup, we need a Debian package for the Taler exchange. This would merely be a first package that our server would then 'apt install' by default. The package will evolve, just like the software will still evolve, for now the goal is to create a starting point for this evolution. The package is expected to simply include all of the binaries and shared libraries of the Taler exchange. Configuring/starting the service is out of scope.

The Taler exchange depends on various other packages, including GNUnet, libjansson and Postgres. Except for GNUnet, you should for now build the exchange it against existing Debian packages. For GNUnet, you should create a custom 'gnunet-taler' package (which contains GNUnet *for* Taler, not Taler itself), which would be largely a simplification (!) of the existing GNUnet packages for Debian. The main changes are:
- build with Postgres and libjansson support (so that the libraries needed by Taler are actually included)
- remove anything related to starting a GNUnet service, setting up SUID/SGID binaries or accounts or groups (gnunet/gnunetdns)
- remove MySQL (if present), remove libextractor (or possibly other dependencies Taler does not require => minification)

The result would be two source packages, one for "gnunet-taler" and one for "taler-exchange", which could build a recent version of the Taler exchange against a recent version of GNUnet (exact versions don't matter right now -- as long as they work together). The build rules for both source packages should then be in the exchange.git/ repo.

Once those two packages exist, setup the Debian archive "somewhere" on taler.net (yes, will need help from root) to host the source packages and binaries. Buildbot should be configured to build "nightly" package binaries from source.

Then, you modify the USB-installer to use our Debian archive and install the taler-exchange package by default.

Future steps (after this bug):
- package other Taler components to make it easier for people to deploy Taler instances
- improve packages to automatically perform key setup steps (database)
- improve USB installer to allow us to make configuration changes post-installer via Git
- improve Taler exchange to make configuration itself easier (hence we don't worry about
  making the package create a good configuration: this will still change)
- improve USB installer to install backup and monitoring logic, etc.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016258)
buckE   
2020-06-11 10:01   
This looks like a number of tasks. But reading it a few times, it looks like the foundational task (ie, the first one to execute) is this:

```
you should create a custom 'gnunet-taler' [debian] package (which contains GNUnet *for* Taler, not Taler itself), which would be largely a simplification (!) of the existing GNUnet packages for Debian. The main changes are:
- build with Postgres and libjansson support (so that the libraries needed by Taler are actually included)
- remove anything related to starting a GNUnet service, setting up SUID/SGID binaries or accounts or groups (gnunet/gnunetdns)
- remove MySQL (if present), remove libextractor (or possibly other dependencies Taler does not require => minification)
```

If this sounds correct, these are my questions:

1 - This appears to be specifying a binary deliverable in the form of a debian package for gnunet-debian. So, gnunet-debian.deb. If that is not the deliverable for this specific task, please specify what the deliverable is.

2 - This binary .deb must be built for which Debian version? Stretch?

3 - Should I use the gnunet-debian.git codebase for this?

4 - Is that codebase documented properly to do the things you said? For example, is it clear in the README how to "build with Postgres and libjansson support," how to "remove anything related to starting a GNUnet service, setting up SUID/SGID binaries or accounts or groups," and how to "remove MySQL (if present), remove libextractor?"

4b - or at least does the README direct one to a place where this is specified?

4c - From where exactly will I get the list of "possibly other dependencies Taler does not require?" If this has not been determined previously, it sounds like something that can only be built in a later version of the gnunet-debian.deb package, because it will only be clear what Taler does not require after the Taler deb is built, after building this dependency. That's fine (and normal), but I want to be clear.

5 - Are any other changes to the gnunet-debian.git code and documentation, required for this task, not specified? Obviously we can't know everything, and we may have to quickly re-iterate on gnunet-debian.deb once we begin on taler.deb and learn that we need more/fewer things. But please don't hold back anything you can think of.

If all of these are clear and documented, I can begin. If not, then please have this dependency (gnunet-debian.git) prepared for this task before I begin. Then I can execute this task, and move on to the next step.

Unambiguously, the best procedure would be for the gnunet-debian and taler dev (or at least someone with some experience building taler on gnunet) to do a run-through of installing taler now, updating the README and making a quick howto/notes list including the above changes from defaults. A 2-hour run-through from an expert, properly documented, will save this n00b many hours of trial and error and frustratingly unclear error message decoding (including asking the dev for help understanding the error messaging, which will take much more than the 2 hours it would take to properly prepare the dependency).

Either way you choose I will proceed once I have the above info.
(0016259)
Christian Grothoff   
2020-06-11 14:31   
1) Well, the gnunet-debian.deb is the final output, but of course we primarily care about the scripts/inputs to produce that .deb, as we will likely have to update it many times in the future. So a script to _produce_ the .deb would be the real deliverable. Always automate, and we're GNU, so we always care about source and not binaries.

2) I don't care which Debian version, stable, testing, unstable, doesn't really matter at this stage.

3) Whatever starting point makes your life easier is fine.

4a) Not sure, removing MySql/libextractor should be trivial -- remove the build-deps and GNUnet should build without those. But ultimately, if the documentation is insufficient / gives you grief, file more bugs and/or ask for help.

4b) Maybe. Mostly I was talking about whatever existing GNUnet debian package you may find. I do not know how well those are documented.

4c) MySql, libextractor are the 'main' ones that come to mind. Again, if the list is incomplete, we'll just iterate. It is fine if you have 'excessive' dependencies in the first iteration(s).

5) Not that I can think of.

Anyway, we _believe_ the current instructions at https://docs.taler.net/taler-exchange-manual.html#installation are adequate and current. If not, please do file bug reports explaining what you encountered and/or improve them if you can.
(0016274)
buckE   
2020-06-16 06:31   
(Last edited: 2020-06-16 07:35)
You're making my head spin. I'm back to not knowing what this is all about. I did everything I could to parse that huge text into deliverables, and again you tell me no (or at least, not "yes!") to every thing I said. I don't know how to begin to interpret this and convert it into tasks at this point.

Maybe when I have time to sit down and parse it all apart.

(0016333)
buckE   
2020-06-22 06:27   
Okay I've made some progress with buildbot and I'll start pulling this apart.
(0016375)
buckE   
2020-06-29 05:24   
I decided the only way to proceed is to try installing the exchange. I installed dependencies (including gnunet from apt) and I'm stuck on (4) under "Getting Started" in Exchange README:

`$ taler-exchange-keyup -m master.priv -o auditor.in`

This produces:

`The binary is not on the system (`$ find . -name 'taler-exchange-keyup'` produces no results).`

So, did the README skip a step? Perhaps compiling?

`$ cd exchange`
`~/exchange$ make`

Produces:

`make: *** No targets specified and no makefile found. Stop.`

What am I missing?


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5830 [Taler] merchant backend API (C) feature have not tried 2019-08-17 19:26 2020-07-24 12:07
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: none OS Version:  
Status: confirmed Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9.1  
Summary: implement request tunneling for wallets
Description: The /proxy API should be implemented to allow the wallet to tunnel requests to the exchange and merchant backend via a PoS terminal.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0014870)
Florian Dold   
2019-09-06 11:51   
Moving this to a later milestone.

Currently the Android PoS app does the tunneling. Doing the tunneling through the merchant backend would be much nicer, but is not really urgently needed.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
3476 [Taler] wallet (WebExtensions) feature have not tried 2014-07-01 18:49 2020-07-24 12:07
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: confirmed Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9.1  
Summary: evil exchange testsuite
Description: There should be an "evil" exchange testsuite, which tries to trigger edge- or error cases in the customer implementation.

Preferably, the result of these test cases should be that the customer can always construct a cryptographic proof that demonstrates the exchange's failure. This is of course only possible for byzantine failures, and not for denial-of-service.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0012906)
Christian Grothoff   
2018-03-27 17:30   
With Twister, we now have in principle the infrastructure in place for this, but we naturally need to modify the (existing?) wallet tests to trigger the Twister at the right times. This sounds like it would be best done in May or June as a collaboration between Marcello and Florian.
(0013233)
Christian Grothoff   
2018-09-17 11:31   
Changing to be against wallet, as this is about testing the wallet and the wallet thus needs to do the required twister instrumentation. (Do we have a separate bug for the merchant backend? Do we need one?)


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6034 [Taler] exchange feature have not tried 2020-01-07 19:17 2020-07-24 12:07
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: low OS Version:  
Status: acknowledged Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9.1  
Summary: Exchange should have separate configuration file
Description: As discussed in a private e-mail, this has a bunch of advantages:

* It is easier to validate a configuration, as we can warn about unused sections and options in some sanity checking tool

* Having one file per component makes it easier to generate more complex setups for testing (i.e. two merchants, two auditors, maybe even two exchanges with different currencies)

* We have had configuration section "clashes" in the past, where the merchant parsed a section intended for the exchange, due to the same prefix. Separate files avoid this issue.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
4810 [Taler] wallet (WebExtensions) feature have not tried 2016-11-20 20:06 2020-07-24 12:05
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: the wallet should support backup and sync
Description: Chrome (and other browsers) offer sync of settings and small values, but we also need to store larger amounts of data, and we need to keep privacy considerations a priority.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0012051)
Christian Grothoff   
2017-04-19 20:26   
https://www.kinto-storage.org/ seems relevant.
(0013337)
Christian Grothoff   
2018-11-14 11:36   
On design choices (background):
https://hacks.mozilla.org/2018/11/firefox-sync-privacy/


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5965 [Taler] auditor text N/A 2019-11-14 22:28 2020-07-24 12:04
Reporter: Christian Grothoff Platform: i7  
Assigned To: Stefan OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: taler-auditor needs better documentation
Description: While there is now a short chapter on the auditor, key things are missing:
- auditor must probe link API [DONE]
- auditor should do probabilistic checks on refreshed coins being reported [DONE]
- need documentation on how the auditor CODE internally works (including auditor DB graphic!) [DONE]
- need documentation on how the auditor TEST works [DONE]
+ someone else should read over it to make sure it is clear
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6112 [Taler] bank (demonstrator) minor N/A 2020-02-25 18:39 2020-07-24 12:04
Reporter: Florian Dold Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: test cases are incomplete for new bank APIs
Description: The test cases are pretty outdated.

We should have basic tests for the following bank APIs:

* wire gateway API (https://docs.taler.net/core/api-wire.html)
* bank integration API (https://docs.taler.net/core/api-bank-integration.html)
* bank access API (https://docs.taler.net/core/api-bank-access.html)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5190 [Taler] deployment and operations tweak have not tried 2017-12-05 13:59 2020-07-24 12:03
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: confirmed Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: test, demo and buildbot do not restart on reboot despite cron job
Description: We have a @reboot cron job, but for some reason it does not actually start these services. According to the syslog, these jobs *are* executed, but maybe the file system or other services are not ready yet?

It looks like buildbot does not have a @reboot cronjob at all.
Tags:
Steps To Reproduce:
Additional Information: Dec 5 13:19:47 tripwire kernel: [ 19.645407] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
Dec 5 13:19:47 tripwire cron[897]: (CRON) INFO (Running @reboot jobs)

[...]

Dec 5 13:19:48 tripwire CRON[945]: (test-blue) CMD (source $HOME/activate; taler-deployment-restart)
Dec 5 13:19:48 tripwire CRON[947]: (demo-blue) CMD (source $HOME/activate; taler-deployment-restart)
Dec 5 13:19:48 tripwire CRON[948]: (test-green) CMD (source $HOME/activate; taler-deployment-restart)
Dec 5 13:19:48 tripwire CRON[944]: (demo-green) CMD (source $HOME/activate; taler-deployment-restart)
Attached Files:
Notes
(0012901)
Marcello Stanisci   
2018-03-27 11:54   
Not sure how to test this! I guess someone should plug a monitor into
Tripwire and see what happens on reboot (?)


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5409 [Taler] codeless payments minor have not tried 2018-07-21 00:11 2020-07-24 12:02
Reporter: shivamkohli Platform:  
Assigned To: Marcello Stanisci OS:  
Priority: normal OS Version:  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9.1  
Summary: Payto URI
Description: Add paytoURI in the order.
Ask for payto URI while a new merchant is registering on codeless payment platform.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0013153)
Christian Grothoff   
2018-07-28 11:46   
We'll need more than a payto://-URI from merchants, as per recent discussions with Marcello. Marcello has been working on a merchant registration page in the Taler backend. So please either coordinate with him, or just leave this one to Marcello entirely.
(0013155)
Marcello Stanisci   
2018-07-28 11:53   
Would also be great to know something more about 'codeless' payments; does a description exist somewhere?
(0013156)
Christian Grothoff   
2018-07-28 11:55   
I don't know if more documentation was written yet, but from GSoC we have this description: "To accept payments with GNU Taler currently some custom (though simple!) code must be written to integrate with an existing web shop or frontend. The goal of this project (codeless payments) is to create a component that sits between the seller's frontend and the GNU Taler merchant backend. This component should have a web interface, where payment buttons or payment forms can be configured that can by copy-pasted into the seller's frontend as static HTML/JS. Bonus goals are inventory management, where the seller can configure the available stock for an item, and will get notified when their stock runs low."
(0013157)
Christian Grothoff   
2018-07-28 11:56   
Anyway, Marcello, did you do any documentation / API specs on the 'merchant registration' logic that you proposed to add to the back office yet? Because I think this is precisely what this bug is about and how it should be addressed ;-)
(0013160)
Marcello Stanisci   
2018-07-28 13:34   
(Last edited: 2018-07-28 13:35)
I did not spec anything for now; as per recent discussion with you, it seems that this data should be collected at _installation_ time, that makes me wonder again if you intend to get the sysadmin enter this information. In this case, there is no need to spec anything for the back-office site.

(0013161)
Christian Grothoff   
2018-07-28 15:34   
Not the sysadmin, but the last step of the backend install could be something like telling the admin that to create/fully configure (?) in instance, they should point the business people to a particular form that they should fill out (per instance).
(0013162)
shivamkohli   
2018-07-28 16:25   
Codeless payment is a platform for the merchant where they can manage their inventory and simultaneously create a 'Buy Now' button for the specific product. This code can be directly copy pasted on the seller's frontend and can be used for 'Pay with Taler'. The documentation for 'codeless' payment is under progress and with be available soon.
For the payto://-URI, I will contact Marcello.
(0014878)
Christian Grothoff   
2019-09-09 15:56   
shivam: do you still plan to work on this?
(0014885)
shivamkohli   
2019-09-11 15:16   
Yes, I can work on this. Please, can you help me get started?
(0015262)
Christian Grothoff   
2020-01-12 22:43   
Marcello writes (truncated): I *try* to recollect the pieces. At that time I was supposed to write some Web interface to let business people enter merchant's bank details into their configuration.

This appeared to match the 'codeless' job, since merchants should at some point give their bank details there as well. That was my
only "involvement".

But on my side, I have *no idea* why the issue suggests to add the merchant's payto-URL into the order.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
4117 [Taler] documentation text have not tried 2015-12-29 11:46 2020-07-24 12:02
Reporter: Florian Dold Platform:  
Assigned To: Stefan OS:  
Priority: normal OS Version:  
Status: assigned Product Version: 0.0  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9.1  
Summary: document fee structure considerations
Description: ... like the rationale we have behind each fee, and which values in the API include which fees.

We should also document the different kinds of incentives that the wallet has for selecting certain coins for payment.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015261)
Stefan   
2020-01-12 10:26   
(Last edited: 2020-05-15 16:44)
GEBÜHREN, die für alle möglichen Interaktionen der Nutzer erhoben werden könnten:
- 2.1 Gebühren beim ABHEBEN (Aufladen von Beträgen auf Coins in einer Wallet)
- 2.2 Gebühren beim AUSGEBEN / BEZAHLEN mit Taler
- 2.3 Gebühren auf REFUNDS (Rückerstattungen bei Rücktritt vom Vertrag)
 -2.4 Gebühren auf jeden REFRESH der Wallet
- 2.5 Closing fees: Gebühren für jedes BEENDEN DER RESERVE
- 2.6 Wire fees: Gebühren auf das CLEARING = aggregierte Buchung von Umsätzen eines Merchants durch den Exchange auf sein SEPA-Konto bei der Geschäftsbank in Fiat-Währung (€, USD, ...)

[Eine weitere Möglichkeit der Kostendeckung bzw. Einkommenserzielung für Exchange-Betreiber wäre, alle oben genannten Gebühren auf Null zu setzen und mit Einkünften aus dem Verfall von Guthaben die Betriebskosten zu decken bzw. zu überdecken. Einige Gutschein-Vertreiber nutzen diese Möglichkeit sogar schon als normales Geschäftsmodell. Für Taler wäre diese Lösung unter Umständen sogar ein "Best case", da ohne Verwirrung der Nutzer durch eine komplexe Gebührenstruktur und/oder Schwankungsbreite der Gebühren. Diese Einkünfte sind jedoch unstetig und unvorhersehbar, eignen sich von daher nicht für eine nachhaltige Finanzierung.]



View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5256 [Taler] deployment and operations text have not tried 2018-01-18 09:52 2020-07-24 12:02
Reporter: Marcello Stanisci Platform:  
Assigned To: Marcello Stanisci OS:  
Priority: low OS Version:  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: repository name should go into build description
Description: Any buildstep has a "step description", like for example "linting in progress",
or "compiling documentation", which appears on the Web interface.

Since a buildstep can work several codebases out, it would be handy to see the codebase name within the step description, like e.g. "linting the blog".
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015542)
buckE   
2020-04-07 06:25   
Which "web interface?" What "buildstep?"
(0016248)
buckE   
2020-06-10 07:30   
> Any buildstep has a "step description", like for example "linting in progress", or "compiling documentation", which appears on the Web interface.

Please point out an example on the web interface, so I understand what exactly the "step description" is. (ie - I will compare the strings you show me to master.cfg to be sure we are talking about the same fields.)
(0016316)
buckE   
2020-06-19 08:29   
Marcello

> Any buildstep has a "step description", like for example "linting in progress",
> or "compiling documentation", which appears on the Web interface.

Please send a link to where I can see this on the Web interface. I did not find these phrases in master.cfg.

> Since a buildstep can work several codebases out, it would be handy to see the codebase name within the step description, like e.g. "linting the blog".

Where would you want to see this? The same place as the link you will send? Please give a full example of what you want to see, where, and what happens now.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6434 [Taler] other tweak have not tried 2020-07-21 09:24 2020-07-24 12:02
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: confirmed Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8.1  
Summary: clean up error codes
Description: We should make a pass over the error codes list before we release 0.8.

There are many ECs that are not used, and others that have a name that's too generic, when it is actually an error that is very specific to some component (such as sync).
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5298 [Taler] wallet (WebExtensions) minor have not tried 2018-03-16 22:45 2020-07-24 11:57
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: wallet does not show a good error message on refund when a refund permission has expired
Description: Right now it just says "..." and the refund page fails to load, which is unacceptable!
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
4880 [Taler] wallet (WebExtensions) tweak N/A 2017-02-05 22:09 2020-07-24 11:57
Reporter: Christian Grothoff Platform: i7  
Assigned To: Florian Dold OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: add images to contracts
Description: It should be possible to specify image previews of the items sold and/or of the author/seller in the contract and display them. (Fully embedded and/or URIs but with hash of resource to authenticate it.)
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0011692)
Florian Dold   
2017-02-06 12:28   
There's a related web standard https://www.w3.org/TR/SRI/, if we use this we don't have to fetch the images and verify them manually in JavaScript. We can just translate it into the corresponding HTML element instead.

I agree that this is a good thing, at least for the merchant logo/banner, I've seen other payment systems do this.
(0011693)
Florian Dold   
2017-02-06 12:29   
Eh, currently the standard only supports the "script" tag, and leaves the "img" tag open for future versions :-(
(0015242)
Florian Dold   
2020-01-07 21:36   
Should this image be associated with the whole order? Or should it be possible to have one image per product in the product list? Maybe both?

Currently *all* of the contract terms used in our various demos have an empty product list and only a summary.
(0015474)
Christian Grothoff   
2020-03-29 22:02   
This is now implemented in the Android wallet. Hence changing category to WebExtension.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6430 [Taler] mechant backend major have not tried 2020-07-16 18:38 2020-07-24 11:56
Reporter: MS Platform:  
Assigned To: Christian Grothoff OS:  
Priority: high OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Responds with 404 for unclaimed orders.
Description: The merchant responds with 404 when a unclaimed (existing!) order is requested
via the GET /private/orders/$ID API. It needs to be fixed to return a object that informs
the requester about the status of the order (as the documentation also suggests).

Merchant test cases got also extended to reproduce this problem.
Tags:
Steps To Reproduce: Run: test_merchant_api
Additional Information:
Attached Files:
Notes
(0016471)
Christian Grothoff   
2020-07-16 19:34   
Just to clarify: this implies looking in the 'orders' table and not only in the 'contracts' table. I thought we implemented this, but maybe not. To be verified.
(0016472)
Christian Grothoff   
2020-07-16 19:35   
On my list, but Jonathan: if you get to it first, also great ;-).
(0016480)
jonathanbuchanan   
2020-07-16 22:23   
I'll work on this. Looks pretty easy to fix on the backend side, but the backend adds some fields to the contract_terms json, so the testing library will have to correctly add all those so we can generate the correct hash for when we look up the order.
(0016481)
Christian Grothoff   
2020-07-16 22:57   
Actually, in this case you MUST NOT generate a contract hash, as for unclaimed orders such a contract hash is meaningless. So the response in this case must be without a contract hash. (That's why we use the term 'order': an order cannot have a contract hash, but a contract has one!). A payment begins with an order, and once claimed we have the contract terms and the hash.
(0016484)
jonathanbuchanan   
2020-07-16 23:07   
Oops. I was looking at the wallet code instead of the merchant code.
(0016485)
jonathanbuchanan   
2020-07-16 23:09   
fixed in 90adbb3..6c04f55.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6410 [Taler] deployment and operations minor have not tried 2020-06-24 02:00 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Fix sphinx script re: no hardcoded paths /home/docbuilder
Description:  # The html-linked builder does not support caching, so we
 # remove all cached state first.
 html: diagrams
- # -W = exit 1 on warning; --keep-going = complete build anyway; write
log to ~/warnings.log
- $(SPHINXBUILD) -W --keep-going -w /home/docbuilder/sphinx-warnings.log
-b html-linked $(ALLSPHINXOPTS) $(BUILDDIR)/html
+# -W = exit 1 on warning; --keep-going = complete build anyway; write
log to ~/warnings.log
+ $(SPHINXBUILD) -W --keep-going -w warnings.log -b html-linked
$(ALLSPHINXOPTS) $(BUILDDIR)/html
     @echo
     @echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6403 [Taler] deployment and operations minor have not tried 2020-06-22 07:46 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: please install twisted under default python3
Description: checker-builder fails. A quick look at the UI shows the error is:

Upon execvpe b'bootstrap' [b'bootstrap'] in environment id 139815360567088
:Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/twisted/internet/process.py", line 406, in _fork
  File "/usr/local/lib/python3.7/dist-packages/twisted/internet/process.py", line 484, in _execChild
  File "/usr/lib/python3.7/os.py", line 581, in execvpe
  File "/usr/lib/python3.7/os.py", line 614, in _execvpe
  File "/usr/lib/python3.7/os.py", line 605, in _execvpe
FileNotFoundError: [Errno 2] No such file or directory: b'/bin/bootstrap'

twisted is not in /usr/local/lib/python3.7/dist-packages. Maybe `apt install python-twisted`?

I also think it would help to symlink python3/ to python3.7/ (and continue to update as we upgrade python3 versions)? We have (ex: in /lib) python3/ and python3.7/. Weird.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016334)
Christian Grothoff   
2020-06-22 08:51   
/usr/local/lib/python3.7/ doesn't even exist anymore. Restarting the buildslave to use the buildbot installation in /usr/bin (instead of /usr/local/bin/) should help.
(0016335)
buckE   
2020-06-23 06:56   
Can you uninstall the version of buildbot-worker in /usr/local/bin? Also /usr/local/bin/pip?

If they are not on the system, any scripts that still use the old versions will fail and we can fix the problems. Right now there is a version (of buildbot-worker) in `/.local/bin`, `/usr/local/bin`, and `/usr/bin'.
(0016336)
buckE   
2020-06-23 07:02   
Also please install pip globally. It's a long story but it might be needed right now to remove local packages installed with pip. Maybe not but it will really help to have it available just in case.
(0016337)
Christian Grothoff   
2020-06-23 08:46   
I've purged /usr/local/bin/* now.

python-pip3 was already globally installed.
(0016338)
buckE   
2020-06-23 09:35   
Great.

Please install python-pip (not pip3). I'm not sure but I think the version of buildbot-worker I am trying to remove might be from python2.7. (on the system the default `python --version` shows python 2.7.18 so this is on the system)
(0016339)
Christian Grothoff   
2020-06-23 09:37   
root@gv:~# dpkg --list | grep python3-pip
ii python3-pip 20.1.1-2 all Python package installer
(0016340)
Christian Grothoff   
2020-06-23 09:39   
We don't have python2 support on the system anymore, installing "python-pip" (for python2) would be a big problem (downgrades, unmet dependencies, etc.).

My best suggestion: manual (rm -r) cleaning...
(0016341)
buckE   
2020-06-23 09:46   
> We don't have python2 support on the system anymore

Um...

```
docbuilder@gv:~/worker$ which python
/usr/bin/python
docbuilder@gv:~/worker$ /usr/bin/python --version
Python 2.7.18
```

> My best suggestion: manual (rm -r) cleaning...

That's an option if necessary but this way seemed cleaner.
(0016342)
Christian Grothoff   
2020-06-23 09:58   
The python2 interpreter is there, but the required dependencies for python-pip are still not apt-able.
(0016343)
buckE   
2020-06-23 10:02   
Okay no problem. Thank you.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6392 [Taler] exchange tweak N/A 2020-06-17 03:55 2020-07-24 11:56
Reporter: jonathanbuchanan Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: low OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Remove TALER_TESTING_cmd_admin_add_incoming_with_instance()
Description: Remove the testing function TALER_TESTING_cmd_admin_add_incoming_with_instance() and its associated logic in admin_add_incoming_run() from testing_api_cmd_bank_add_incoming.c.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016373)
Christian Grothoff   
2020-06-27 18:56   
TALER_EXCHANGE_refund2 should also be removed at that time.
(0016504)
Christian Grothoff   
2020-07-22 22:04   
Looks like this can die now...
(0016510)
jonathanbuchanan   
2020-07-23 22:25   
Removed in 0e808b64..c24a18e1.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6264 [Taler] Web site(s) feature N/A 2020-05-27 09:03 2020-07-24 11:56
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: low OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: setup buildbot job to check website for dead links
Description: I don't know which Free Software would work best here, ideally something available in Debian. Please identify a suitable package, let someone with root know to install it on taler.net, and deploy it via buildbot. If there are broken links, it should send e-mail notifications - we can define a functional alias pointing to Stefan.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0015973)
buckE   
2020-05-27 09:13   
This is very clear, thank you for the definition.
(0016249)
buckE   
2020-06-10 08:01   
Christian: what MTA should I use in the script? "mail" etc. is not found on the system, and I know there are some planned changes. Please suggest a method of sending mail from the correct account (which I assume is whichever user account manages the 'sites-worker' buildbot worker and runs ./build-sites.sh - I'm still not sure which account that is yet but if you know, please mention that here).
(0016252)
Christian Grothoff   
2020-06-10 11:36   
Ah, I think _buildbot_ has a way to send messages to the last committer. At least we sometimes from some job get e-mails. So buildbot has that capability. Likely the shell script wrapper only has to return a failure code *and* the buildbot job must be configured accordingly.

Buildbot can also figure out for you who are the committers since the built broke, so you can notify the right people. So the e-mail should be done by buildbot, the body should be the output of the sphinx command, and the main modifications are likely to (a) make the shell script return an error code to indicate failure, and (b) tell buildbot to inform the developers. --- and it is possible that (b) is already the case, and only (a) needs to be done.
(0016254)
buckE   
2020-06-11 06:16   
Your reply makes sense for https://bugs.gnunet.org/view.php?id=6223 but this issue is different because:

1 - sphinx is not involved (so the body is actually going to be a wget log - clumsy but I like the simplicity of using wget instead of a less common program)

2 - we aren't e-mailing the most recent committer, but Stefan

So I *think* it will still be necessary to have an MTA on the server to send this e-mail. At least that seems the most elegant. And since this is already something you are considering, would it make sense to wait for that or investigate some other method like a hack to buildbot?
(0016257)
Christian Grothoff   
2020-06-11 09:53   
We could e-mail the most recent committer, if buildbot truly cannot e-mail Stefan. We could also just post to IRC, that would also suffice.
Oh, and yes, replace sphinx by wget here, I confused the two bugs.
(0016265)
buckE   
2020-06-14 07:59   
OK I'll do IRC first, then look into how specific the e-mail feature can get. Hopefully it is as simple as writing a new buildbot service specific to Stefan, and scheduling that service.

Would you like this to be under a "linkchecker" user? If so please create that account and give me access.

I assume I should create a "bootstrap-linkchecker" file? like "deployment/bootstrap-docbuilder"?
(0016266)
Christian Grothoff   
2020-06-14 13:19   
linkchecker@taler.net is now setup. And yes, deployment scripts that will make it trivial in the future to migrate the setup to a new system are always appreciated.
(0016295)
buckE   
2020-06-18 07:28   
linkchecker BUILDBOT job running successfully.

e-mail notifications through Buildbot successful (tested sending to self)

Currently configured to send to self (for temporary testing only) and to brokenlinks@taler.net.

CHRISTIAN PLEASE DO: Please alias brokenlinks@taler.net to Stefan, etc.

CHRISTIAN QUESTION:

Currently, output of wget is written to /home/linkchecker/linkchecker.log. This contains a list of broken links. I like using wget for this, but buildbot e-mail does not seem to have the ability to send arbitrary attachments (only buildbot logs). So I think we have these options:

1 - Provide Stefan wtih shell access to linkchecker@taler.net (I know, we just discussed this right? Bad option.)

2 - We make linkchecker.log available via nginx, perhaps behind an http password?

3 - configure MTA on taler.net to use localhost `mail` command instead of buildbot reporter

4 - Something else I am not thinking of. Maybe do not use wget (but it's really so appropriate - and then what exactly would be better for solving this issue anyway)?
(0016301)
Christian Grothoff   
2020-06-18 14:59   
I would just "cat linkchecker.log" as one of the build steps. Then this _is_ what the buildbot logs, as the output of all the commands you run in buildbot becomes part of the log. So I'd simply "if failed links, cat linkchecker, exit with failure (=> buildbot creates e-mail with the output from 'cat')". And if there are no failed links, simply exit with success (=> no e-mail).
(0016313)
buckE   
2020-06-19 08:04   
> I would just "cat linkchecker.log" as one of the build steps.

Oh, clever. Done. Confirmed working (log comes as attachment in e-mail). Job runs at 0745 daily.

Please set up the email alias from `brokenlinks@taler.net` to stefan or anyone you want. I can also specify specific users if you want.



View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6237 [Taler] Web site(s) minor always 2020-05-14 14:36 2020-07-24 11:56
Reporter: LUG Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: PDF Onboarding documentation link broken
Description: The docs page[1] links under Internals -> Onboarding to a PDF version of the onboarding docs[2], which is not available (404 HTTP error).

I can recommend setting up a broken link checker for the Taler website to prevent this issue in the future.

[1]https://taler.net/en/docs.html
[2]https://docs.taler.net/pdf/developers-manual.pdf
Tags:
Steps To Reproduce: Visit https://docs.taler.net/pdf/developers-manual.pdf
Additional Information:
Attached Files:
Notes
(0015889)
nikita   
2020-05-14 17:57   
(Last edited: 2020-05-14 17:59)
I no longer work for Taler, I'm the wrong person to assign anything to. On the same thought, for gnunet.org, we could do this in www_shared, but "broken link checker" exists out there as many applications, so no need to write one.

(0015890)
LUG   
2020-05-14 18:13   
That must have been an auto assignment, apologies. I assigned to Christian now for triage.
(0015891)
Christian Grothoff   
2020-05-14 18:28   
I have fixed the auto-assignment now.
Stefan: please fix the link. Then re-assign the bug to bucke, asking him to interpret the report as: 'setup link checker CI job'.
(0015895)
Stefan   
2020-05-15 10:46   
Can anybody tell me the link to the requested PDF file? I have been searching for it quite a time, finally given up. Or should we renounce on the PDF? No matter to get rid of the link in case we don't need such a file, but a nicely layoutet document would be nice for readers who want to join our team.
(0015896)
Christian Grothoff   
2020-05-15 12:33   
I just noticed that _this_ one wasn't properly generated. I've tried to fix the generation logic, *IF* my fix worked, the file _should_ appear as
https://docs.taler.net/pdf/developers-manual.pdf (eventually).
(0015897)
Stefan   
2020-05-15 13:17   
I'm sorry to say that the fix did not work out. For better consistency, it is recommended to generate the PDF file as 'taler-developers-manual.pdf' in order to comply with the names of other files from our documentation website. So far, every PDF had a "taler-" string in its name, as far as I have seen. That said, the link would be going like https://docs.taler.net/pdf/taler-developers-manual.pdf

I will monitor this bug and update the link then.
Kind regards and thanks for your work in advance!
(0015898)
Stefan   
2020-05-15 15:09   
The link is working fine, now. Thanks for your work!
I will assign this bug to Buck now
(0015962)
buckE   
2020-05-26 08:20   
Stefan, what exactly is the bug? Anyway I don't have access to the website, and I am just learning about this "Onboarding PDF" now from reading this. Please be specific. What is wrong and exactly what do you want changed?
(0015964)
Stefan   
2020-05-26 08:29   
Hi Buck,
there is no issue anymore with this "bug" as the link to the said document works fine. I have tested it now once again.

Referring to Christian's advice on May 15th, I should ask you to "interpret the report as: 'setup link checker CI job'". Hopefully you know better than me what to do with this task. It's about setting up a program for automatically checking broken links on the Taler website, I suppose.

Kind regards!
Stefan
(0015971)
buckE   
2020-05-27 02:49   
Changing to "resolved" although I did nothing. Reason:

From Stefan:
> Hi Buck,
> there is no issue anymore with this "bug" as the link to the said document works fine. I have tested it now once again.

Misc:
> Referring to Christian's advice on May 15th, I should ask you to "interpret the report as: 'setup link checker CI job'". Hopefully you know better than me what to do with this task. It's about
> setting up a program for automatically checking broken links on the Taler website, I suppose.

No I have no idea what this means, either the comment from May 15th or the phrase "interpret the report as: 'setup link checker CI job'" but I assume that if there is a task that relates to these ideas I will see it eventually.

Kind regards!
Stefan


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6234 [Taler] deployment and operations minor have not tried 2020-05-07 05:32 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Inferred: back up weblate
Description: Use backup script to back up weblate and rsync off-site.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015865)
buckE   
2020-05-07 06:32   
Done. @weekly backups. Details in /home/weblate/NOTES


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6231 [Taler] deployment and operations minor have not tried 2020-05-04 09:05 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Inferred from Christian: weblate first projects
Description: 3) Import a first set of projects (also good for you to test/expand your
own docs from (2)):

In particular:
- https://git.taler.net/cashier-terminal-android.git/
  with the strings in app/src/main/res/values/strings.xml
  [note: I do NOT know myself where the translations would
   go here, but it's an Android App.]
- https://git.taler.net/merchant-terminal-android.git/
  with the strings in app/src/main/res/values/strings.xml
  [ditto]
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: Screenshot_2020-05-06 Weblate.png (12,141 bytes) 2020-05-06 07:04
https://bugs.gnunet.org/file_download.php?file_id=892&type=bug
Screenshot_2020-05-06 GNU Taler Merchant Terminal - Android.png (81,881 bytes) 2020-05-06 07:04
https://bugs.gnunet.org/file_download.php?file_id=893&type=bug
Notes
(0015851)
buckE   
2020-05-05 06:23   
* which branch? master?

* Does weblate's key have access to these repos?
(0015858)
buckE   
2020-05-06 07:04   
- added "master"
- Weblate has access to the repos
- Weblate seems to autodetect the translation files (see screenshot).

See second screenshot for outstanding issues. I think we'll just learn what these mean as we use Weblate.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6230 [Taler] deployment and operations minor have not tried 2020-05-04 09:04 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Inferred from Christian: buildbot with weblatetest
Description: > You access buildbot under buildbot.taler.net for the Web interface, and
> using
>
> $ git clone git+ssh://git@git.taler.net/deployment.git
>
> (and git commit/push) to configure (buildbot/master.cfg).
>
> You can use the 'checker-worker' to add additional tasks for experiments.
>
> Maybe better: why not use the weblatetest@taler.net user (which AFAIK is
> now useless, right?) to configure/setup your own buildslave and add that
> to the slave list?
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015859)
buckE   
2020-05-06 08:43   
> You access buildbot under buildbot.taler.net for the Web interface

Right but I'm asking what credentials to use?
(0015860)
Christian Grothoff   
2020-05-06 12:21   
Those credentials are also in the Git. But that Web UI is not really used for anything, all the real stuff is done via Git.
(0015876)
buckE   
2020-05-11 05:14   
Okay I'll look online for some buildbot howto instructions so I can start learning this.
(0015877)
buckE   
2020-05-11 08:27   
I'm estimating 32 hours of research/testing/virtual machine work before I'm ready to create a test project. But if there is someone willing to provide some doc on how we handle this, that goes down to about two or three hours. If I'm to understand deployment.git/README.md and deployment.git/buildbot/README, I add 24 hours to my estimate.
(0015881)
Christian Grothoff   
2020-05-11 09:48   
http://buildbot.net/ really is the documentation for buildbot. the buildbot/README merely says which Debian packages you need for the features of Buildbot we use. The top-level README is only giving the entire deployment.git structure, but you really only need the stuff in buildbot/master.cfg to get started. Basically, that file already contains 'workers' with 'passphrases', so setting up a new buildslave for testing is basically adding another entry to the c["workers"] array and running the buildbot tool to configure and launch the buildslave under the respective account.
(0015963)
buckE   
2020-05-26 08:22   
When I know what buildbot is and how we use it, I am sure that comment will make a lot of sense. :) I will begin my research tomorrow.
(0015972)
buckE   
2020-05-27 09:11   
Small note: this is starting to make sense.
(0015978)
buckE   
2020-05-29 06:43   
Maybe it's still making sense. Without info it is extremely slow going but I continue to read and grep and guess.
(0015982)
buckE   
2020-05-29 09:58   
> Basically, that file already contains 'workers' with 'passphrases'

Does it? I don't see those "workers" anywhere else.

> , so setting up a new buildslave for testing is basically adding another entry to the c["workers"] array

Well, I can copypasta with new words like "testing" but I don't know what exactly that would be doing.

> and running the buildbot tool

What buildbot tool? How do I run this?

> to configure and launch the buildslave under the respective account.

What account? I don't know what you mean. I think there are must be some secrets I am not aware of. I've read the Buildbot Manual and the Buildbot Tutorial, and these statements and master.cfg do not indicate steps I can take based on what I have learned from those docs.
(0015984)
Christian Grothoff   
2020-05-30 12:09   
To add a new buildslave, you:
1) add a line like:
      worker.Worker("lcov-worker", "lcov-pass"),
    to the c["workers"] array. This specifies the $NAME and $PASS for the worker. This is how the buildbot *client*
    will authenticate to the buildmaster. We don't really care about security here, as the worker simply 'volunteers' to
    run code on behalf of the master. These are NOT the username/password you would find in /etc/passwd.
2) You login (say via SSH) as some username (say weblatetest). Here, you use
    $ buildbot-worker create-worker DIRECTORY http://buildbot.taler.net:9989/ $NAME $PASS
   to create a buildbot worker directory using $NAME and $PASS to log into the buildmaster on port 9989
   (or localhost, whatever is applicable)
3) you use
   $ buildbot-worker start
   to launch the buildslave.
4) You configure which jobs the buildslave should run (and when) in the master.cfg.

Not sure how any of the above CANNOT be documented in the buildbot documentation, we got it from there.
(0016217)
buckE   
2020-06-01 09:17   
Aha! (2) actually helps illuminate things a lot. Thank you!
(0016218)
buckE   
2020-06-01 10:54   
Here is the result of "$ buildbot-worker create-worker DIRECTORY http://buildbot.taler.net:9989/ $NAME $PASS" when run as "weblatetest" user on taler.net:

weblatetest@gv:~$ buildbot-worker
Traceback (most recent call last):
  File "/usr/local/bin/buildbot-worker", line 6, in <module>
    from buildbot_worker.scripts.runner import run
ModuleNotFoundError: No module named 'buildbot_worker
(0016224)
buckE   
2020-06-02 11:08   
Progress:

buildbot-worker create-worker buildbot-weblatetest buildbot.taler.net:9989 buildbot-weblatetest <password>
mkdir /home/weblatetest/buildbot-weblatetest
mkdir /home/weblatetest/buildbot-weblatetest/info
Creating info/admin, you need to edit it appropriately.
Creating info/host, you need to edit it appropriately.
Not creating info/access_uri - add it if you wish
Please edit the files in /home/weblatetest/buildbot-weblatetest/info appropriately.
worker configured in /home/weblatetest/buildbot-weblatetest
(0016225)
buckE   
2020-06-03 08:47   
There is still something missing. Using for reference: https://docs.buildbot.net/latest/tutorial/firstrun.html#creating-a-worker

When I start the new worker (that I created above) with "buildbot-worker start buildbot-weblatetest", it fails. I am supposed to look at "twistd.log" but it is not in /var/log or in the /home/weblatetest directory.

If you see https://docs.buildbot.net/latest/tutorial/firstrun.html#creating-a-master, there are two locations being discussed but I do not necessarily know what these are. I learned above that one of them (where I create the worker) is the /home/weblatetest directory onn taler.net. The only other location I have access to is my own local terminal. But I am looking for what is referenced here:

 "Meanwhile, from the other terminal, in the master log (twisted.log in the master directory)"

I really just need a brief run-down on the setup. It would save so much time and I could focus on buildbot, instead of learning our setup.
(0016227)
Christian Grothoff   
2020-06-03 13:06   
Two issues:
1) buildmaster had trouble re-loading its configuration because of the python3.7/3.8 migration
2) your config had buildbot.taler.net, but the master.cfg only binds to loopback (127.0.0.1) for security reasons;

I've fixed those. However, https://bugzilla.redhat.com/show_bug.cgi?id=1557687 also affects Debian, so when restarting the buildmaster I had to disable the Web interface. This should ideally be fixed in Debian, but as a work-around we could pip install buildbot locally under buildbot-master@taler.net. I've added Buck's SSH key to that account so he can read the logs there and bring back the Web interface.
(0016228)
buckE   
2020-06-04 08:57   
2) excellent. I only used the hostname because of your instructions, and I was concerned about the difference between this and the current docs (https://docs.buildbot.net/latest/tutorial/firstrun.html#creating-a-worker). Probably you built on an older version before localhost was the correct protocol. One question answered.

Regarding buildbot-www, do I need the web interface? I was getting the impression it was not necessary from your earlier comments

The worker still does not load. I do not expect you to debug but if you understand the error please tell me. Also I've posted a follow-up to a similar existing buildbot bug report. Here is the twistd.log output:

2020-06-04 08:15:42+0200 [Broker,client] ReconnectingPBClientFactory.failedToGetPerspective
2020-06-04 08:15:42+0200 [Broker,client] While trying to connect:
        Traceback from remote host -- builtins.RuntimeError: rejecting duplicate worker
        
2020-06-04 08:15:42+0200 [-] Stopping factory <buildbot_worker.pb.BotFactory object at 0x7f6016beac40>
2020-06-04 08:15:42+0200 [-] Main loop terminated.
2020-06-04 08:15:42+0200 [-] Server Shut Down.
2020-06-04 08:23:34+0200 [-] Loading buildbot.tac...
2020-06-04 08:23:34+0200 [-] Loaded.
2020-06-04 08:23:34+0200 [-] twistd 18.9.0 (/usr/bin/python3 3.8.3) starting up.
2020-06-04 08:23:34+0200 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
2020-06-04 08:23:34+0200 [-] Starting Worker -- version: 2.7.0
2020-06-04 08:23:34+0200 [-] recording hostname in twistd.hostname
2020-06-04 08:23:34+0200 [-] Starting factory <buildbot_worker.pb.BotFactory object at 0x7f778f699ca0>
2020-06-04 08:23:34+0200 [Broker,client] ReconnectingPBClientFactory.failedToGetPerspective
2020-06-04 08:23:34+0200 [Broker,client] While trying to connect:
        Traceback from remote host -- builtins.RuntimeError: rejecting duplicate worker
        
2020-06-04 08:23:34+0200 [-] Stopping factory <buildbot_worker.pb.BotFactory object at 0x7f778f699ca0>
2020-06-04 08:23:34+0200 [-] Main loop terminated.
2020-06-04 08:23:34+0200 [-] Server Shut Down.
(0016229)
Christian Grothoff   
2020-06-04 09:58   
Eh, I had already started the slave under the weblatetest account. So you're getting this message because somehow you are trying to start/connect a *second* one (with the same credentials).

The Web interface might have made this easier for you to understand, that's why it is "needed". It's not strictly needed for us to use the buildbot (and certainly not to configure it), but it can be convenient to simply see which workers are running without having to read logs...
(0016232)
buckE   
2020-06-05 06:56   
> Eh, I had already started the slave under the weblatetest account.

How did you do that? Is it automatic or did you type the same command I typed?

Can you disable it? Should you disable it? I'm trying to learn how the pieces fit together. If it's normal for you to start it, great. Although I'm not sure how I would control it then when I want to make changes?
(0016233)
Christian Grothoff   
2020-06-05 17:29   
I don't know which command you used, I used:

weblatetest@gv:~$ cd buildbot-weblatetest/
weblatetest@gv:~/buildbot-weblatetest$ buildbot-worker start
Following twistd.log until startup finished..
The buildbot-worker appears to have (re)started correctly.

You can disable it with 'stop'. Integration with systemd was not done (yet), as this is just for testing.
Now, once it is running, you don't usually ever do anything with the buildslave (modulo integration with systemd to start it on system boot), all actions are controlled via the buildmaster (via deployment.git).
(0016236)
buckE   
2020-06-08 08:33   
Okay great. What I'm asking though is why you started it instead of me, since I was testing it. That means that there is some other process/step (whatever you were involved in) that I did not know about, and should know about, to proceed. Of course I can stop | start it but why did this happen in the first place, and will it happen again? And should it, or was it a mistake, etc.?
(0016237)
buckE   
2020-06-09 04:09   
(Last edited: 2020-06-09 07:37)
I've just run "pip install buildbot.www" and "buildbot.www" (not the full buildbot, just the -www and .www packages) on buildbot-master user account. I did not do "pip install buildbot" as you suggested because I think that will create two buildbot executables (the debian and the pip) which we probably want to avoid. Please correct me if I am wrong.

Lines 1019-1029 ("www" definintion) of master.cfg uncommented and re-committed to deployment.git.

I do not see the web interface at https://buildbot.taler.net/ yet, and netstat does not show use of the port in master.cfg. But I assume it will restart with a fresh buildbot cycle automatically? I still do not understand our config, including where and how often the automatic reads/runs are configured, but I think this is correct.

There is also the possibility that the webUI fails because we are using python2.7: https://bugs.gnunet.org/view.php?id=6369

I did decide to take the risk and run "buildbot restart" but the error was:

buildbot-master@gv:~$ buildbot restart
error reading '/home/buildbot-master/buildbot.tac': No such file or directory
invalid buildmaster directory '/home/buildbot-master'

Maybe I could symlink buildbot-master to master/ (or just specify "master" as the command line option) and re-run the restart but that is way too far to go without checking first that it is ok. But if you confirm OK I will restart buildbot.

(0016238)
buckE   
2020-06-09 05:52   
weblatetest job starts | stops fine. I'm not sure yet where to add scripts/instructions but we have cleared the recent problems.
(0016239)
buckE   
2020-06-09 05:57   
weblatetest job starts | stops fine. I'm not sure yet where to add scripts/instructions but we have cleared the recent problems (with the weblatetest build worker)
(0016240)
buckE   
2020-06-09 06:16   
(also did pip install buildbot-waterfall-view buildbot-console-view)
(0016241)
buckE   
2020-06-09 08:04   
Update: after hours of waiting for webUI to show up, I realized there must be a problem with the master.cfg reloading. So I tried it manually ("buildbot restart master") and saw that the webUI authentication was also commented out. Now the master reloads, and the webUI is working: https://buildbot.taler.net/#/
(0016242)
buckE   
2020-06-09 08:05   
The original task (and also the task of setting up the WebUI) has been completed.
(0016245)
Christian Grothoff   
2020-06-09 11:56   
(Last edited: 2020-06-09 11:57)
https://bugs.gnunet.org/view.php?id=6230#c16239 => it's just your playground worker, so I think that one we can leave for experiments without too much documentation or scripting.

For 'real' jobs, we should have the worker launched as a systemd user service and create additional, worker-specific accounts. That said, you surely could use that worker to test out new CI/CD jobs.

You already got your first ones: Web site link checking and generating alerts when there are warnings building the docs.git Sphinx documentation (which you may want to directly integrate with the existing doc-builder).



View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6229 [Taler] Web site(s) feature N/A 2020-05-03 14:25 2020-07-24 11:56
Reporter: Christian Grothoff Platform: i7  
Assigned To: buckE OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: enable machine translation
Description: I think it would be useful to enable MT in Weblate, as at least for me such MT translations usually give a good starting point from which one can refine grammar and choice of words:
https://docs.weblate.org/en/latest/admin/machine.html

At our (low) translation rates, I believe GoogleTranslate and MyMemory should both be effectively free (not sure about some of the others from the list). Google requires creating an account, MyMemory simply needs us to provide a contact address (in case of excessive use/API troubles). Plus it's Google. So for now, I'd suggest we enable the MyMemory service. I don't see a data protection issue here, as *these* strings are obviously fully public _and_ the requests are made not from the user's browser but by Weblate, so there is also no real tracking potential.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0015849)
buckE   
2020-05-05 04:43   
This is working. Within a component, navigate to the language you want to translate to, and click "Machine Translation" to be given options. using MyMemory.
(0015850)
buckE   
2020-05-05 04:44   
This is working with MyMemory. Navigate to the component, then the language. Then choose "Machine Translation"


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6228 [Taler] deployment and operations minor have not tried 2020-05-01 04:38 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Weblate: Add translation howto to docs.taler.net "Internationalizations"
Description: Over time while learning:

> 2) Document the process of adding a project/repository for i18n to
> Weblate in the Taler developer manual.
Tags:
Steps To Reproduce:
Additional Information: Process started. See "Internationalization"
Attached Files:
Notes
(0015820)
buckE   
2020-05-01 05:13   
https://docs.taler.net/developers-manual.html#internationalization

I will mark this closed because the alternative is to leave it open until the weblate project folds, we stop using it, or electricity stops happening.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6227 [Taler] deployment and operations minor N/A 2020-05-01 04:10 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Please evaluate weblate GPG signing
Description: Issue: in a message to me Christian said "I believe the automated push of Weblate will NOT work with our policy on GnuPG signed commits"

It looks like Weblate supports GPG-signed commits: https://docs.weblate.org/en/latest/admin/optionals.html?highlight=spam#signing-git-commits-with-gnupg

Does this comport with our policy of GPG-signed commits? :https://docs.taler.net/developers-manual.html#committing-code

It seems to, at least in theory, but there is also a degree to which GPG signatures represent a human "signing off" on a commit, and that would be lost with an automated commit.

Please assign to me if you want me to pursue.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015819)
buckE   
2020-05-01 04:26   
Function added. Maybe we just see how it works? Unless you don't want an automatic signature in principle.

https://weblate.taler.net/keys/
(0015821)
Christian Grothoff   
2020-05-01 08:00   
Oh, wow. Cool. While that's obviously not as, eh, meaningful as a human signature, it should do. After all, it's easy for a security audit to check that all commits made by the Weblate GnuPG key are on .po files. So I think that's totally acceptable for us!

So please do deploy!
(0015823)
buckE   
2020-05-01 08:37   
Deployed.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6226 [Taler] deployment and operations minor have not tried 2020-05-01 04:05 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Please decide on SPAM prevention for weblate
Description: At this time, anyone (world) can register an account at weblate.taler.net. This could be a problem. Options:

1 - Weblate plays well with akismet: https://docs.weblate.org/en/latest/admin/optionals.html?highlight=spam#spam-protection

2 - Manual-registration only

3 - World registration but Manual approval? (not sure if supported, but probably)

Note: default access level is low, and can not create new projects. That requires superuser intervention.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015822)
Christian Grothoff   
2020-05-01 08:02   
I think sending data to third parties like Akismet is not compatible with our privacy stance.

Let's see if we get substantial amounts of spam, and *then* would prefer option (3).
(0015824)
buckE   
2020-05-01 08:40   
I'm very glad to hear you say that. Resolving. If we have a problem in the future we will create a new bug ticket.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6223 [Taler] deployment and operations feature N/A 2020-04-26 01:17 2020-07-24 11:56
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: inform us about sphinx warnings
Description: There is an existing buildbot job that runs 'sphinx' on the documentation (docs.git) to generate the documentation page. The underlying CI job should be enhanced to inform the last committer if Sphinx generates warnings (i.e. about syntax errors or bad links).
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0015878)
buckE   
2020-05-11 09:05   
> There is an existing buildbot job that runs 'sphinx' on the documentation

Where *exactly* can I find this "buildbot job?"
(0015882)
Christian Grothoff   
2020-05-11 09:52   
deployment.git/buildbot/master.cfg, the "DOC_FACTORY" (and related: the doc-worker, DOC_BUILDER and doc-builder, and DOC_SCHEDULER).
(0016247)
buckE   
2020-06-10 07:24   
Ah, I'm starting to see this. Honestly it was not well written, and the above info was not the answer to the question I asked. But from what I've learned about buildbot and using the above as indirect hints, I think the task is actually something like this:

```
Buildbot calls deployment/buildbot/build-docs.sh on the docs.git repo's file to build (make) the documentation for sphinx. When it runs, it should e-mail the most recent committer to Git in the event of an error. Task: investigate sphinx and buildbot to determine appropriate BASH scripting to do this. Deliverable: check in enhanced build-docs.sh.
```

Does that sound accurate?

I could set up a local demo sphinx environment so I can learn how to grab any errors that sphinx generates (so I can send those in the e-mail). And I need to learn how buildbot stores the most recent commit info so I can reference that in the bash script (unless you know, and can share the info). Once I understand those things, the actual script should not be difficult but that learning curve is not trivial.

Alternative: what user is this job (and therefore build-docs.sh) run under? And can I have permission to that account? Because if I can, then perhaps I could manually run the build-docs.sh file as that user, inputting bad docs, and view the errors on the console. Does that make sense to you as a way to test *breaking* a sphinx build, so that I can work with the errors? (I'll probably also just do the development of the script in that user's home directory in that case. Much faster than building a test environment, if you think it safe enough.)
(0016251)
Christian Grothoff   
2020-06-10 11:28   
(Last edited: 2020-06-10 11:29)
Yes, what you write would be accurate, modulo that it is not just in the event of an error (hard failure), but also in the event of a warning.

The account in question is docbuilder@taler.net, you can now SSH into that (for testing, as you proposed).

(0016273)
buckE   
2020-06-16 06:27   
Okay I have this on my to-do list after the test project and linkchecker, which I am working on concurrently.
(0016296)
buckE   
2020-06-18 07:30   
Starting on this next, as linkchecker is in beta release.
(0016297)
buckE   
2020-06-18 08:13   
STATUS:

 - reporter created in master.cfg; e-mail verified working (sending to me)
- Makefile options convert warnings into errors but also sphinx will `--keep-going` with build on warnings. (ie - Buildbot shows failure on sphinx warnings, but sphinx continues, and writes to /home/docbuilder/warnings.log)

QUESTION #1:

Same as with the linkchecker, how do you want to provide the "warnings.log" file to interested parties?

(Maybe there is a way for buildbot to send arbitrary logs, but I don't think so. I think only bb logs. Text reads:

addLogs
(boolean). If True, include all build logs as attachments to the messages. These can be quite large. This can also be set to a list of log names, to send a subset of the logs. Defaults to False.

)

QUESTION:

Current reporter sends to me (for testing) and also " sendToInterestedUsers=True." The manual says "Interested Users are authors of changes and users from the owners build property." Is this acceptable? When I check in change now, all "InterestedUsers" should get the e-mail. Let's see who complains.
(0016300)
Christian Grothoff   
2020-06-18 14:36   
Q #1: I would usually want to see them in the build logs of the build job on the buildbot web site *AND* in the body of the notification e-mail.
Q #2: You're unlikely to immediately get complaints, but it would be good if we could limit notifications to the last committer.
(0016314)
buckE   
2020-06-19 08:24   
> Q #1: I would usually want to see them in the build logs of the build job on the buildbot web site *AND* in the body of the notification e-mail.

Done, using your suggestion from the linkchecker. Simple, elegant. Log arrives as attachment to e-mail and is also available in BB UI.

> Q #2: You're unlikely to immediately get complaints, but it would be good if we could limit notifications to the last committer.

At this moment the function sends to recent committers and also "owners" but I'm not sure where "owners" is defined. We can overwrite the function, but maybe there are other uses of this function that require the owners to be included. It's a little foolish that there is no "sendToLastCommitter" but there isn't. Anyway by including "owners" we will see who is an owner, and if they should be, and maybe learn what "owners" means. Maybe you will remember where it is defined? (I didn't search hard but it's not in master.cfg.)

Below is the overwrite reference, so I can start from there if you choose to re-open.

https://docs.buildbot.net/latest/manual/configuration/reporters.html?highlight=addlogs#mailnotifier-arguments
sendToInterestedUsers
(boolean). If True (the default), send mail to all of the Interested Users. Interested Users are authors of changes and users from the owners build property. Override MailNotifier getResponsibleUsersForBuild method to change that. If False, only send mail to the extraRecipients list.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6208 [Taler] deployment and operations minor have not tried 2020-04-23 08:42 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: reopened  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: weblate: complete production deployment sanity check
Description: Complete steps here: https://weblate.taler.net/manage/performance/

1 - Log in as admin

2 - Go to URL

3 - Complete "Configuration errors"

4 - Complete "System checks"

5 - Verify https://docs.djangoproject.com/en/3.0/howto/deployment/checklist/
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015775)
buckE   
2020-04-24 08:21   
Note to self

Current outstanding issues reported as bugs to weblate:
https://github.com/WeblateOrg/weblate/issues/3784
https://github.com/WeblateOrg/weblate/issues/3783
https://github.com/WeblateOrg/weblate/issues/3782
(0015791)
buckE   
2020-04-28 09:21   
All outstanding errors/issues are due to fixes coming in 4.0.2, or to our unusual config of running the development server via nginx proxy. Fixing these issues will require proper config as in: https://docs.weblate.org/en/latest/admin/install.html#running-server

Closing issue.
(0015825)
buckE   
2020-05-01 08:58   
Reopening re: production deployment means we are able to fix (and probably should fix) these errors: https://weblate.taler.net/manage/performance/
(0015846)
buckE   
2020-05-04 08:52   
* Errors regarding SVN and HG ignored

* Errors regarding HTTPS ignored re: nginx reverse proxy

Christian:
Integrated error reporting options are Sentry and Rollbar, which rely on Google.
This is the section on error reporting: https://docs.weblate.org/en/weblate-4.0.3/admin/install.html#collecting-error-reports

Backups can be handled a few ways. I recommend a monthly cron job that backs up the `weblate` psql database, `/home/weblate/weblate-env` directory, and rsyncs them off site somewhere. Please create a ticket for me if you want me to do something, or you can just use this as a base for your script:

weblateBackup.sh
```
#!/bin/bash -e

# Set today's date as variable
today=$(date +"%Y-%m-%d")

# Dump the weblate database
pg_dump weblate > weblateDB"{$today}".bak
#su - weblate -c pg_dump weblate > weblateDB"{$today}".bak

# Create the archive
tar -czf weblateBackup-$today.tgz weblateDB"{$today}".bak /home/weblate/weblate-env/

# Move to safe location:

# Clean up
rm weblateDB"{$today}".bak
```



View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6207 [Taler] deployment and operations minor have not tried 2020-04-23 07:59 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Please install docutils if weblate back-end docs are required
Description: Weblate's Django back-end has a "Documentation" link. It might have useful information. It seems to depend on `docutils` package running on server.

Also maybe (?) this is required for weblate's main translation features.
Tags:
Steps To Reproduce: 1 - Log in as admin
2 - Visit https://weblate.taler.net/admin/doc/
Additional Information: https://docutils.sourceforge.io/
https://packages.debian.org/search?suite=stretch&searchon=names&keywords=docutils
Attached Files:
Notes
(0015749)
Christian Grothoff   
2020-04-23 11:17   
python3-docutils is/was already installed on the server. If you need to start a service, I'd say just do it as weblate. If you need an nginx service entry and/or a DNS name, let me know.
(0015784)
buckE   
2020-04-27 06:05   
Solution was to `pip install docutils` into the Weblate virtualenv.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6206 [Taler] deployment and operations minor have not tried 2020-04-23 07:42 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: reopened  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: weblate admin interface (i) links lead to dead pages
Description: On weblate 4.0.1, (i) information icons lead to pages that do not exist
Tags:
Steps To Reproduce: Install weblate 4.0.1
Visit yoursite.com//accounts/profile/
Click on (i) link to the right of Languages
land on https://docs.weblate.org/en/weblate-4.0.1/user/profile.html#languages
Additional Information: Reported to weblate team: https://github.com/WeblateOrg/weblate/issues/3769

Note that bad links also come from running `weblate check --deploy` suggesting this is not an installation/config issue but a bug. Probably.
Attached Files:
Notes
(0015741)
buckE   
2020-04-23 07:43   
Resolving as is (probably) not related to local install and is reported to weblate team: https://github.com/WeblateOrg/weblate/issues/3769

Expectation is this will be resolved in weblate 4.0.2?
(0015742)
buckE   
2020-04-23 08:51   
https://github.com/WeblateOrg/weblate/issues/3769
(0015743)
buckE   
2020-04-23 08:51   
Resolved upstream: https://github.com/WeblateOrg/weblate/issues/3769


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6205 [Taler] deployment and operations minor have not tried 2020-04-23 06:37 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: weblate: test import of some project
Description: Demonstrate/document how to import a translation project into weblate.

Deliverables:

1 - example on weblate.taler.net

2 - how to add projects document (target audience: world/public)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015811)
buckE   
2020-04-30 08:40   
1 - `weblatetest` project added successfully

2 - docs already exist: https://docs.weblate.org/en/latest/ (haha good luck)

Notes:

a. superuser status required to create a new project (https://weblate.taler.net/admin/weblate_auth/user/)

b. Django user management (https://weblate.taler.net/admin/weblate_auth/user/) will be deprecated in 4.0.2

c. weblate public key (https://weblate.taler.net/keys/) must be added to any repo that weblate will write translations to


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6204 [Taler] deployment and operations minor have not tried 2020-04-23 06:29 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: weblate: registration demo and howto document
Description: Depends: https://bugs.gnunet.org/view.php?id=6201

Deliverables:

1 - Register a new non-priv user on weblate

2 - Document process for global collaborators

3 - Brief report how to prevent SPAM signups
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015793)
buckE   
2020-04-28 11:25   
Resolution:

1 - Registration successful for new non-priv user using web interface.

2 - "Howto" not required. Just go to weblate.taler.net and register

3 - SPAM: https://docs.weblate.org/en/weblate-3.10.3/admin/optionals.html#spam-protection

Question: should we enable akismet and spam protection? Please consider in context of https://bugs.gnunet.org/view.php?id=6224

Notes:
existing users will not get registration e-mail, even if deleted and signed up again.
manual registration through Django admin interface will not documented, as it is to be deprecated in 4.0.2 in favor of new weblate admin UI


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6203 [Taler] deployment and operations minor have not tried 2020-04-23 06:23 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: weblate: systemd service run by weblate user
Description: Christian suggested running the systemd file by a non-priv user. (Not running *as* weblate user but running *by* weblate user so Christian would not have to start weblate.service.)

CHRISTIAN: please provide a link to what you mean. I did not know this is possible, but assumed root had to add to /etc/systemd/system regardless of user/owner?

Also you said you would have to add weblate user to the right group. Which group, just because I am curious? Please let me know when you did that and reassign to me and I'll investigate.

(Alternatively, I could run this as a cron job or even a screen session permanently under weblate?)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015748)
Christian Grothoff   
2020-04-23 11:14   
https://wiki.archlinux.org/index.php/Systemd/User

I've activated lingering (# loginctl enable-linger weblate) for the weblate user.
(0015771)
buckE   
2020-04-24 06:58   
Great! We are successful for weblate.service and celery-weblate services running locally. weblate user is logged out but the site is still up at https://weblate.taler.net/ Closing issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6202 [Taler] deployment and operations minor have not tried 2020-04-23 06:16 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: weblate: update admin e-mail
Description: update admin e-mail address to weblate@taler.net

How to reproduce this?
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015739)
buckE   
2020-04-23 06:17   
users page is here: https://weblate.taler.net/admin/weblate_auth/user/

Verify here: https://weblate.taler.net/accounts/profile/#account

Also here (we assume) when e-mail is set up: https://weblate.taler.net/accounts/email/


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6201 [Taler] deployment and operations minor have not tried 2020-04-23 06:09 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Please provide e-mail settings for Weblate setup
Description: CHRISTIAN: per our chat please provide the values for these settings in an e-mail to me and I will update weblate:

EMAIL_HOST (mail.taler.net? taler.net?)

EMAIL_HOST_PASSWORD

EMAIL_HOST_USER (weblate@taler.net?)

EMAIL_PORT
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015747)
Christian Grothoff   
2020-04-23 11:11   
(Last edited: 2020-04-23 11:11)
host: localhost
password: none
user: none (IF you need to configure a sender, use weblate@taler.net).
port: 25

(0015773)
buckE   
2020-04-24 08:04   
Hm. This seems incorrect? Re: weblate user can not send e-mail using `mail user@hostname` on the command line. And I imagine that is what would be used by python/django running as the weblate user?

So instead, I think we need to authenticate weblate user through the mail system?

For an example, here is a similar configuration using GMAIL settings: https://stackoverflow.com/questions/31324005/django-1-8-sending-mail-using-gmail-smtp

But maybe I am wrong, in which case please explain how mail can be sent from the command line as weblate user, so I can debug where the problem is.
(0015779)
Christian Grothoff   
2020-04-24 12:49   
Hmm. I checked, and I _think_ a configuration option for localhost relaying was missing. Now that I've set this, I can do:

# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 gv.taler.net ESMTP Postfix
HELO localhost
250 gv.taler.net
MAIL From: grothoff@taler.net
250 2.1.0 Ok
RCPT To: grothoff@gnunet.org
250 2.1.5 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
hello
.
250 2.0.0 Ok: queued as 4C50131C0276
quit
221 2.0.0 Bye
Connection closed by foreign host.

So it should work now.
(0015785)
buckE   
2020-04-27 06:32   
Interesting. I'm not sure that weblate's "localhost" function will work that way but maybe. I assumed it would look for a `mail` function locally, that seems common.

Anyway sending an e-mail to myself using that method from weblate user failed. No errors - just didn't receive the e-mail. Message was "250 2.0.0 Ok: queued as 6F52531C0016" Time: 4:24UTC in case you want to check logs.
(0015786)
buckE   
2020-04-27 07:15   
Cancel that request. The e-mail came though eventually.
(0015788)
Christian Grothoff   
2020-04-27 11:32   
buckE: please set bugs to resolved if you think they are resolved. (One can always re-open if truly needed).
(0015792)
buckE   
2020-04-28 11:21   
Confirmed working 28 April. Registration e-mail, and feedback e-mail sent.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6177 [Taler] wallet (Android App) minor have not tried 2020-04-13 19:21 2020-07-24 11:56
Reporter: grote Platform:  
Assigned To: grote OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Render Markdown ToS
Description: The ToS, once available in Markdown format should be rendered nicely and have collapsible sections.

https://github.com/noties/Markwon could be used.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015959)
grote   
2020-05-25 15:32   
The Android wallet app now renders the ToS in markdown using the Markwon library and offers expandable sections.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6160 [Taler] deployment and operations feature N/A 2020-04-06 13:13 2020-07-24 11:56
Reporter: Christian Grothoff Platform: i7  
Assigned To: buckE OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: deploy weblate for translation of Web site(s) and App(s)
Description: Weblate is a page to allow translators to contribute translations. We should setup weblate.taler.net for this (native deployment, not Docker!)

First step(s) someone with root would have to do include:
1) Installing Debian packages that are dependencies, as per Weblate setup instructions
2) Create a user account ('weblate') to run the service
3) Create 'weblate' database
4) Create DNS entries

Then
5) Setup pip/weblate itself, providing port for nginx to reverse-proxy to

Finally again root:
6) adapt nginx configuration

and last
7) test, integrate, import existing po files, document, etc.

Florian or I could do 1-4 rather quickly, just let us know a bit before you're ready to start with (5).
[This is filed now, in anticipation of existing tasks being completed.]
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files: weblateWorking.png (9,843 bytes) 2020-04-20 10:48
https://bugs.gnunet.org/file_download.php?file_id=886&type=bug
Notes
(0015538)
buckE   
2020-04-07 03:37   
This sounds simple enough. I would like to better understand the server architecture. For example "weblate.taler.net" will point to what server?

I think now might be a good time to create (or inform me about, if it already exists) a three-tiered system. The development system is on the developers personal computer, or possibly a VM (preferably one that lives on a metal server that we own and can create VMs on), the stage system is one I, or whomever you wish, could have sudo/root access to and would do things like this task on. Then the third layer would be live and you or Florian would test my deployment on stage, read the docs I wrote on how I built the server, and then deploy on some live server.

The stage server should always be an exact match for live.

I don't want to make this simple task more complicated than it needs to be, but having this sort of infrastructure in place will make it easier to "systemitize" deployments of all kinds in the future. Anwyay, something to think about.

But the shorter version of everything I just said is: Is there a testing server we could do this on first, using "test.weblate.taler.net" as the target URL?
(0015547)
Christian Grothoff   
2020-04-07 10:21   
weblate.taler.net will be a CNAME to gv.taler.net, like most subdoamins of taler.net. gv.taler.net has a virtual hosting setup with a reverse proxy pointing to the (loopback) port with the weblate service. Given that today we only have _one_ (physical) server for taler.net, there is no proper 'stage' in your sense. stage.taler.net (for the Web-site staging) is merely the exact same setup targeting a different Git branch.
I agree that systematic staging on an 'equivalent/equal' server setup might be nice, not sure when we can get there. For the Weblate, I really don't see much of a risk though: running another httpd on a loopback port under some $USER is hardly going to endanger the rest of the system ;-).
(0015569)
buckE   
2020-04-08 08:14   
> I agree that systematic staging on an 'equivalent/equal' server setup might be nice, not sure when we can get there

I know to an extent I am just experiencing the difference between what I am used to and a new system, and soon I will be used to taler's procedures. But it's difficult for me to feel comfortable installing major subsystems, that require trial-and-error, on an existing live server. Maybe a cheap VPS? Since it would only have public/generic info, there is no security risk. We could even make access to it Tor-only so it is not available on the public internet. I am just brainstorming, and it is not directly related to this specific task. Anyway I can develop in a local VM first, so that is something.

Specific to this task: exactly what is the the goal? You said "Weblate is a page to allow translators to contribute translations. We should setup weblate.taler.net for this...." Okay, but what exactly will be translated? What do you expect to *see* when this task is complete?
(0015581)
Christian Grothoff   
2020-04-08 13:58   
You're overcomplicating things. There also is no risk or need for a VPS. You'll get an SSH account (and local Postgres access) and can configure Weblate to bind against some (free) TCP port on loopback. You write a systemd service file with how to launch the service under that $USER. That's it.

Result: Weblate instance running on weblate.taler.net, where we then can import po files, add users, manage translations, etc. A few of us setup as 'administrators' on the sub-domain. This should be fast and easy, maybe a 2h task.
(0015592)
Christian Grothoff   
2020-04-08 19:25   
DNS/TLS and SSH access to weblate@taler.net for buck have been setup. However, I still need to run the 'apt install' steps from the Weblate instructions before Buck can take over here.
(0015677)
buckE   
2020-04-17 11:08   
> You're overcomplicating things.

Probably. That is common in the absence of details, or the institutional knowledge of how you like to handle these things.

> You'll get an SSH account (and local Postgres access) / SSH access to weblate@taler.net for buck have been setup

Aha! That's good useful information that was missing from the spec. Now I see why you are not worried about the risk to the broader system. Hopefully you understand that without knowing about this sandbox (weblate user), I was right to worry. But these are the parameters to proceed under. Great! I love definition. Please provide the Postgres details when you have them. (https://docs.weblate.org/en/latest/admin/install.html#database-setup)

> and can configure Weblate to bind against some (free) TCP port on loopback

Aha! That is more specific-to-this-system, very useful, but very unguessable information that was not in the spec. Should be no problem!

> You write a systemd service file with how to launch the service under that $USER. That's it.

Aha! Do I need to say the next part? :)

> There also is no risk or need for a VPS.

Well there is always a need to be able to test a system before running live, and I can't do that if I don't have control over the web server, DB server, firewall, etc (unless it all installs with no debugging necessary). So I'll build this out in a VM first. No problem!

> maybe a 2h task.

If we had a test server, less. But since I will be creating a test VM from scratch, probably slightly more.

Questions:

- How do you feel about using virtualenv as recommended in the weblate docs?

- Do you want me to run celery? https://docs.weblate.org/en/latest/admin/install.html#celery

- Do you expect to need any of the optional dependencies? https://docs.weblate.org/en/latest/admin/install.html#optional-deps

- Are there any other assumptions you are making that are not explicitly stated? Any standard procedures in how you handle this server that are not universals? (In this case, I doubt it as everything stated so far paints the picture of how the system will run, which was a blank slate previously.)

- What are you expecting to *see* and *do* with weblate when it's installed? If it's sandboxed under the weblate user, it won't talk to other applications on the server, right?
(0015678)
Christian Grothoff   
2020-04-17 11:16   
I've now setup a database 'weblate' which the 'weblate' user owns. So you can create/drop/insert/delete/update/etc. on that database as needed. No credentials, use the UNIX domain socket to connect to Postgres and your UID is used to check permissions. For example:
$ psql weblate
psql>
(0015679)
Christian Grothoff   
2020-04-17 11:24   
As for 'testing', I see no reason why you can't test it already as prescribed: you bind against a free port on loopback. That's only accessible to those few with an account ON the system. You can use SSH port forwarding to forward the loopback port to your client system and run a browser against it. So bound-to-loopback *is* your firewall. As the weblate database is used for nothing else, feel free to use it for experiments.
virtualenv: sure, why not?
Celery: it reads like it is needed. We could use a systemd user service for this.
Optional dependencies: none are needed.
Other assumptions: hah, no idea ;-).
See/do: Taler core team (incl. you, but not only you) will import our .po and .xml files from the various Taler components (software, website), and then allow volunteers from the community (or professional translators, if needed and affordable) to sign up and contribute translations. Taler core team (incl. maybe you) will then export the translations and use them when packaging releases of the software or deploy to the Web site. See, for example, https://taler.net/de/.
(0015686)
buckE   
2020-04-18 05:36   
Weblate is tested working on a VM. Install took about 30 minutes once the VM was ready. Ten more for systemd.

REQUESTS:

1.

> However, I still need to run the 'apt install' steps from the Weblate instructions before Buck can take over here.

Please do this when it is convenient, then please post back when you're done and I'll install weblate in a screen session to test.

2. (for later)

Then there will be an intermediary step when I ask you to import two systemd files (celery-weblate and weblate.) Celery intructions for root are here:

https://docs.weblate.org/en/latest/admin/install.html#background-tasks-using-celery

Sections:
Running Celery as system service
Environment configuration to be placed as `/etc/default/celery-weblate`:

systemd file for weblate:

--/etc/systemd/system/weblate.service--
[Unit]
Description=Weblate Server
After=multi-user.target celery-weblate.service

[Service]
Type=simple
Restart=always
RestartSec=1
User=user
ExecStart=/home/weblate/weblate-env/bin/python3 /home/weblate/weblate-env/bin/weblate runserver

[Install]
WantedBy=multi-user.target
----

BUT please do not enable the systemd services before I install celery and weblate, which I will do when you take care of the pre-reqs (1).


3. Please verify that I can run this on port 8000. As `weblate` user I can not (be sure if I) see all used ports. (Note It is non-trivial to use a port other than 8000.)


MISC:

> As for 'testing', I see no reason why you can't test it already as prescribed: you bind against a free port on loopback.

Well that's fine, right? Different sysadmins do things differently, may or may not see the other's reasons, and anyway what is "prescribed" to me is of course limited to *what* the task is, not *how* I prefer to accomplish it. It so happens that I will not deploy something on a new, live server until I have tested the something first. You know this server intimately, and are willing to take more chances. I am being cautious, and it's my call to make when I am responsible for the task. But I will try to answer:

Binding against a free port on loopback is (i) rather difficult when one is not root since only root knows what ports are free (see above) (ii) actually *doing* the thing I want to test before doing it, so it's beside the point (which was running a VM to test before going to the live server.)

> You can use SSH port forwarding to forward the loopback port to your client system and run a browser against it.

Sure I could but doing so would not be "testing" since production will not run this way. It's a good debug step if testing fails, but so far that didn't happen. (But FYI I am adept with SSH tunnels for the many times they will be useful on this or any other projects.)

> See/do:

Okay from your answer it sounds like this will be a follow-up task so I will wait for that.

Next:

Weblate UI has a password. How do you want me to get this password to you?
(0015690)
Christian Grothoff   
2020-04-19 01:02   
Debian packages for weblate are installing as I write this.

Port 8000 is free, I'm not sure why it is non-trivial to use another port, I'd have thought this to be a simple configuration option in any sane software package. But, whatever. 8000 is free.
Also, any (non-root) user can see which ports are in use: the command is "netstat -ntl". (-n no DNS, 't' for TCP, '-l' for listen).

I was proposing SSH only for your testing. Anyway, I've now configured nginx, so weblate.taler.net points to port 8000 now. Note that we may need to revise the Content Security Policy before it works properly with this setup (so for testing, you may still want to use SSH).
(0015693)
buckE   
2020-04-20 03:47   
> Debian packages for weblate are installing as I write this.

Great! But it looks like I need root's help with psql. `weblate migrate` fails with "django.db.utils.ProgrammingError: permission denied to create extension "pg_trgm"
"

So can you please install pw_trgm?

When I tried manually:

```
weblate@gv:~$ psql -U weblate
psql (12.2 (Debian 12.2-4), server 11.6 (Debian 11.6-2~sid1))
Type "help" for help.

weblate=> \c weblate
psql (12.2 (Debian 12.2-4), server 11.6 (Debian 11.6-2~sid1))
You are now connected to database "weblate" as user "weblate".
weblate=> CREATE EXTENSION pg_trgm;
ERROR: permission denied to create extension "pg_trgm"
HINT: Must be superuser to create this extension.
weblate=>
```

> I'd have thought this to be a simple configuration option in any sane software package.

Me too but it does not appear so unless I am missing something, which is always possible.

> Also, any (non-root) user can see which ports are in use: the command is "netstat -ntl". (-n no DNS, 't' for TCP, '-l' for listen).

Well sure, you as root know that a non-root user can see all the used ports because you can compare them. But a non-root user does not know what he does not know. And it is directly applicable. For example, running the command you suggested shows:

`Active Internet connections (only servers)`

So I must ask myself: what about things that aren't servers, like (in theory) a root-owned node app on port 8000?

But running the command I use (netstat -utlpn) is more explicit:

```
(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
```

So I needed to verify with you as it would be completely reckless for me to run something on a port like 8000, on a live server, when there is even a *small* chance of conflict.


> I was proposing SSH only for your testing.

Right. I confused things by using "testing" to mean two different things:

1 - Testing (staging) a new subsystem (weblate) before deploying it on a live server.

2 - Testing (demoing/debugging) the actual deployment (for which SSH will be great, good idea)
(0015694)
buckE   
2020-04-20 03:55   
Ah! From the install page (https://docs.weblate.org/en/latest/admin/install/venv-debian.html):

If you don’t want to make Weblate user a superuser in PostgreSQL, you can omit that. In that case you will have to perform some of the migration steps manually as a PostgreSQL superuser:

CREATE EXTENSION IF NOT EXISTS pg_trgm;
(0015697)
Christian Grothoff   
2020-04-20 10:39   
CREATE EXTENSION pg_trgm; executed.
(0015698)
buckE   
2020-04-20 10:48   
Great! Weblate is up.

Currently running in a `screen` session under `weblate` user, port :8000, verified with "ssh -N -L 8000:localhost:8000 weblate@taler.net". Alternatively you could install w3m if you only want to test locally (then "w3m http://127.0.0.1:8000" should work.)

Celery is not running but that depends on the celery-weblate service, so please start this when you are ready to go live. The weblate systemd service also can be started at any time.

I don't know how you want me to send you the admin password but I have it available when you're ready.
(0015700)
Christian Grothoff   
2020-04-20 10:52   
Admin PW: just GnuPG e-mail it to me and Florian.
(0015701)
buckE   
2020-04-20 10:55   
Done. Closing issue. Please reopen if there are any problems with the systemd service, etc.
(0015702)
buckE   
2020-04-20 10:56   
systemd services delivered
weblate running on port 8000 (currently/temporarily in screen session)
(0015706)
Christian Grothoff   
2020-04-20 12:05   
Is celery also running? Note that you can reach the site under https://weblate.taler.net/ already.
(0015707)
Christian Grothoff   
2020-04-20 12:10   
(Last edited: 2020-04-20 12:45)
I've tried to invite myself as a "normal" user, but did not get the invitation e-mail.
Similarly, I created weblate@taler.net and tried to add that address for the admin account, but that didn't work either.
=> I suspect somehow the setup is unable to sent emails?

All of the (i) links to further documentation go to 404 pages (seems, eh, unusual).
Also, /manage/performance/ lists a bunch of errors/warnings.

(0015708)
Christian Grothoff   
2020-04-20 12:11   
So point (7) of this issue is still wide open.
(0015716)
buckE   
2020-04-21 05:16   
> Is celery also running?

Not on my end. It will be handled by the celery-weblate systemd service once you set that up and create /etc/default/celery-weblate:. Here is the link again: https://docs.weblate.org/en/latest/admin/install.html#background-tasks-using-celery

> I've tried to invite myself as a "normal" user, but did not get the invitation e-mail.
> => I suspect somehow the setup is unable to sent emails?

Right. I don't recall seeing e-mail setup in the specs(?) and I assume you did not expect me to do this as you did not provide the requisite information I would need (meaning, the e-mail server settings to use.)

Here is the doc: https://docs.weblate.org/en/latest/admin/install.html#configuring-outgoing-e-mail

If you want to provide the values for those variables, I can add them to the settings.py file. But that wouldn't make sense because I do not have access to the e-mail server/accounts, so I could not test/debug and we'd end up playing a silly game of telephone (ex. "I just sent a test e-mail, did it come through?" "no, try again" etc.) This is more appropriately a root task because root controls the e-mail, no? But if you want to provide the settings, I will add them.

> All of the (i) links to further documentation go to 404 pages (seems, eh, unusual).

It does seem unusual. But without specifics (ex - which link(s)) I can't say anything specific about why you might be experiencing that. The first thing I noticed is that the (i) links I found go to URLs on docs.weblate.org, so those errors are unrelated to this install.

Example:
http://127.0.0.1:8000/accounts/profile/#notifications (to the right of Notification settings)
https://docs.weblate.org/en/weblate-4.0.1/user/profile.html#subscriptions

If the URLs you have trouble with are not off-site, the next thing I would do is test using an SSH tunnel and 127.0.0.1:8000 vs weblate.taler.net. If it fails only on weblat.taler.net I would look at the nginx logs. I can do this comparison if you want to provide a specific bug report, but of course if the problem is only on weblate.taler.net I won't be able to debug the nginx config. This is another example of why it is not a great idea to have non-priv users working on systems like this, and why stage servers are so valuable.

> So point (7) of this issue is still wide open.

Of course it is. That reads: "7) test, integrate, import existing po files, document, etc."

That line was not usable on its own (ex: test *what*?, integrate *what* with *what?, import *which* po files, etc.) so I asked some of those questions. And you clarified that the deliverable was "Result: Weblate instance running on weblate.taler.net, where we then can import po files, add users, manage translations, etc. A few of us setup as 'administrators' on the sub-domain. This should be fast and easy, maybe a 2h task."

You have that, although it will require root-level info for setting up e-mail for certain components to work correctly. I can set that up once I have more info. And as far as "A few of us setup as 'administrators' " goes, well obviously that is not actionable in the absence of exactly who to set up as administrators. When you have that info, please feel free to send it and (assuming e-mail is done) I can add some administrators if you'd like.

And of course there may be the 404 bug to deal with, and I am happy to do that when I have more info.
(0015740)
buckE   
2020-04-23 06:43   
Resolving this issue as replaced by these tasks:

https://bugs.gnunet.org/view.php?id=6201 (Christian)
 
https://bugs.gnunet.org/view.php?id=6202 (buck)

https://bugs.gnunet.org/view.php?id=6203 (Christian)

https://bugs.gnunet.org/view.php?id=6204 (buck)

https://bugs.gnunet.org/view.php?id=6205 (buck)


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6145 [Taler] other feature N/A 2020-04-01 16:23 2020-07-24 11:56
Reporter: Christian Grothoff Platform: i7  
Assigned To: buckE OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Package Taler exchange for Debian
Description: We want to eventually install a Taler Debian package for the production deployment, instead of compiling from source on the production system. For this, the existing exchange TGZ should be packaged, with the (existing) GNUnet package (and others) as dependencies.

As we cannot rely on Debian always integrating our latest versions into their build farm, we also need to setup our own Debian archive so we can add it to the sources.list on the production system to install from there. The deliverable here is again the scripts and configuration (nginx) files to setup the archive (and sign the binaries, etc.).
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0015571)
buckE   
2020-04-08 10:27   
> the existing exchange TGZ should be packaged

Where is this existing exchange TGZ? Or should I just start from scratch with the exchange.git source?

> We want to eventually install a Taler Debian package for the production deployment

Specifically, this "Taler Debian package for the production deployment" means a .deb package to install and run the exchange, yes?
(0015580)
Christian Grothoff   
2020-04-08 13:52   
(Last edited: 2020-04-13 10:13)
https://ftpmirror.gnu.org/taler/

(0015635)
buckE   
2020-04-13 08:08   
Thank you. How does this relate to the current version in exchange.git?
(0015636)
Christian Grothoff   
2020-04-13 10:42   
It doesn't, but when we make a new release it should be easy/trivial to update the package for the new release.
(0015645)
buckE   
2020-04-14 03:20   
Sorry, by "this" I meant the TGZ. How does the TGZ that the spec specifically suggests using as the base for this project relate to the exchange.git which is what we discussed verbally?
(0015646)
Christian Grothoff   
2020-04-14 11:22   
The TGZ is created from exchange.git by running 'make dist' in exchange.git.
(0015671)
buckE   
2020-04-17 08:59   
Right, so you understand my confusion then. The spec says "the existing exchange TGZ should be packaged" but there also is the exchange.git, which should be (I think?) the base of this project. Why would the spec mention the "existing exchange TGZ" then? Given the existance of the exchange.git, this seems like a non-sequitor *yet it is the actual spec.* This must be cleared up before I can begin.
(0015673)
Christian Grothoff   
2020-04-17 09:44   
Again, forget about exchange.git for this. It exists. Sure. It relates to the TGZ.
But what should be packaged is taler-exchange-0.7.0.tar.gz from the FTP server as the bug has always said.
How is this difficult to understand!?!?
(0015695)
buckE   
2020-04-20 04:26   
> But what should be packaged is taler-exchange-0.7.0.tar.gz from the FTP server as the bug has always said.

Perfect!

> How is this difficult to understand!?!?

Just growing pains. In our conversation, you mentioned the source being the git repo, not the TGZ and I wanted to make sure I was doing the right thing. But you are 100% right: I should go on the spec above only. Thank you for clearing it up. I'll start on this soon.
(0015717)
buckE   
2020-04-21 06:46   
Taler is not ready to be packaged, at least not at the quality level that I like my deliverables to meet. I am going to walk through this not because I want feedback or am willing to proceed at this time but out of courtesy because I want to be sure you understand why this is so.

1 - exchange/contrib/exchange-template does not exist in the taler-exchange-0.7.0.tar.gz file, though it is referenced as a step in the INSTALL file. It *does* exist in the repo, but you were very explicit that it is the tar.gz that should be used.

2 - But one persists. I figured I'd use the version in the repo anyway because there's no option. However there is not enough information to smoothly build and configure this. For example, https://docs.taler.net/taler-exchange-manual.html#configuration talks about what tools to use to configure the exchange, but not what the variables mean or values are. Maybe this is documented elsewhere but it is out of scope for this particular task (possibly why you wanted me using the .tar.gz?)

3 - But one persists. "You need libgnunetutil to build this program." Okay, sudo apt install gnunet. Well actually it's gnunet-dev. OK. Now:

  *** You need libgnunetjson to build this program.
  *** Make sure you have libjansson installed while
  *** building GNUnet.

  NOTE: libjansson4 and libjansson-dev are both installed

4 - One persists. Which version of gnunet should I be looking for libgnunetjson in anyway? v10 in the Debian stable repos? v12 on gnunet.org, which says "The following is a list of issues with GNUnet 0.11.0 that will need to be addressed before we might consider GNUnet deployable..." thereby indicating that it is not ready for release? Certainly I would not want a .deb file with my signature on it to require a package that has an oversight like this.

5 - But one persists. Except the debian buster gnunet-dev package does not install libgnunetjson, and unzipping v12 doesn't show it. Maybe I have to compile v12 to build the library?

4 - The gnunet v12 README says "These are the direct dependencies for running GNUnet:" and proceeds to list - not packages but - individual programs within the packages, making the hunt for the proper packages a guessing game.

I'm throwing in the towel. It is easy to create a debian package out of a tgz that is properly packaged and documented, which was what I gave an estimate for. It might also be easy to hack through these issues if I had experience with the project. But to be blunt: the taler exchange, and possibly gnunet (not sure) is not mature enough to be packaged as a .deb that I could take pride in at this time.

I am very willing to help you get it there, but that will require big and systemic changes and is out of scope for this task.
(0015718)
Christian Grothoff   
2020-04-21 08:34   
Remarks:
1) That's some configuration files, not critical, maybe nice to have. Still, you simply found a bug, we either should stop using them (and remove the reference in INSTALL) or package them properly.
2) Yes, packaging != configuration, for the first round at least.
3) libgnunetjson is a part of GNUnet that is _only_ build by GNUnet if at GNUnet-compile-time libjansson-dev is present. Sounds like the GNUnet package you got doesn't include it. => Need to fix the GNUnet package first before you can get this to work.
4) Taler usually requires a 'matching' GNUnet version, i.e. Git master only builds with GNUnet git master. We usually _try_ to get a matching GNUnet release just before a Taler release, but IIRC that didn't happen for 0.7.0. So you'd need the GNUnet git master from around the time of 0.7.0's release. VERY messy, I know.

Anyway, thanks for finding out the issues, I was unaware of (1) and that the GNUnet package didn't include libgnunetjansson. I think the last point (and the lack of suitable GNUnet releases) are the first we need to fix.
(0015744)
buckE   
2020-04-23 10:43   
Christian,

Compiling gnunet-debian from git repo, I get an error "sqlite3 not found, but sqlite3 is required" but sqlite3 is installed on my system. Using buster. Is this a known issue? Something you have an idea where to start looking maybe?
(0015746)
buckE   
2020-04-23 10:52   
Closing this issue and replacing with these:

https://bugs.gnunet.org/view.php?id=6210
https://bugs.gnunet.org/view.php?id=6211
https://bugs.gnunet.org/view.php?id=6212

Christian, please take a quick look and verify I understand the objectives.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6144 [Taler] Web site(s) feature N/A 2020-04-01 16:20 2020-07-24 11:56
Reporter: Christian Grothoff Platform: i7  
Assigned To: buckE OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: WooCommerce installation / demonstrator
Description: BFH students developed last year a Taler payment plugin for WooCommerce. We should add the resulting demonstrator as woocommerce.demo.taler.net.

For this, we need a VM running WooCommerce with the Taler payment plugin, including _good_ instructions and/or shell scripts for how to set it up from scratch. The resulting VM should be integrated with demo.taler.net via a reverse-proxy setup configuration file in the existing nginx HTTPS server (for the respective virtual host name).

As the student plugin is a bit dated (we changed some (minor) aspects of the protocol), getting it to run will likely require some (minor) modifications to the existing payment plugin.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files: API-Key-Field.png (63,290 bytes) 2020-04-04 03:07
https://bugs.gnunet.org/file_download.php?file_id=881&type=bug
demo.site (13,336 bytes) 2020-04-05 02:08
https://bugs.gnunet.org/file_download.php?file_id=882&type=bug
Notes
(0015486)
buckE   
2020-04-02 07:09   
Excellent, thank you. Where should the " _good_ instructions and/or shell scripts" (which will be later used to build "woocommerce.demo.taler.nen" be committed?
(0015487)
Christian Grothoff   
2020-04-02 10:03   
I'd put it into our Git repository with the WooCommere-Taler extension.
(0015488)
Christian Grothoff   
2020-04-02 10:06   
Oh, I should say the 'generic' instructions should go into that Woocommerce-Taler extension Git repo.

The very specific details for our _demo_ deployment should likely go into the onboarding manual in docs.git, and scripts specific to our installation (like the nginx configuration) go into the gv-maintenance.git (to which you probably don't *yet* have access).
(0015493)
buckE   
2020-04-03 00:41   
Okay. First commit made, just beginning the process. I moved the plugin to a /plugin directory for clarity. Yes the demo will be a different project essentially. A special case of this general one.
(0015494)
buckE   
2020-04-03 05:17   
Making good progress. Plugin is installed, some phrasing updated, and docs, scripts, assets are progressing well. Please consider making a backup of your current copy of the repo before doing a pull. I made major changes to the directory structure and minor changes to the plugin.

Christian, can you create a woocommerce-taler-demo-test@taler.net e-mail forwarder that forwards to me? I will use it to set up a demo *merchant* account and my personal customer account can be used to test as a customer. Or do you have a better suggestion?
(0015496)
Christian Grothoff   
2020-04-03 10:35   
E-mail alias should be setup now.
(0015517)
buckE   
2020-04-04 03:07   
E-mail alias seems to be working (buck@taler.net), but I am confused how to acquire the API Key that I need to test the plugin. I thought I would find it by creating a new Taler account and clicking on https://backoffice.demo.taler.net/ when logged on. But that was something different.

https://bank.demo.taler.net/profile does not offer this.

The page https://docs.taler.net/core/api-merchant.html discusses the API and it looks like `merchant_pub` is the variable holding the public key for the merchant. But I still don't see where to acquire that key.

Also the "GNU Taler Payment URL" is not clear to me.

It's probably right in front of me somewhere. Can you point me to where?
(0015523)
Christian Grothoff   
2020-04-04 10:16   
See section 6.1.4:
  headers={"Authorization": "ApiKey sandbox"}

"ApiKey sandbox" is the actual API-key of the payment URL
http(s)://backend.demo.taler.net/ (because it is for testing, also HTTP works!).
(0015530)
buckE   
2020-04-05 01:54   
Ah, okay. This info is for the demo and it will allow me to test this WooCommerce plugin. But what I'm looking for is how I can *acquire* an API key. Because I am writing a doc for end-users of the Taler WordPress/WooCommerce plugin, and most of those users will not be using the demo site, or have a Taler server of their own. They will be webstore owners who make an arrangement with a Taler provider, and part of that arrangement will be to sign up for an API key. If this is a process that can be done online (instead of manually, in a conversation between parties), then I would like to know how so I can add this to the docs.
(0015531)
buckE   
2020-04-05 01:55   
Also is there a good way to verify that the API key is being accepted? I am sending requests to the backend using curl/json. I do not get authorized:

```
$ curl -X GET -H "Content-type: application/json" -H "Accept: application/json" 'https://backend.demo.taler.net/check-payment{"Authorization":"ApiKey sandbox"}'
<html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx</center>
</body>
</html>
```

It does not look like the Taler plugin generates logs (or maybe I don't know where to find them) so I am looking for a place to start. The debugging is necessary because after filling out the fields with the information you gave me above, adding an item to the card, and attempting to pay with GNU Taler, I see this error:

`There seems to be a problem with the payment process, please try again or send the following message to a system administrator: 400 - request`

This could be anything. Bad address, wallet issue, etc. So I want to debug one step at a time, beginning with manual (curl/json?) authorizing with the API key.
(0015532)
Christian Grothoff   
2020-04-05 02:08   
The API key and authorization is completely handled by nginx. So users that do not run their own Taler merchant backend will have to obtain it from their backend provider -- and that backend provider would have to implement the access control in some kind of reverse proxy (like we do with nginx).

As for your failing authorizations, I am confused. Here is the respective excerpt from the nginx configuration:

    # match the ApiKey part ignoring case, and the actual key
    # with case-sensitivity on.
    if ($http_authorization !~ "(?i)ApiKey (?-i)sandbox") {
      return 401;
    }

I've also attached the full configuration of nginx for your reference. That's where the 401 comes from...
(0015533)
buckE   
2020-04-05 04:56   
I don't see where $http_authorization is defined. What is this referring to? My experiences with nginx authentication use something like this: https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/#config.

Are there any examples at this time where this authentication method and API Key are working? I would like to see the exact string that is being sent to the webserver in such a working example.
(0015534)
Christian Grothoff   
2020-04-05 12:21   
$ curl -X GET -H "Content-type: application/json" -H "Accept: application/json" -H "Authorization: ApiKey sandbox" 'https://backend.demo.taler.net/check-payment'
{
  "code": 1008,
  "hint": "order_id required"
}
(0015539)
buckE   
2020-04-07 05:49   
Making progress. I don't know how much information you like so I am giving you a medium amount of detail.

The plugin definitely has problems. Some fields are mislabeled, some are missing, but that is not the cause of the earlier problem. This error above was a generic WooCommerce error that would have been shown regardless of the backend error. The backend error was caused because I was using GBP, not KUDOS. I added KUDOS to the WordPress site and now I have a new problem (so, progress!, and added to the Howto doc for people who will use KUDOS or other custom currency.)

Go to the checkout page, enter data, start the transaction, I am told "function xdg-open wants to open this application" which was very strange to me, but I chose to accept. And then the checkout screen hangs (can not click anything.) Checking on the transaction shows me this:

$ curl -X GET -H "Content-type: application/json" -H "Accept: application/json" -H "Authorization: ApiKey sandbox" 'https://backend.demo.taler.net/check-payment?order_id=wc_order_P1jNw5WBqZEBX-15'
{
  "taler_pay_uri": "taler://pay/backend.demo.taler.net/-/-/wc_order_P1jNw5WBqZEBX-15",
  "contract_url": "https://backend.demo.taler.net/public/proposal?instance=default&order_id=wc_order_P1jNw5WBqZEBX-15",
  "paid": false,
  "already_paid_order_id": null


So I think that the problem is that the wallet is not being called. That was in Brave, and also the fee (40KUDOS) was more than I had in the wallet. So I am ignoring that problem for now.

Lowering the price to 3KUDOS, and using a Firefox browser with a wallet that has 3.x KUDOS in it, I make the transaction and I get this error:

```
The address wasn’t understood

Firefox doesn’t know how to open this address, because one of the following protocols (taler) isn’t associated with any program or is not allowed in this context.

    You might need to install other software to open this address.
```

The URL in the URL bar is "taler://pay/backend.demo.taler.net/-/-/wc_order_XIQ3OvPphCHzt-16"

So I think now the problem is with the taler *browser* extension, not the WordPress plugin. Does this sound correct to you?

How do you expect a browser to learn how to handle a `taler://` URI?

Note that this is different from the Brave error, because with the Brave error, the transaction was registered on the backend. But in this case, the same `curl` command shows that the backend was not informed:

```
$ curl -X GET -H "Content-type: application/json" -H "Accept: application/json" -H "Authorization: ApiKey sandbox" 'https://backend.demo.taler.net/check-payment?order_id=wc_order_P1jNw5WBqZEBX-16'
{
  "code": 2913,
  "hint": "unknown order_id"
}
```

So we have several problems, but I am sure you will agree that that's good. And maybe your understanding of the system will suggest a good place to start debugging.

Note: Chromium Browser behavior is exactly like Brave. xdg-open window asks: "https://10.137.0.28 wants to open this application." and I choose to allow, not cancel.
(0015543)
buckE   
2020-04-07 07:11   
On another note, is there a "standard" for the currency symbol for KUDOS? I chose "KS" not for the abbreviation (like GPB/EUR) but the symbol like "$". I think it is important to be consistent whenever possible.
(0015545)
Christian Grothoff   
2020-04-07 10:11   
The debugging of where/how things changed in the protocol (wallet-interaction, backend) is not something I expected you to (primarily) do. Let's get the site running on taler.net, and *then* Florian, Marcello and I will take a look and (likely) fix the things that are due to (usually trivial, once understood) protocol changes.

I don't see any documentation/instructions on how to setup Wordpress/Woocommerce/etc. on the server in the Git yet.

As for your questions: we change the version when it runs ;-). exchange != backend: exchange = exchange.git, backend = merchant.git.
(0015567)
buckE   
2020-04-08 06:51   
nginx config instructions and sample wordpress.nginx.site file added to repo (under server-build/Server-Build-Instructions.md and server-build/assets/wordpress.nginx.site)

Tested working in my dev VM. (YMMV if installed on a site with other nginx configs.)
(0015568)
buckE   
2020-04-08 06:56   
At this point, server-build/Server-Build-Instructions.md should have clear directions for building a Debian 10 server from scratch to end with a WordPress/WooCommerce site, running on apache2 or nginx, with a LetsEncrypt or self-signed SSL, with GNU Taler configured, KUDOS supported, a sample product added, and it should fail in the above way.

If you have problems, or you need something else from me please tell me. (ex. Maybe the error above is not with the plugin after all?) But at this time I will assume I am completed with my side, and I am very interested what solves the problem I describe in https://bugs.gnunet.org/view.php?id=6144#c15539
(0015582)
Christian Grothoff   
2020-04-08 13:59   
Ok, I'll take a look at the build instructions and will report back.
(0015591)
Christian Grothoff   
2020-04-08 19:19   
Ok, let's see.

Given that PHP is a mess with nginx, the choice of Apache makes sense.

Not sure about MariaDB, but good enough.

Firewall: eh, no. Sure, you can leave a remark to ensure that those ports ought to be open, but please don't do this on our systems.

Let's encrypt: doesn't _necessarily_ belong here IMO, good for generic instructions, but just to be clear on our server you'll need to set it up with just _HTTP_. TLS termination will be done by nginx.

wget: not sure about using -latest explicitly here, that _may_ eventually result in an incompatible/broken version. I'd at least use $VERSION and then one can do VERSION=latest in the shell script (see below).

Overall, the current instructions result in a typing-marathon (see my initial report "and/or shell scripts for how to set it up from scratch"). We should minimize the instructions more and at least for our setup replace almost everything with "run # woocommerce-installer-script.sh". The shell script can then have the details from your write-up as comments. Steps that cannot be fully automated (like setting password for MariaDB) could be output (echo) at the end with some text "reminder: you should probably do this" including of course why.

Nevertheless, good starting point.

So next steps:
(1) Boil instructions down to what we actually need, i.e. create a (robust!) shell-script that does all of the main setup logic we need (HTTP-only, no firewall). So assume 'minimal' Debian installation was finished, and then the instructions should be "run this shell script", and _then_ optionally "run this to set passwords" and otherwise "configure now like this the Browser".

(2) You can now log into test-woocommerce@taler.net. The user is in the 'kvm' group. Please setup a minimal Debian 10 VM that listens on TCP:localhost:9999, and forwards that traffic to the internal Apache2:80 (http). Run your installation script. https://woocommerce.test.taler.net/ is already setup (DNS, TLS, X509) as a reverse proxy to http://localhost:9999/.

(3) Try to automate our _specific_ setup. I.e. find a way to avoid having to use the browser at all. Maybe there is some Wordpress CLI tool (similar to drush for Drupal), or worst case hack the settings with SQL into the database from a shell script. The script should take key parameters as arguments, like the currency (KUDOS) and the URL of the merchant backend.

Please make sure that (2) and (3) are _very_ easy and fast to do. Ideally have a single (shell) command to run to generate the Debian kvm/qemu VM, setup an easy way to login to the guest VM for anyone who is "test-woocommerce" on the host. We'll want to be able to easily and *reliably* re-do the setup, including on other systems as well as for the woocommerce.demo.taler.net-site.
(0015593)
buckE   
2020-04-08 21:34   
I'm confused. Why are you talking about "our server?" The task as I understood it was to document how a generic person who wants to sell his products online can build a Debian/WordPress/WooCommerce/GNU-Taler shop from scratch. I think the instructions do this. We did not lose any time over the confusion, because everything up until installing GNU Taler was very quick. But I mention this because it is not the only task I have where I am learning the *actual* task later.

Also, if the goal is to install something on one particular server, why would we have generic instructions and/or BASH scripts for a one-time installation? You understand why I'm asking, right? Why automate something that we will do one time? I think this is an indication that I still do not know the *actual* goal.

But now that you define (2) and (3) I can re-do this to that end. I am not familiar with WP-CLI or WC-CLI but they should be not very difficult to work with. Notes:

> wget: not sure about using -latest explicitly here,

Agreed. This is why I preferred instructions to a shell script for the generic instructions. If it breaks, it will be clear where and why. For example, installing PHP7 instead of PHP13 or whatever. But this will be different for a specific installation, and the script will specify "this script installs WordPress version X, PHP7.3, etc."

> Let's encrypt: doesn't _necessarily_ belong here IMO, good for generic instructions,

Right, this was intended to be generic instructions for a GNU Taler user who wants to create a WooCommerce store.

> irewall: eh, no. Sure, you can leave a remark to ensure that those ports ought to be open, but please don't do this on our systems.

Right. Again, I had no indiciation this had anything to do with "our systems," I thought I was writing docs for end users.

Basically I think this is a completely new task that borrows nothing from the original one except how to add KUDOS. Okay, I'll get started.
(0015594)
Christian Grothoff   
2020-04-08 21:53   
Ok, let me clarify:
1) we need instructions for _others_ to reproduce their own setup. Good, we have some of those. As I tried
    to indicate, I'm pretty happy with those even though I think some more scripting can reduce the hassle
    to the user. But, that was only half the goal:
2) we ALSO need to do our own deployment (for testing and for demonstration). We ALSO want to be
    prepared to re-do our own deployment reasonably quickly (migrating servers, updating to a new version,
   we also usually run 2-4 setups on the same server (!) of almost everything (!) for testing and development).
   Finally, we want things to be maximally reproduceable. Hence this should be semi-automated.
(0015595)
buckE   
2020-04-08 23:09   
Thank you. It's making much more sense.
(0015598)
buckE   
2020-04-09 08:16   
Do you want me to commit the (4GB) base VM image?
(0015599)
buckE   
2020-04-09 09:13   
(More like 2G now)
(0015600)
Christian Grothoff   
2020-04-09 09:47   
Obviously not. We very, very rarely put generated code into Git. The image should simply live in /home/woocommerce on gv.taler.net. The shell-script to create such an image should live in Git.
(0015606)
buckE   
2020-04-10 01:48   
Is it feasible to add KUDOS support to the plugin? It would avoid one more hack:

https://docs.woocommerce.com/document/add-a-custom-currency-symbol/#

Status: At this time I have a script that goes from a basic Debian VM to a functional basic WP site available to host at port 9999, and I have that basic Debian VM. This is working and reproducible.

But going from functional WP site to the end goal by cli is much more complicated due to the nature of WP. The wp-cli tool doesn't allow things like including custom code in a child theme's function.php file. Sure I can do that on the BASH terminal - until there is a new theme with a new version of WP and we have to re-create the script for that customization.

(I'm also not sure how to build a VM with a functioning OS without logging into it to install the OS, or without importing a filesystem, but that's for later.)

One suggestion I have is that I create a base WooTaler VM that has a functioning site that just turns on and runs. Then on this guest system, if a user wanted to upgrade it (ex: when GNU Taler Plugin is upgraded), that would require logging on, git pull an *update* script for Debian/PHP/WP/WC/GNU-Taler-Plugin (that was requested/tested for package compatibility at that time), run it, test the website, log off the VM. Of course that's just as easy as rebuilding the VM.

Note: gv.taler.net does not have rsync installed. I don't know if that is intentional but I would eventually want to use it to upload the VM.
(0015607)
buckE   
2020-04-10 09:22   
> The script should take key parameters as arguments, like the currency (KUDOS) and the URL of the merchant backend.

Okay if we want to make that part of the script, the "recommended" method would be to add wp-cli support to the taler plugin (maybe a good idea anyway?) : https://wpreset.com/add-wp-cli-support-wordpress-plugin/

If you don't want to do that, I assume I can set the parameters manually in the PHP of the plugin, but depending on how the plugin is written, it may or may not be overwritten on upgrade.
(0015610)
Christian Grothoff   
2020-04-10 11:50   
Yes, we should probably avoid the KUDOS hack by putting something into the plugin, that sounds certainly better than hacking custom code in a child theme's function.php file. Or re-use some other currency symbol, I just liked the idea of using "KU".

rsync: eh, I had expected you'd create the 'final' VM on the server locally (should save you a ton of bandwidth, and if things are sufficiently automated, shouldn't be extra work). But sure, we can install it.

wp-cli support in the plugin: yes, that would be good to add. If this is something _you_ feel you can do, that would be perfect (as a follow-up task).

About "(I'm also not sure how to build a VM with a functioning OS without logging into it to install the OS, or without importing a filesystem, but that's for later.)": I was thinking of using the FAI/Simple-CDD facilities of Debian:
https://debian-handbook.info/browse/stable/sect.automated-installation.html
This is _also_ what I was thinking for the 'bigger' task of creating a simple USB stick for a fully-automated reproduceable exchange deployment (except there we then need to also create some Git-based setup to interact/upgrade the running system).

About upgrade-script: sure, having an upgrade script would be nice, too. Especially for production deployments where one would want to minimize downtime and also not throw away the DB :-).
(0015611)
Christian Grothoff   
2020-04-10 11:51   
Oh, and as long as *our* plugin doesn't support wp-cli, it's OK to leave those steps interactive in the Browser. Please open a bug that our plugin should support wp-cli, pointing to the wp-cli documentation and saying which options are relevant here.
(0015619)
buckE   
2020-04-11 06:45   
TL;DR: There exists a README and scripts that successfully build a server and site when run on barebones Debian. Automating the image creation is next. Should be easy compared to the wp-cli nonsense.

> Yes, we should probably avoid the KUDOS hack by putting something into the plugin, that sounds certainly better than hacking custom code in a child theme's function.php file. Or re-use some other currency symbol, I just liked the idea of using "KU".

For now the script provides and exit instruction to handle this in the GUI along with a couple of other unavoidable things.

> I had expected you'd create the 'final' VM on the server locally

Okay but that seems reckless. And it would provide no opportunity for debugging if there are errors. But now that I'm understanding the actual specs better, I do see how this *could* work and if you like actually doing it, that's what I will do. No need for rsync then at this time (but it is useful, no?)

> https://debian-handbook.info/browse/stable/sect.automated-installation.html

Okay now I'm starting to understand the project goals. I'll look into this.

> This is _also_ what I was thinking for the 'bigger' task of creating a simple USB stick

Interesting. There must be parts of that project I don't yet understand.

> having an upgrade script would be nice, too

That was a suggestion if we would have a base disk image. If we're creating the image on the fly, this seems redundant?

> as long as *our* plugin doesn't support wp-cli, it's OK to leave those steps interactive in the Browser

Done.

> Please open a bug that our plugin should support wp-cli, pointing to the wp-cli documentation and saying which options are relevant here.

Okay. Although you also said "wp-cli support in the plugin: yes, that would be good to add. If this is something _you_ feel you can do, that would be perfect (as a follow-up task)" so I am confused but I will open the bug and assign it to you. You can reassign how you want.
(0015621)
Christian Grothoff   
2020-04-11 11:11   
rsync is already installed, even if you won't need it this time.

I wouldn't say that we "are creating the image on the fly". We *automate* doing so, sure. But as I said, the upgrade script has a different purpose (not so much for our deployment, but for end-users that need good uptime). Anyway, the upgrade script is not really important IMO (and was not part of the original plan).

cli-bug: it's OK, let's discuss it once we're done with the existing bugs (this one, and the exchange.deb).
(0015632)
buckE   
2020-04-13 03:18   
I'm looking at FAI and it is very interesting. I will use it for this if you want, but it is not more elegant than my suggestion of using a base VM image. Basically my concern from above: "I'm also not sure how to build a VM with a functioning OS without logging into it to install the OS, or without importing a filesystem" is not answered by FAI. Based on my reading, FAI (not surprisingly) requires importing a filesystem, and also having the FAI server available to every client that will manifest a VM. So it is less elegant than having a base .img file (although the required FAI ISO is *smaller* than the base IMG file would be, but I don't see how that helps in this specific case.)

However if the real purpose of this task is as a trial for the larger task, (a) I'm not sure there aren't better ways to accomplish that either (really, FAI seems not different in principle from using Puppet or Ansible) but (b) okay at least I understand - though I wish I'd known the real task when I'd started. Anyway I'm proceeding and at worst I will be much more familiar with FAI shortly.

But the image is ready at this time if we changed course to meet the specific goals of an automated WooTaler (my nickname for it) installation. (And the docs are written for a non-automated general purpose install.)
(0015638)
Christian Grothoff   
2020-04-13 11:01   
The FAI has the advantage that it works for bare-metal installations, which is what we will want for the Taler exchange deployment eventually. So you getting up to speed on FAI here has other advantages.

Eh, FAI doesn't require importing a filesystem *or* running an FAI server. Maybe you are reading up on something else!? See Debian handbook I quoted above, section 12.3.2.1, choices 1 or 2: local preseed.cfg, stored in initrd root or on the USB key. Very easy. (I'd use the 2nd option.). That said, FAI is just the base tool, likely we'll want the full Simple-CDD (12.3.3) in the end.

So all you do is (1) create the simple-sdd.conf and then given that configuration _anyone_ can (2) invoke build-simple-cdd to create an ISO for the installer image, and (3) run, qemu to boot from that installer image, plus a target drive to create the image. Finally, (4) run qemu against that image, and voila Wordpress/WooCommerce. Steps (2-4) can be in a shell script:

#!/bin/sh
# create installer
build-simple-cdd ARGS installer.img
# create VM image
qemu ARGS installer.img target.img
echo "You can now launch with $ qemu ARGS target.img"

(Admittedly, the shell script won't be that simple for long, given that you'd want to test for the tools like build-simple-cdd to be locally available, actually create the target.img, maybe take arguments for the size of the on-disk image, etc.).

This way, our Git will contain a tiny configuration file and a shell script and some instructions (README), and everything else, all binaries, can be generated by the user from source!
(0015652)
buckE   
2020-04-15 09:32   
Ah. Well, okay. I think simple-cdd is a complicated solution to a simple problem, and if you always had this in mind it would have been nice to read in the original spec, *but* it's no problem.

> Eh, FAI doesn't require importing a filesystem *or* running an FAI server.

Actually I think it does, according to the docs (yes, the link you posted.) Using any method other than simple-cdd, FAI needs it for the basic setup (ex: initrd lives on the client) but even when using simple.cdd it appears it is necessary when doing upgrades (because debconf may require different options for new versions of Debian.) But that might not matter for this task, since upgrades might not be needed (or can be done with an upgrade script?) and for the larger task having a server will be a possibility I guess? FAI/simple-cdd won't be difficult, but it's definitely not a tool I would have used for this task without specific request or else this would have been done some time ago.

Anyway, let's call this the final spec (please) and I'll take care of it shortly.
(0015653)
Christian Grothoff   
2020-04-15 10:44   
When I used FAI in the past, I didn't need a server. I still don't find anything in the handbook link I sent to suggest otherwise. And I'm not sure I see how it helps with upgrades, there clearly one needs to make a new configuration image. Anyway, if simple-cdd allows you to do it without, then just do that and we'll both be happy. ;-).
(0015665)
buckE   
2020-04-16 09:08   
> When I used FAI in the past, I didn't need a server.

If you can provide the details of this, it will be useful information. Other than simple-cdd, each method I looked at did require this. s-cdd does not require a server (so far?) but it is a solution I would never have considered, still do not like for this task, and each attempted build is taking about 3 hours before failing and trying again, so it will take a while. But I take it that it is the solution you like for this, so I will simply proceed with it.
(0015666)
Christian Grothoff   
2020-04-16 09:18   
Sorry, it was in the *distant* past (~ 20 years ago). Is s-cdd CPU, IO or bandwidth-bound? What takes so long?
(0015667)
buckE   
2020-04-16 10:51   
The current error is:

build/debian-cd: Use of uninitialized value $default_desktop in regexp compilation at /home/buck.../update_tasks line 255

Errors like this are not out of the ordinary of course. This is just where I'm at at the moment. Creating the s-cdd structure took about 8 minutes, customizing the file was another 10. Now I'm just debugging this problem, and each edit to the preseed file (tasksel/desktop "none" or "false" or "tasksel-kde" etc) requires a rebuild (actually I learned it doesn't, but only after my second one.) It takes so long because (as I understand the doc), s-cdd builds a custom debian install CD based on the preseed (which will have defaults that make the installation non-interactive - isn't this what you said you wanted? I'm doing it because you've specifically asked, not because I think I'm on anything like a good track.)

I don't see any other examples of this error online, although there may be a tasksel bug regarding desktop environments for a close version, not sure yet.

To the other point:

> When I used FAI in the past, I didn't need a server.

The issue is server *or* existing filesystem. And of course that relevant to the tools I was looking at before you specified using simple-cdd. From the doc:

There are several places where the installer can get a preseeding file:

* in the initrd used to start the machine; in this case (meaning it requires an existing filesystem - that has initrd)

* on the boot media (which is explicitly out of scope, except that we're creating the boot media on the fly with simple-cdd, though I still don't see why)

* from the network (again, explicitly out of scope)

So anyway, that's where I am. Particularly annoying since this VM will not need a desktop. But that's life. Just working through an error message. But at least I think I don't have to rebuild each time anymore.
(0015675)
buckE   
2020-04-17 10:44   
At this time I think there was a bug in the version of tasksel in buster. And another bug in the upgraded version.

Specifically: I upgraded my system as per Debian project's recommendations (really only recommended upgrading tasksel but I upgraded my build system to bullseye), got a new error (related to bullseye InRelease file, and error is the same on my custom build and the default my-simple-cdd files), and reported that to reportbug. Putting this on hold until I receive a response from Debian.

If using the Demo WooTaler site is a priority in the meantime, the VM can be made available and instructions are in woocommerce-taler.git/server-build/QEMU-autobuild/README.md.
(0015676)
Christian Grothoff   
2020-04-17 10:57   
Ok. Please add a link here to the Debian bug. Putting up WooTaler is not a 'priority', it is/was just a good introductory task for you.
(0015687)
buckE   
2020-04-18 06:12   
Bug is here:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956958

Note that there was a different error on buster ($default_desktop not defined) and other bug reports suggested to me this could have been fixed in recent version of tasksel, which is why I upgraded to bullseye. It is not clear if that first error will reappear after resolving this issue, and I see that taler.net runs stretch so it may be necessary to upgrade tasksel if you really want to run simple-cdd when these are resolved.

> Putting up WooTaler is not a 'priority'

Okay. My nerd-brain has trouble with that because that is explicitly the task. Because "introductory task" aside, there is a question of which is a more elegant solution for this actual task:

1. upload a VM; run it with a reverse proxy

2. install pre-requisites for simple-cdd; git pull the preseed, postinst, etc. files; possbly install a custom version of tasksel; run simple-cdd; which downloads a custom Debian install CD; and creates a custom VM; and then runs a postinst script that; does another git pull for the buildWootaler.sh file; and builds the VM from step (1); which requires user intervention that is already completed in the Demo Site VM.

I have no complaint against simple-cdd per se but the original task reads:

"For this, we need a VM running WooCommerce with the Taler payment plugin, including _good_ instructions and/or shell scripts for how to set it up from scratch. The resulting VM should be integrated with demo.taler.net via a reverse-proxy setup configuration file in the existing nginx HTTPS server (for the respective virtual host name)."

(1), above, answers this (plus my instructions in the git repo)

So as someone who tends to be obsessively focused on staying in scope and making deliverables specific to specific tasks, I would like to suggest using the VM for this task, and then having a broader conversation about the larger automated build server idea, and simple-cdd will certainly be one part of that. I am skeptical it will be elegant for that eiher, but it is *cool* and it looks easy enough to use once these errors are out of the way.
(0015689)
Christian Grothoff   
2020-04-19 00:38   
Sure, go ahead and use the image so we can close this one. Note that we will need to get a working simple-cdd image generator for the exchange deployment.
(0015696)
buckE   
2020-04-20 05:11   
> Sure, go ahead and use the image so we can close this one.

Great! Currently rsync-ing two images to `/home/demo-blue`:

1 - WooTaler.img-Base - the base image, inside which you can run the build script (buildWooTaler.sh) if you want to (though I still do not understand the reason for this script, anyway it exists and it works.)

2 - WooTaler.img-LiveSite - a fully-functional WP/WC/GNUTaler site VM with sample product and appropriate copy (recommended)

Run at Host's :9999 with: "$ qemu-system-x86_64 -m 1024 -enable-kvm -k en_us -hda /home/demo-blue/WooTaler.img-LiveSite -net nic -net user,hostfwd=tcp::9999-:80"

Further instructions (not really needed) in woocommerce-taler.git/server-build/QEMU-autobuild/README.md

> Note that we will need to get a working simple-cdd image generator for the exchange deployment.

You mean the "big"/USB system you and I have talked about? Sure, if you have decided you want to use simple-cdd for the big project that's no problem.

Final note: woocommerce-taler/server-build/my-simple-cdd has the working-copy profiles/ files that could eventually build this WooTaler VM using simple-cdd once the bug is cleared up. (But I'd need to test them in a dev environment before recommending running them live.)
(0015699)
Christian Grothoff   
2020-04-20 10:51   
Rationale for (1): we'll eventually want to do the full re-building from source in a continuous integration. Ideally _including_ building the base image (hence the simple-cdd for that). But that can wait.
As for the 'big' USB/System: yes, that's the one. As for 'decided', I just don't know anything else that would satisfy our requirements. If you have something, let me know. Now, I hope the result will be "tiny" (at least smaller than WooCommerce!).
(0015703)
buckE   
2020-04-20 11:06   
> Now, I hope the result will be "tiny" (at least smaller than WooCommerce!).

Yeah I was surprised how big this VM needs to be.

> we'll eventually want to do the full re-building from source in a continuous integration

This is what I don't understand since there are so many things that will have to be customized in WP/WC manually, and each VM build will be singular and last a long time. So the time to create a new build script + inevitable manual GUI input is probably significantly more than the time to just update the single VM manually.

Also if it is not a single-use VM, and there are multiple projects using the VM-build script, they would all have to install and be comfortable with simple-cdd. Does that mean that Fedora users are out?

> As for 'decided', I just don't know anything else that would satisfy our requirements.

I haven't read the actual requirements/project details so I can't offer suggestions at this time (other than very generic thoughts like we discussed (ex, ansible, which I don't really want to use if it is avoidable.)

Current status: 1.7/1.8GB transferred on WooTaler.img-Base. I'll set the other going as soon as the first is done.
(0015705)
buckE   
2020-04-20 11:33   
Current status: WooTaler.img-Base uploaded. WooTaler.img-LiveSite (recommended) uploading now
(0015710)
buckE   
2020-04-20 23:12   
WooTaler.img-LiveSite (recommended) is uploaded.
(0015711)
buckE   
2020-04-20 23:14   
> BFH students developed last year a Taler payment plugin for WooCommerce.

Changes made to plugin (mostly language/translation issues)

> We should add the resulting demonstrator as woocommerce.demo.taler.net.

Out of scope (re: requires root access)

> For this, we need a VM running WooCommerce with the Taler payment plugin,

VM available in /home/demo-blue

> including _good_ instructions and/or shell scripts for how to set it up from scratch.

Instructions and shell scripts available in woocommerce-taler.git

> The resulting VM should be integrated with demo.taler.net via a reverse-proxy setup configuration file in the existing nginx HTTPS server (for the respective virtual host name).

Out of scope (re: requires root access)


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6140 [Taler] wallet (JS core) minor have not tried 2020-03-30 15:27 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: wallet does not fully check the individual reserve history items
Description: Right now the wallet only deals with the current balance. Instead, the wallet should keep its own version of the reserve history that is compared with the one from the exchange.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015816)
Florian Dold   
2020-04-30 10:35   
Implemented in ef0acf06bfb. The wallet now keeps a local history of the reserve and reconciles it with the one from the exchange.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6099 [Taler] deployment and operations minor have not tried 2020-02-18 23:18 2020-07-24 11:56
Reporter: Christian Grothoff Platform: i7  
Assigned To: Florian Dold OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: taler-deployment-prepare fails to properly setup exchange bank account
Description: Florian writes (I did not check):

"taler-deployment-prepare": Generate the right configuration *based on
your environment*, copy keys (if necessary for your environment),
initialize databases if necessary, all in the right order. *Caution:*
The part that sets up the bank account password of the exchange might
either not exist or be broken, that must currently be done manually.
That's a bug!
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0015381)
Christian Grothoff   
2020-02-18 23:19   
When this is fixed, we need to update the developer manual, which for now mentions the bug explicitly.
(0015608)
Florian Dold   
2020-04-10 10:55   
This has been fixed in taler-deployment-prepare for a while.

I've also updated the docs to remove the warning now.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6051 [Taler] exchange feature have not tried 2020-01-17 20:54 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: create twister test case to check if operations with the wire gateway are re-tried with the same request_uid
Description: Currently this is never tested anywhere.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016465)
jonathanbuchanan   
2020-07-16 02:49   
tested in 332b429f..b9a11865.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6050 [Taler] exchange feature have not tried 2020-01-17 20:52 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: create twister test case that sets keys expiration to zero / invalid
Description: ... this would force then existing test to re-fetch keys before every operation, and this has uncovered a use-after-free bug before. Thus we should add this as a permanent twister case, instead of manually setting the exchage's response header.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016447)
jonathanbuchanan   
2020-07-13 23:44   
test case resolved in b0c352d..4ff3f42a.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6010 [Taler] mechant backend tweak have not tried 2019-12-25 11:31 2020-07-24 11:56
Reporter: Marcello Stanisci Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Remove non-twisted tests from twisted test-suite
Description: Merchant twisted test-suite (test_merchant_api_twisted.c) contains a few tests
that do NOT use any feature offered by the Twister. Those tests should be moved
into the non-twisted tests collection: test_merchant_api.c
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016382)
Christian Grothoff   
2020-07-05 00:08   
Jonathan: I think you may have already done this when you put the twister test for the merchant back in (at least I hope there are no tests remaining in the with-twister merchant test that do not actually need the twister).

If so, please mark as resolved.
(0016395)
jonathanbuchanan   
2020-07-08 03:48   
Removed outdated code in 0203e5d..c7ecaf1.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6008 [Taler] deployment and operations minor have not tried 2019-12-23 15:42 2020-07-24 11:56
Reporter: Marcello Stanisci Platform:  
Assigned To: Florian Dold OS:  
Priority: high OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: fix permissions on keys for blue/green demo setup
Description: Deployed keys should be assigned to the 'demo' group and only have read/write permissions.
The attached script addresses this; should be applied and tested.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: permissions.diff (3,463 bytes) 2019-12-23 15:42
https://bugs.gnunet.org/file_download.php?file_id=863&type=bug
Notes
(0015217)
Christian Grothoff   
2019-12-24 00:33   
The chmod -R was pretty fatal, it removed -x from directories... Also, 440 is not ok, as some of our tools like to open for writing and fail if they cannot (possibly unnecessarily so, but that's another issue for later!).
(0015609)
Florian Dold   
2020-04-10 10:58   
There should be two parts to this fix:

* alias taler-exchange-keyup in ~/activate to something that first runs they keyup command and then fixes up permissions
* add some logic in taler-deployment-start to check if the permissions are correct, and then give a nice warning
(0015880)
buckE   
2020-05-11 09:12   
This "task" has no context.

> Deployed keys

Deployed where?

> should be assigned to the 'demo' group

Well that's probably on taler.net right? Okay, and this is something root user can do.

> and only have read/write permissions.

For the group only?

> The attached script addresses this

What script?

> should be applied and tested.

Applied in what context? As in, by which user on which server? This may be answered by answering the above.
(0015883)
Christian Grothoff   
2020-05-11 09:55   
Deployed on taler.net, using the scripts in deployment.git/bin/, which for the demo are (to be) run manually following the procedure in the onboarding/developer manual as per the demo upgrade procedure, which is documented here:
 https://docs.taler.net/developers-manual.html#demo-upgrade-procedure
(0015884)
Florian Dold   
2020-05-11 11:23   
I've already fixed this out of necessity while doing a new demo deployment about 3 days ago:

https://git.taler.net/deployment.git/commit/?id=1889794f88dac0cac98da7d180a617f8750b1091


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6000 [Taler] mechant backend tweak N/A 2019-12-20 10:27 2020-07-24 11:56
Reporter: Christian Grothoff Platform: i7  
Assigned To: jonathanbuchanan OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: test case logic is messy
Description: Florian wrote:
I've quoted a *partial* example below,of some multi-screen macro in a
test case. This raises so many questions:

* why does the macro have varargs when they are not used?
* why are there local variables in the macro?
* WTF is this a macro and not a function? it's used only once ...
* why is *everything* defined in a big string? Why don't we have helper
function that take a literal struct that describes the data and then
generates the contract terms? This is such an unreadable,
unmaintainable mess right now.

The old currency format used to look like this, in many places:

  \"pay_deadline\":\"\\/Date(99999999999)\\/\",\

Again raising the questions of:
 * did nobody think that maybe the quoting required here is a bit
excessive and should be somehow factored out? There are dozens of lines
like this.
 * why is there this magic constant, should't "/never/" or { t_ms:
"never" } work instead of some arbitary constant?

Marcello, since I'm super busy with updating some other stuff that's
critical for the congress demo, please update the test cases.

For now, just make sure that it works. But long term, we need to
untangle this mess.

- Florian


#define ALLOCATE_ORDERS(...) \
  char *order_worth_5; \
  char *order_worth_5_track; \
  char *order_worth_5_unaggregated; \
  char *order_worth_10_2coins; \
  \
  GNUNET_asprintf \
    (&order_worth_5, \
    "{\"max_fee\":\
       {\"currency\":\"%s\",\
        \"value\":0,\
        \"fraction\":50000000},\
       \"refund_deadline\": { \"t_ms\": 0 },\
       \"pay_deadline\": { \"t_ms\": 99999999999 },\
       \"amount\":\
         {\"currency\":\"%s\",\
          \"value\":5,\
          \"fraction\":0},\
        \"summary\": \"merchant-lib testcase\",\
        \"fulfillment_url\": \"https://example.com/\",\
        \"products\": [ {\"description\":\"ice cream\",\
                         \"value\":\"{%s:5}\"} ] }", \
    currency, \
    currency, \
    currency); \

[ goes on like this for many more lines ]
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016445)
jonathanbuchanan   
2020-07-11 22:52   
The macro seems to have been removed, but we're still passing a json string to TALER_TESTING_cmd_merchant_post_orders, which isn't ideal. I'll change the function to generate this string internally.
(0016446)
jonathanbuchanan   
2020-07-12 02:05   
fixed in 89f4530..d545306


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5952 [Taler] Web site(s) feature N/A 2019-10-28 13:08 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: move www.git to GNU-style build system and allow "make install"
Description: This has already been done for landing.git. We should use the new (submodule-based) build system. This makes it easy to do a "make install" instead of having to copy stuff manually in the BB jobs.

See the Makefile in landing.git for where/how to copy the website contents.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015781)
nikita   
2020-04-25 16:05   
This is finished, right?
- We use the new build-common for configure steps
- We have an install rule in the makefile
- We run install in the update_www.sh

I'm still not happy with the artifacts lying around and us doing an uninstall instead of having 2 directories which just get used (current state, new state).
(0015815)
Florian Dold   
2020-04-30 10:26   
I would consider this done.

The atomic copying of the newly build and installed website to the location that is served by nginx is IMHO a concern of the buildbot and not the www repo.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5931 [Taler] deployment and operations major have not tried 2019-10-16 17:30 2020-07-24 11:56
Reporter: nikita Platform:  
Assigned To: nikita OS:  
Priority: high OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: modify walletbuilder builder to split out into 2 new jobs
Description: > - worker-demo
> - worker-test

AFAIK these are just some *very* basic health checks that see via curl if the exchange/merchant are reachable.

the taler-wallet-cli health checks should replace those two workers, and they should be given a better name.
Like taler-{test,demo}-healthcheck
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015021)
nikita   
2019-10-22 17:28   
The builders exist since yesterday, but I'm waiting on the reply for the email dated from
Fri, 18 Oct 2019 17:48:59 +0000
to proceed with this.
(0015789)
nikita   
2020-04-27 14:40   
I checked deployment.git and it appears to be fixed.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5874 [Taler] documentation text have not tried 2019-09-04 15:36 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: taler error codes should be listed in the documentation
Description: Right now they're only listed in the source code.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0014906)
Christian Grothoff   
2019-09-16 09:54   
Are you proposing listing each error code in the respective API call where it can be generated, or a central list of all error codes?

I'm all for having a list of all error codes with the respective API call, but would (at this point) not favor basically a 1:1 duplication of the C enumeration in the documentation (we should eventually have a database with the error codes and generate the C header, Java, Python and other sources *and* documentation from that).
(0015553)
Florian Dold   
2020-04-07 17:09   
We already have a central place for error codes now, in the taler-util repository.

At some later point, we might want to move the error codes to https://gana.gnunet.org/


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5828 [Taler] merchant backend API (HTTP specification) minor have not tried 2019-08-17 14:59 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: merchant backend should have an API to query metadata such as currency
Description: This is required for integration with PoS terminals that use the backed.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015555)
Florian Dold   
2020-04-07 17:12   
Duplicate of 0005939


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5512 [Taler] wallet (WebExtensions) major always 2019-01-25 18:32 2020-07-24 11:56
Reporter: davidak Platform:  
Assigned To: Florian Dold OS:  
Priority: high OS Version:  
Status: resolved Product Version: 0.5  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: Wallet adds only 2000000 KUDOS and get stuck
Description: I have withdrawn a huge ammount of KUDOS from the bank into my wallet, multiple times.

It don't appear directly in the wallet, but it seems that every coin is added by itself, so it takes a very long time until all this KUDOS are actually added.

It appears that the wallet get stuck after adding 2000000 coins to the wallet. It starts again when i disable the browser extension and enable it again.

(Since Taler should work like cash, nobody would have such amounts in ones wallet)
Tags:
Steps To Reproduce: 1. withdraw 10000000 KUDOS from the bank
2. see if they all get added to your wallet balance
Additional Information: Chromium 71.0.3578.98
Extension 0.5.1
Attached Files: Screenshot from 2019-01-25 16-18-58.png (186,802 bytes) 2019-01-25 18:32
https://bugs.gnunet.org/file_download.php?file_id=796&type=bug
Screenshot from 2019-01-25 17-31-48.png (107,474 bytes) 2019-01-25 18:32
https://bugs.gnunet.org/file_download.php?file_id=797&type=bug
Screenshot from 2019-04-16 21-48-32.png (366,773 bytes) 2019-04-16 21:52
https://bugs.gnunet.org/file_download.php?file_id=827&type=bug
Notes
(0013540)
Christian Grothoff   
2019-01-31 19:16   
Do you get any interesting log messages in your debug logs?

Oh, and as for this being slow, that's simply because the denominations that were setup are just for much smaller amounts. But of course if the wallet does get stuck, this is still a bug that needs fixing.
(0014297)
davidak   
2019-04-16 21:52   
>Do you get any interesting log messages in your debug logs?

like this? (see screenshot)
(0014793)
Florian Dold   
2019-08-17 18:51   
In the screenshot, the errors are related to the deposit operation, and not withdraw.

If the bug is in the wallet, it should be easier to show one 0004738 and 0004108 are resolved.
(0015188)
Christian Grothoff   
2019-12-18 23:25   
Believed fixed, also better diagnostics available in latest wallet.
(0015218)
Christian Grothoff   
2019-12-24 08:44   
Seems this one is NOT fixed, and David is reporting some, eh, interesting "rounding loss" happening as well here. Please do look into this.

To get 10000000 KUDOS, simply open an account and then hack the Postgres database balance for that account. On demo, the respective SQL statement should be in the history...
(0015885)
Florian Dold   
2020-05-11 18:45   
Multiple performance and responsiveness issues needed to be fixed.

Now the wallet handles very large amounts properly.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5286 [Taler] wallet (WebExtensions) tweak have not tried 2018-02-22 11:16 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: duplicate  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: i18n for react components does not have test cases
Description: There's some non-trivial behavior in there (mostly regarding the handling of nested components, nested {...} blocks and white space), so there should be more tests.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015554)
Florian Dold   
2020-04-07 17:10   
Duplicate of 4818


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5276 [Taler] wallet (WebExtensions) minor have not tried 2018-02-07 15:05 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: consider restricting wallet permissions
Description: In the light of a recent critical security issue in a popular extension [1], I've been thinking about wallet security. And not only about the security of the coins you have, but "will all my passwords and private data be compromised if the Wallet has a serious bug".

Currently Chrome/ displays for the wallet "Permissions: Read and change all your data on websites you visit". This is obviously bad, both technically and for user confidence.

Our goal should be that it displays "Has no special privileges" (which is probably technically impossible) or "Can read and write your data on https://w.taler.net" (bear with me for the reason for this domain).

Then we're completely off the hook in regards to serious exploits, nobody can use the wallet to exploit other websites unless Chrome/FF itself has a serious bug.

Even if somebody hacks our Chrome Web Store account and uploads a rogue extension, after the auto-update users will have to approve the new extended permissions of the rogue extension.

As a preliminary technical measure, we could restrict the extension [2] to only be able to access URLs of the form "https://*/taler-payment/*". This makes us relatively safe, but because of Chrome's policy it will still show as "Permissions: Read and change all your data on websites you visit". This would require adjusting some URLs though, so not sure if this intermediary solution is worth it right now.

Now there is a better solution though, with only minimal trade-offs (it only affects people who use NoScript):

Pages can communicate to extensions directly without any special permissions, but to do that they need the extension ID. For many reasons this should not be hard-coded in the merchant, so we need some other way to get the extension ID. This is where https://w.taler.net comes in, this site itself can be blackholed (it wouldn't even matter if it's compromised), but the merchant (or rather JavaScript on a merchant backend page) will use it to get the extension ID to send the message to. When the extension is installed, it will catch the request and send back its ID, if it doesn't exist or it's compromised, worst case is that the "pay" message is sent to another extension that the user already installed.

This requires JavaScript on the merchant backend's site that triggers the payment. For noscript payments, the user would have to trigger the payment manually by opening the popup (with the "activeTab" permission, which still displays as "Has no special privileges we can read the current page if the popup is open". But this is a reasonable price to pay for having good security.

We lose the ability to do presence detection only when the user has disabled JavaScript, which is IMHO also a reasonable tradeoff.

[1] https://bugs.chromium.org/p/project-zero/issues/detail?id=1527&desc=2
[2] https://developer.chrome.com/extensions/match_patterns
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0013257)
Florian Dold   
2018-09-28 11:10   
Instead of a domain, it might make more sense to use an IP address that can't be routed, such as 240.0.0.1
(0015848)
Florian Dold   
2020-05-04 16:21   
The wallet now can run with reduced permissions. Full permissions can be granted on an opt-in basis.

This is according to the resolution (*not* the original proposal) of https://docs.taler.net/design-documents/001-new-browser-integration.html


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5209 [Taler] merchant backend API (C) minor have not tried 2017-12-10 14:47 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: low OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: no change required  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: test cases should not use fixed port numbers
Description: For example the merchant lib test uses port 8081, which clashes with some buildbot port.

We can also run into cases where two tests are executed simultaneously, with some bad luck one of them will fail.

We can find free ports e.g. with this method: https://www.dnorth.net/2012/03/17/the-port-0-trick/

The idea is to:
1) select a port that is currently free
2) start the exchange (or whatever component we need to run)
3) if it failed (because of a race on the port), try again with another port for some small number of tries

And on systems where it's not supported, we fall back to a fixed port number.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0012650)
Christian Grothoff   
2017-12-10 15:11   
We actually have logic for picking free ports like that in GNUnet (testing subsystem uses it). But it's usually quite a bit of extra work to achieve this level of flexibility, especially as we would have to _write_ configuration files with those port numbers in them in many cases.

So for now, I think we should simply avoid conflicting port numbers on taler.net, maybe use more, eh, unusual ones, but consistently changing the test logic to pick free ports is really not worth the trouble.
(0012651)
Florian Dold   
2017-12-10 15:19   
I've already changed the offending port, and now the tests work again.

Then let's not make the dynamic port selection a priority now, but eventually I'd like to have it, since when more people do something with the backend we'll hear about this again eventually ;-)
(0015537)
Florian Dold   
2020-04-06 21:00   
I think this not relevant anymore, as we have the netjail now!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
4818 [Taler] wallet (WebExtensions) feature have not tried 2016-11-27 22:20 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: test cases missing for i18n
Description: ... especially the pluralization JSX component should be tested.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015552)
Florian Dold   
2020-04-07 16:52   
Finally fixed in e404f5e6d30. The i18n react components are tested via the enzyme library.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
4466 [Taler] other tweak have not tried 2016-04-22 19:37 2020-07-24 11:56
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 0.0  
Product Build: Resolution: won't fix  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.7.1  
Summary: figure out a nice way to integrate compiled files in web-common into other repositories
Description: Right now we're using submodules, which are generally a pain. Also when we want to use typescript in web-common (for presence detection and general wallet communication) we would need to have a typescript compiler and compile that one file from scratch every time.

Checking in the generated file into the git would be an alternative, but it's ugly.

We need some project-local package manager (just like npm etc.) that doesn't suck. Guix unfortunately doesn't really help here, since it just manages global packages (and not packages on a per-project level).
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015847)
Florian Dold   
2020-05-04 13:22   
We don't really have this problem anymore, as web-common doesn't use any TypeScript anymore.

Furthermore, if we ever have source-level dependencies on the build output of some other repo, we should use a "prebuilt" branch in the git repo.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6224 [Taler] deployment and operations minor have not tried 2020-04-28 11:25 2020-07-24 11:56
Reporter: buckE Platform:  
Assigned To: buckE OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.7.1  
    Target Version: 0.8  
Summary: Consider Production Deployment of Weblate
Description: Problem:

Running weblate with built-in webserver requires running in Debug mode, which is not recommended. Details are here:
https://docs.djangoproject.com/en/3.0/ref/settings/#std:setting-DEBUG

Options:

The alternatives are

1 - Run Debug = False with no other modifications, which will cause CSS styles to fail

2 - Perform undocumented customizations on the built-in webserver or

3 - Deploy weblate in production environment as described here: https://docs.weblate.org/en/latest/admin/install.html#running-web-server

(Note: this is probably why port 8000 was not customizable)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: weblate.site (2,152 bytes) 2020-04-29 13:30
https://bugs.gnunet.org/file_download.php?file_id=888&type=bug
weblate.site-30.04.20 (2,042 bytes) 2020-04-30 08:19
https://bugs.gnunet.org/file_download.php?file_id=889&type=bug
Notes
(0015796)
Christian Grothoff   
2020-04-28 16:50   
(Last edited: 2020-04-28 16:50)
Sounds to me like I should just do (3) using

    listen 80;
    server_name weblate;
    root /usr/share/weblate;

    location ~ ^/favicon.ico$ {
        # DATA_DIR/static/favicon.ico
        alias /var/lib/weblate/static/favicon.ico;
        expires 30d;
    }

    location /static/ {
        # DATA_DIR/static/
        alias /var/lib/weblate/static/;
        expires 30d;
    }

    location /media/ {
        # DATA_DIR/media/
        alias /var/lib/weblate/media/;
        expires 30d;
    }

    location / {
        include uwsgi_params;
        # Needed for long running operations in admin interface
        uwsgi_read_timeout 3600;
        # Adjust based to uwsgi configuration:
        uwsgi_pass unix:///run/uwsgi/app/weblate/socket;
        # uwsgi_pass 127.0.0.1:8080;
    }
}

for the nginx configuration, using /home/weblate and /home/weblatetest for DATADIR respectively. Please confirm if you agree that this is what I should do.

(0015799)
buckE   
2020-04-29 10:02   
I confirm. Let's see if we're right.

I have disabled the weblate systemd service that starts weblate (ie - the site is now down)

I set DEBUG = False. So when you restart nginx, if weblate.taler.net site comes up and is not obviously style-less, this worked.

If we are running weblate this way, do you still want celery to run under weblate user? I am not sure if it matters, and maybe we try it how it is first? Anyway these are instructions if you prefer: https://docs.weblate.org/en/latest/admin/install.html#running-celery-as-system-service
(0015807)
Christian Grothoff   
2020-04-29 13:28   
We still need to run uwsgi:
$ uwsgi --home=/home/weblate/weblate-env --module weblate.wsgi:application -s /home/weblate/uwsgi.sock
(please setup a systemd user service for that).
Also, we need to make sure uwsgi.sock is group-accessible (660) as otherwise nginx can't read the socket. There is likely some other option needed to set the permissions (=> please investigate and update the systemd service file accordingly).

Finally, the static resources of django need to be generated, otherwise the site won't work. See:
https://docs.djangoproject.com/en/3.0/ref/contrib/staticfiles/#django-admin-collectstatic
Note that this command won't work until you've configured django-admin for weblate. Messy, I know.

I'm attaching the nginx config we are using.
(0015808)
buckE   
2020-04-30 07:44   
First:
I missed your mention of '/home/weblatetest' above. weblatetest has nothing to do with setting up weblate. weblatetest has nothing to do with this ticket. (That is a test project under weblate, not related to testing the installation of weblate.)

uwsgi:
  - user-level systemd running with --chmod-socket=66-
  - Result:
           - uwsgi running
           - srw-rw---- 1 weblate weblate 0 Apr 30 07:06 uwsgi.sock

The file is in /home/weblate/uswgi.sock (Will the socket still be found by nginx at uwsgi_pass unix:///run/uwsgi/app/weblate/socket ?)

Because we're staying with user-level config, I will send updates to the nginx config soon.
(0015809)
buckE   
2020-04-30 08:19   
Please use the attached file's settings instead. diff is probably easiest but:

```
server {
  listen 443 ssl;
  listen [::]:443 ssl; ## listen for ipv4; this line is default and implied
  # listen [::]:80 default_server ipv6only=on; ## listen for ipv6

  root /home/weblate/webroot;
.
.
.
location ~ ^/favicon.ico$ {
        # DATA_DIR/static/favicon.ico
        alias /home/weblate/DATA_DIR/static/favicon.ico;
        expires 30d;
    }

    location /static/ {
        # DATA_DIR/static/
        alias /home/weblate/DATA_DIR/static/;
        expires 30d;
    }

    location /media/ {
        # DATA_DIR/media/
        alias /home/weblate/DATA_DIR/media/;
        expires 30d;
    }
```
(0015813)
buckE   
2020-04-30 09:35   
PS I think django-admin is already configured and running `weblate configurestatic --noinput` creates the static files. But we'll see when the new settings are implemented.
(0015817)
Christian Grothoff   
2020-04-30 11:37   
New settings implemented, site works (at least static resources load nicely!).


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6060 [Taler] wallet (Android App) feature have not tried 2020-01-22 19:57 2020-07-24 11:56
Reporter: grote Platform:  
Assigned To: grote OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.7.1  
    Target Version: 0.8  
Summary: Render pending events on main screen and top of history
Description: In most cases pending operations turn into history events quickly.
Sometimes a pending operation indicates an error that requires attention from the user (i.e. when an operation failed because there is no internet connectivity)
Some pending operations require something to happen *outside* the control of our software (for example when we are waiting for a bank transfer between the user's bank account and the exchange)

The schema for pending operations is defined here:
https://git.taler.net/wallet-core.git/tree/src/types/pending.ts
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015294)
grote   
2020-01-22 19:59   
Florian Dold, how should we handle pending operations that will never finish? Is there a way for the user to cancel/remove them?
(0015295)
Florian Dold   
2020-01-22 20:47   
Every pending operation should have some "cancel" / "dismiss" action. What happens for this typically depends on the type of pending operation and would be implemented by wallet-core.

When implementing the rendering, you should just assume that every pending operation has some "pendingOperationId" (just like history events have unique identifiers), and the wallet offers some "dismissPendingOperation" or maybe "cancelPendingOperation" API. Some types of pending operations might also offer different "actions", but dismissing them should be implemented first.

For some types of pending operations the wallet would then automatically create a history event for the cancelled pending operation.
(0015297)
grote   
2020-01-23 17:00   
> When implementing the rendering, you should just assume that every pending operation has some "pendingOperationId" (just like history events have unique identifiers), and the wallet offers some "dismissPendingOperation" or maybe "cancelPendingOperation" API.

Do I need to assume their existence or are they already implemented somewhere, so I can actually use them? Or do you want me to just add a non-functional dismiss button everywhere.
(0015956)
grote   
2020-05-25 15:19   
The pending operations are now only shown in developer mode. For users they have been replaced with transactions (past and *pending*) which abstract away internal details and surface only information that ordinary users should be able to understand.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6438 [Taler] mechant backend minor have not tried 2020-07-24 11:37 2020-07-24 11:55
Reporter: MS Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.8  
Summary: assert+abort instead of reporting error
Description: The merchant backend ends with assert+abort instead of returning a descriptive
error response to the client.

This behavior has been observed while trying to PATCH a merchant instance using
a wrong currency.

Jul 24 11:15:12-294318 taler-merchant-httpd-32299(MP57Q1VREBRAVWJHTTTZNEXY5R) ERROR Assertion failed at amount.c:370. Aborting.
Jul 24 11:15:13-353462 taler-merchant-httpd-13047 INFO Starting taler-merchant-httpd
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016512)
MS   
2020-07-24 11:42   
This issue also suggests that upon instance *creation* there is no check on the currency.
(0016514)
Christian Grothoff   
2020-07-24 11:55   
Right, we should add such a check and 400 if the currency is 'bad'.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6222 [Taler] deployment and operations feature N/A 2020-04-26 01:15 2020-07-23 21:48
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Verify doxygen comments in CI
Description: A buildslave should be setup to run 'doxygen' on the exchange.git, merchant.git and taler-mdb.git (whenever they change) to collect Doxygen warnings. A select few (graph too big) can be ignored, in all other cases the buildmaster should send an e-mail to the last committer with the warnings (or a link to the CI job output with the warnings).

A new buildslave account (with SSH access for Buck) should be created by root@taler.net for this once you (Buck) are ready to take this on.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016324)
Christian Grothoff   
2020-06-19 12:50   
1) There is (right now) not a dedicated buildslave for exchange/merchant/taler-mdb, and there will never be one for taler-mdb.

2) The 'verification' feature is again simply to check for warnings that doxygen outputs. Same as with Sphinx. Using doxygen: nothing to it, just to exchange.git/doc/doxygen/ and run 'make fast' (after apt install doxygen).

3) The idea was to setup one buildslave (process, user-ID), but have multiple factories and different triggers -- no need to re-build exchange's doxygen if only taler-mdb changed.

doxygen@taler.net was created.
(0016371)
buckE   
2020-06-26 11:33   
Note: this is on hold until logs and other BB questions are answered for sphinx and can be used here too.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5642 [Taler] documentation text have not tried 2019-03-12 13:17 2020-07-23 21:47
Reporter: Marcello Stanisci Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Improve section about the exchange bank account setup.
Description: > For _me_ it was crystal-clear, but maybe we should switch to a more
> didactical style
> for this section where Nuria is stuck?

I think the first problem is that the example given does not work.
The payto://sepa/ is not implemented, we should instead give a
payto://x-taler-bank/ here (or no payto and leave it for later).

Also nothing says how you get the "right" bank account number. In
section 3.6.1, the bank account number is MISSING in the
payto://-example provided.

I think it would be good to add example commands ($ taler-config ...) to
run, to refer to the Taler bank (bank.git) documentation for 3.6.1, and
to give some way to _check_ that everything is correct that doesn't
involve starting the exchange. For example, running

$ taler-exchange-wire && echo everything OK

and ensuring that taler-exchange-wire does NOT return 0 if the
configuration has mistakes in it.


Tags:
Steps To Reproduce:
Additional Information: The description comes from a private discussion that was triggered by some users trying to install the exchange,
and it refers to the exchange manual.
Attached Files:
Notes
(0014679)
nikita   
2019-07-15 18:51   
ack, will get to it after GSoC.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6394 [libeufin] sandbox minor have not tried 2020-06-18 17:05 2020-07-23 13:41
Reporter: Florian Dold Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: figure out how banks in Germany de-duplicate payment initiations and use the same logic in the sandbox
Description: Currently the sandbox doesn't do any de-duplication of pain requests.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016305)
Florian Dold   
2020-06-18 21:53   
GLS uses (messageId, paymentInformationId) as the key for de-duplication.

Now we just need to implement this in the sandbox, too.
(0016499)
MS   
2020-07-22 21:15   
Implemented and tested here: 3c295e0d1d6bbee577caf2b4d9ce4e71a03fc82c
(0016500)
MS   
2020-07-22 21:34   
(Last edited: 2020-07-22 21:34)
Reopening as the de-duplication is only enforced at the database level, and a nice response that informs the requester is still needed.
Right now there is the catch-all handler that still responds with 500 Internal server error.

(0016506)
MS   
2020-07-23 13:41   
Resolved here: c716d23073be8846d0765fa766de341dbc953a88.

When a duplicate payment gets submitted, the sandbox responds with EBICS_PROCESSING_ERROR,
in a signed XML response.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6180 [Taler] deployment and operations feature N/A 2020-04-14 21:46 2020-07-22 23:39
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: setup codespell as part of the CI for all of our source projects
Description: See https://github.com/codespell-project/codespell
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0015672)
buckE   
2020-04-17 09:01   
I actually use codespell in my editor (Atom) and it's very useful.

This spec is not detailed enough to start. It's kind of a note, not a spec.

" setup codespell as part of the CI for all of our source projects"

As *what* part. Set up *to do what* exactly? How will I know it is done? What tests will you run to verify task completion?

Also, what is "the CI" and how do I access it?
(0015674)
Christian Grothoff   
2020-04-17 09:49   
Yes, this is a note, not a spec. The idea was to provide an idea about next steps, not a full spec.
But I can give you more details:

The CI is buildbot. You access it via deployment.git, where you can edit the buildbot master configuration. We can probably re-use an existing slave for _this_ task, but could also deploy additional slave accounts for this or other tasks in the future.

Setup means to run codespell against the source repositories whenever they change and generate a report. The report could be a Web site (say codespell.taler.net) or sending an e-mail to the last committer in case spelling errors were found (I'd prefer this option).

The task is complete if developers get a notification whenever they commit a spelling mistake into the code (basically against any git.taler.net repository with code in a language that codespell supports).
(0016321)
buckE   
2020-06-19 09:19   
Okay this seems simple enough now. I take this as the defliverable:

> The task is complete if developers get a notification whenever they commit a spelling mistake into the code (basically against any git.taler.net repository with code in a language that
> codespell supports).

QUESTIONS:

- Do you want this to run on *every* codebase?

- What about the ones that do not currently have a buildbot worker?

- Because this will take time to install on each codebase, please specify the highest-priority ones to start with. And above that, a low-impact one that I can use to test. :)

- Do you know a buildbot method to create a meta worker? ie - something that can be called from any shell account with a worker? I don't think this exists and I don't see it in the docs, but I might as well ask before I go another way. Otherwise I guess it should be easy to write a generic "codespell.sh" script, and run codespell.sh from each other script called by each worker, since those scripts already have access to the deployment/buildbot directory.

- Can you install codespell on taler.net? This is preferable to installing locally on "every" shell account that runs a buildbot worker on a codebase. However, it is not available for stretch: https://packages.debian.org/search?keywords=codespell&searchon=names&suite=stable&section=all Maybe a global pip install: https://github.com/codespell-project/codespell
(0016325)
Christian Grothoff   
2020-06-19 12:54   
Yes, every codebase on git.taler.net, and likely eventually also git.gnunet.org (as a service to GNUnet).

This should be a separate buildslave (process/user-ID) that only runs the spell checker. It has nothing to do with other jobs, and again may be run on code bases for which we have no other jobs.

For each codebase, the main work is to write the trigger (if git changes, do run spellchecker) and a factory (checkout code, run spellchecker, report). Overall, I would _hope_ you can write a python function "setup_spellchecker(ARG)" which takes as ARG the URL of a git repo and then that function creates the trigger and the factory rules for that Git repo. Maybe we need additional args to specify the programming language, but overall I would hope for a "write once, use for any repo"-style implementation.

Most important repos: exchange, merchant, wallet*.
(0016326)
Christian Grothoff   
2020-06-19 12:55   
(Last edited: 2020-06-19 12:56)
I've setup a user codespell@taler.net to run one buildslave for all of the buildbot jobs (for _all_ repos).

(0016370)
buckE   
2020-06-26 11:32   
Note: this is on hold until logs and other BB questions are answered for sphinx and can be used here too.
(0016505)
Christian Grothoff   
2020-07-22 23:39   
Worker is now running, output at:
https://buildbot.taler.net/#/builders/19/builds/3/steps/7/logs/stdio


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6365 [Taler] merchant backend API (HTTP specification) feature have not tried 2020-06-04 17:45 2020-07-22 22:03
Reporter: Florian Dold Platform:  
Assigned To: jonathanbuchanan OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: allow "forgettable" fields in contract terms
Description: With some fields in the contract terms, the merchant doesn't really have any good reason to store them long term.
For example, it can be reasonably expected that the merchant deletes the delivery address after the delivery happened.

However, when the delivery address is in the contract terms, deleting this part would change the hash of the contract terms.

Thus, some fields of the contract terms should be "forgettable", in the sense that only the hash counts, and the merchant can delete the plain value after it is not needed anymore.

E-mail citation for a possible generic solution below.

-----------------

Would make sense to have parts of the contract terms that are
"forgettable". It makes computing the contract terms hash harder
though, so we should agree on some sensible, generic rules where we
don't have to change to much to make more parts forgettable!

I'd propose the following scheme that makes introducing this rather cheap:

{
  delivery_location: {
    country: "Switzerland",
    city: "Zurich",
    street: "Foostreet",
  },
  _forgettable: ["delivery_location"],
}

and

{
  delivery_location: null,
  _forgotten: {
    delivery_location: <hash of forgotten delivery_location>
  }
}

have *exactly* the same hash code. At the same time, the
"forgettability" is completely transparent to applications that don't
care about it!
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016463)
Christian Grothoff   
2020-07-16 00:53   
We need a salt to make brute-forcing forgettable parts impossible. I propose that the salt is included in the forgettable attribute, thus:


{
  delivery_location: {
    country: "Switzerland",
    city: "Zurich",
    street: "Foostreet",
  },
  _forgettable: { "delivery_location" : "SALTVALUE" } ,
}
(0016464)
Christian Grothoff   
2020-07-16 01:58   
Draft implementation done in 90e756dd..e1ad498b, needs testing.
(0016466)
Christian Grothoff   
2020-07-16 14:52   
Implemented in 5658c4c5..c326a5bd. Needs wallet support and a merchant API.

For the merchant API, I'm thinking of an endpoint
PATCH /private/order/$ORDER_ID/forget
where the uploaded JSON includes an array of paths to fields to forget. Those must have already been forgettable (otherwise 409 conflict). We need paths as we may want to forget some attributed deeply nested inside of an object or array inside of the contract. For the path specification, I think the 'jq' command line tool includes a good design for the syntax.
(0016467)
Christian Grothoff   
2020-07-16 14:53   
Jonathan: please draft a specification for such an endpoint (docs.git), and then after review by Florian or myself, please implement it. This has a low priority at this point, so feel free to pick it up when it fits your mood.
(0016502)
jonathanbuchanan   
2020-07-22 21:40   
Implemented as of 68978c9..0314d6d.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6437 [Taler] merchant backend API (C) major have not tried 2020-07-22 16:47 2020-07-22 21:54
Reporter: Florian Dold Platform:  
Assigned To: Christian Grothoff OS:  
Priority: high OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: errors in database initialization files are not reported
Description: When there is an error in, say, merchant-0001.sql, the database initialization does not fail. Only later when the merchant tries to actually access the DB, errors are raised because tables were not created.

Instead, the code should already check for errors when executing merchant-0001.sql.

The logs currently make it look like the changes from the sql file were executed successfully (see "Steps to Reproduce" for details).
Tags:
Steps To Reproduce: Do something in merchant-0001.sql that causes the transaction to abort.

The logs will contain something like this:

Jul 22 20:04:28-970467 taler-merchant-dbinit-367481 INFO Applying SQL file `/home/dold/local/share/taler//sql/merchant/merchant-0001.sql' on database postgres:///talercheck
Jul 22 20:04:29-145005 pq-367481 DEBUG Preparing SQL statement `COMMIT' as `end_transaction'

However, running the script manually with "psql -f ..." will show that the transaction was rolled back.
Additional Information:
Attached Files:
Notes
(0016503)
Christian Grothoff   
2020-07-22 21:54   
Fixed in 422e7a031..8703a0516 (gnunet.git). psql needs the "--set ON_ERROR_STOP=1" argument to actually return with a failure in case of syntax errors in the SQL.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5945 [Taler] Web site(s) minor have not tried 2019-10-22 19:17 2020-07-22 21:37
Reporter: nikita Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: confirmed Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: buildbot: add job for demo.taler.net website
Description: the demo pages seem to have no automatic rebuild, elements I removed are still present.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016317)
buckE   
2020-06-19 08:33   
What are the "demo pages" ? Please provide details including:

- a link to the live URL
- git repo name
- buildbot function name, if you know it
- shell account user for the BB worker, if you know it


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6132 [Taler] other minor have not tried 2020-03-21 16:30 2020-07-22 17:38
Reporter: Florian Dold Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: merge shop, donations and survey
Description: These packages contain demo applications for Taler. It is frankly quite frustrating to maintain the same boilerplate files for each of them. There is no reason at all to keep them in separate repositories, they should be merged into one.

Possible name: taler-merchant-demos?
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016496)
MS   
2020-07-22 17:35   
taler-merchant-demos.git exists now.

Notice that the usual web-common submodule is not checked anymore because it was last updated 3 years ago!
Instead the common CSS files, for example, are within a top-level folder that gets symlinked whenever a shop site
needs it.
(0016497)
MS   
2020-07-22 17:38   
Common functions were also moved to a shared module, although an additional review to delete leftovers from the code is still needed.
(0016498)
MS   
2020-07-22 17:38   
Also important: at this stage, the new repository is NOT installed by the deployment script!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5916 [GNUnet] webpage minor have not tried 2019-10-02 14:01 2020-07-22 16:00
Reporter: nikita Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: redirect script
Description: write a script to automate the creation of static redirect pages
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6436 [Taler] documentation minor have not tried 2020-07-22 15:18 2020-07-22 15:18
Reporter: Florian Dold Platform:  
Assigned To: Stefan OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: document / figure out what markup works inside code blocks ; fix docs errors
Description: See summary.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6199 [Taler] exchange trivial have not tried 2020-04-22 13:30 2020-07-21 21:33
Reporter: fefe Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 0.7.0  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: loading the terms of service files into memory seems wasteful
Description: This is about load_terms in exchange/src/mhd/mhd_legal.c.

Basically, the function loads a bunch of files into memory. Does a stat to find the length, then mallocs that much (without bounds!) and reads the file contents into that buffer.

These files will usually not be very large so it probably does not matter much. However, I would suggest using mmap instead of malloc+read. In both cases you get a pointer, but it puts less strain on the OS and the memory usage of your application.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015730)
Christian Grothoff   
2020-04-22 15:07   
If we REALLY cared about performance, we could even use MHD's sendfile() here, but then we'd burn an fd per ToS/PP-file.
(0015738)
Christian Grothoff   
2020-04-22 22:48   
Moreover, if we REALLY care about performance, we should probably keep the compressed response in RAM instead of re-doing the compression per request. That'll be worse for RAM, but almost certainly beats re-doing gzip all the time.
(0016494)
Christian Grothoff   
2020-07-21 21:33   
Implemented in d9e871b5..6d52922c


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6088 [libeufin] sandbox minor have not tried 2020-02-11 15:43 2020-07-21 12:55
Reporter: Florian Dold Platform:  
Assigned To: Marcello Stanisci OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: generate c52/c53 responses correctly in the sandbox
Description: Currently we just return one XML document.

Instead, we must return a zip-container of XML files.

Each of the XML documents must be "immutable" after it has been created, and it must have a file name that is correct according to the German ZKA rules.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015902)
Marcello Stanisci   
2020-05-15 18:25   
The C53 response from the Sandbox is now valid (it was successfully tested with the GLS client).
The C52 *should* also be valid, as the difference with C53 is only one element name (but not the content thereof).

As for the other requests of this bug (immutability of filenames and their ZKA-compatibility), I would say
they go a bit too far from what the sandbox is today used for, and therefore I would consider this bug closed.
(0016493)
MS   
2020-07-21 12:55   
No real need to keep this open. 0006269 applies more now.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6428 [libeufin] sandbox minor have not tried 2020-07-09 20:41 2020-07-20 18:51
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Parse pain.001 the way nexus does with camt.
Description: Sandbox is still using XPath queries to pick values from pain.001 documents.
It should instead use the more structured API (XmlElementDestructor, ..) that Nexus is currently using to parse CAMT messages.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016491)
MS   
2020-07-20 18:51   
Addressed here: ede82f0331fae4bfdd9464f8626ca004fb15aa66


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6126 [Taler] other text have not tried 2020-03-13 13:44 2020-07-18 08:06
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: consider criteria for reasonable fee structures and deriving summarizing metrics
Description: This could both be beneficial for customers and to exchange operators. It doesn't really make sense for any exchange to define a "EUR:10" coin, a "EUR:0.001" coin and nothing in between!

Some snipped from an email discussion related to displaying fees to the user in the wallet:

-----
What's also still missing is some way to view the fee structure of the
chosen exchange for deposits / refreshes. Though I have no good idea
how to represent this data in a user-friendly way. Showing the actual
fee structures might be useful for developers, but IMHO no actual user
would bother to look at them, nor even know how to interpret them. It's
also very hard to break it down into some more intuitive metrics such as
percentages, as it wildly varies based on what part of the fees the
merchant covers.

(This should be a separate discussion, but I'm writing it down here so I
don't forget about it )

One (imperfect) way to break it down would be with the following two
numbers:

 1. What is the "range" of money I typically would transact? This
depends on the largest and smallest denominations offered.

 2. How much does it cost you to keep money in your wallet? This would
be computed from the validity period of denominations and refresh fees.

 3. What's are the average total fees for payments (in percentage), and
what percentage of these fees can a merchant cover? (Merchants can
cover wire and deposit fees, not so much refresh fees).

So some output would be: "With the exchange exchange-eur1.taler.net,
typical transactions are between 0.001 Eur and 100 Eur. Fees for
typical transactions range from 0.1%-0.3%. Merchants can cover 90% of
these transaction fees. During one month, keeping digital cash with this
exchange costs you 0.1% on average.

Now it's very easy to argue "These numbers are useless! It depends too
much on the amounts, since we have this discrete coin structure". But a
good counterpoint to this would be that if it varies too much, the fee
structure is broken!

So maybe we can actually come up with a reasonable way to compare fees
under certain constraints, and then for exchanges that don't follow
these constraints, the wallet declares defeat and tells the user
"Warning: This exchange has a fee structure that is difficult to predict".
-----
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6188 [Taler] exchange tweak N/A 2020-04-17 19:25 2020-07-18 00:53
Reporter: oec Platform:  
Assigned To: Christian Grothoff OS:  
Priority: none OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Lift binary arguments into function names
Description: Compare

```
get_coin_transactions(foo, bar, buz, GNUNET_OK, buf);
get_coin_transactions(foo, bar, buz, GNUNET_NO, buf);
```

against

```
get_coin_transactions_including_recoup(foo, bur, buz, buf);
get_coin_transactions_without_recoup(foo, bur, buz, buf);
```

Advantages of the later:

- function names become more descriptive
- for developers: less likely to pick the wrong function than to use the wrong boolean argument
- for code auditors: easier to reason about

Non-exhaustive list of APIs, that would benefit from this:

- `get_coin_transactions`, i.e. `postgres_get_coin_transactions`
- `decode_keys_json`
- `TALER_EXCHANGE_check_keys_current`


PS:

Also, of course a lot of APIs in gnunet would benefit from this:

- `GNUNET_OS_start_process`
- `GNUNET_CURL_job_add`
- `GNUNET_DISK_pipe`
- `GNUNET_ATS_AddressInformationCallback`
- `GNUNET_ATS_PeerInfo_Iterator`
- `GNUNET_ATS_performance_list_addresses`
- ...
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015684)
Christian Grothoff   
2020-04-17 19:46   
Ok, sounds reasonable, assuming the two "new" functions internally call the old one unless they don't share much code.
(0015685)
Christian Grothoff   
2020-04-17 19:46   
If you find other candidates, please add as notes to this bugreport.
(0015709)
oec   
2020-04-20 18:20   
Also:
- postgres_have_deposit , have_deposit
- TALER_EXCHANGEDB_DepositCallback
- TALER_EXCHANGEDB_WirePreparationCallback
(0016470)
Christian Grothoff   
2020-07-16 18:17   
Looking at the first details, I'm not so convinced:
- `get_coin_transactions`, i.e. `postgres_get_coin_transactions`
  --- that's in the Postgres DB API. Any change here means that all plugins will have to
       provide both functions, we have to declare both functions in the header, initialize
       them in the plugin struct, etc. Very messy, for very limited gain (the function
       is not used all that often).
- `decode_keys_json` --- that's not even in an API, that's a purely internal function
      inside a file, and only called twice. Also the two code paths are inherently very
      mangled, so it is not like it would make sense to really have two functions.
- `TALER_EXCHANGE_check_keys_current` --- here I would worry about the
     combinatorial explosion, as it takes two boolean arguments. I think here
     a good solution would be to replace the two booleans with an enum, that way
     the caller will pass readable flags, and we get down from three args to two.
(0016476)
Christian Grothoff   
2020-07-16 20:34   
c326a5bd..b9f1384b improvesthe check_keys_current API, introducing an enum.
(0016477)
Christian Grothoff   
2020-07-16 20:43   
b9f1384b..4fde7604 updates GNUNET_CURL_job_add() to avoid boolean arg.
(0016478)
Christian Grothoff   
2020-07-16 21:08   
Todo:
- `GNUNET_OS_start_process` - use enum?
- `GNUNET_DISK_pipe` - use enum?

ATS APIs: dead with TNG anyway, no point.
(0016486)
Christian Grothoff   
2020-07-17 22:43   
GNUNET_OS_start_process() API fixed in c7e00e63..62963ae4 (exchange).
(0016487)
Christian Grothoff   
2020-07-18 00:53   
80ba1c6ebe03021dc464d4c3273607d1fa990de5 (exchange.git) updates the GNUNET_DISK_pipe() API. While there may be others left that in principle have this problem, I'm resolving the bug as the ones that where specifically mentioned have been addressed or at least reviewed. If somebody finds others, a new bug should be opened.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6433 [GNUnet] TCP transport feature N/A 2020-07-17 12:05 2020-07-17 12:07
Reporter: t3sserakt Platform:  
Assigned To: t3sserakt OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.14.0  
Summary: Writing a test to simulate an attack on the tcp communicator KX to test the replay protection.
Description: There is a replay protection for the tcp communicator implemented using a nonce send together with the ephemeral key. If the nonce is not send back the queue will be destroyed. To test this there has to be a peer not sending back the nonce within the defined limits. One possible way to do this is to build a variant of the tcp communicator to be used for the second peer in the communicator test. This second build changes some parts of the communicator code to send a wrong challenge or none at all.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5710 [GNUnet] transport service feature N/A 2019-05-02 14:28 2020-07-17 12:07
Reporter: Christian Grothoff Platform: i7  
Assigned To: OS: Debian GNU/Linux  
Priority: urgent OS Version: squeeze  
Status: confirmed Product Version: 0.11.0  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: TNG meta issue
Description: This is the transport "next generation" (TNG) meta issue. Bugs in TRANSPORT that we expect to be fixed by the TNG re-design should be *related* to this bug. Issues that are to be addressed so we can call TNG "done" should be come *child* issues of this bug.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
4734 [Taler] wallet (WebExtensions) feature have not tried 2016-10-17 16:07 2020-07-16 23:00
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: low OS Version:  
Status: acknowledged Product Version: 0.9  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: display what the new balance will be when confirming contracts/reserves
Description: Sometimes it's a bit hard to see how fees are add up, especially with refresh fees after a purchase.

Thus the UI should always display what the balance will be after we confirm an action that changes it.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016474)
Florian Dold   
2020-07-16 20:01   
I'm really not sure:

* what we would show there with multiple currencies *and* different auditors / exchanges for the same currency
* whether this feature improves anything and why this is necessary

(None of the banks or payment apps that I know show this either!)

This feature complicates the logic for little gain, since the UI would also have to update this balance if it increases or decreases in the background.

I would put this as "improvements for financial literacy" post-1.0.
(0016482)
Christian Grothoff   
2020-07-16 22:59   
Sure, this is UX stuff that should definitively be discussed more, if possible with Belen.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6432 [Taler] merchant backend API (C) tweak have not tried 2020-07-16 20:15 2020-07-16 21:10
Reporter: Florian Dold Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: consider making API error responses more developer friendly
Description: Some endpoints of the merchant now seem to *only* return the numeric "code", which makes it really tedious for developers who want to integrate with the merchant (or even configure it!).

For example, if I mistype some endpoint name I get
{
  "code": 10
}
which is really unhelpful.

We already generate a header with the error codes, we should consider adding the alphanumeric name for the error as "non-normative" extra information.

By now (younger than Taler ;-) ), there is also an RFC that defines the format for the "application/problem+json" MIME type: https://tools.ietf.org/html/rfc7807. Using this might be a bit too much, as all errors are required to have a canonical URL for the error type, such as "https://taler.net/errors#DEPOSIT_DENOMINATION_KEY_UNKNOWN".

However, just adding the alphanumeric name for the error code to the error JSON would already go a long way!
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5213 [Taler] wallet (WebExtensions) feature have not tried 2017-12-11 19:45 2020-07-16 20:02
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: feedback Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.7.1  
Summary: wallet's "tree view" should be turned into a human-friendly, detailed balance sheet
Description: ... including last seen error messages for the case where some amount is pending.

Currently the tree view is only useful for developers (if at all), and the balance overview has too little detail and no explanations. We'd have to go to the background page to see what errors happened if an amount stays "pending".
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016475)
Florian Dold   
2020-07-16 20:02   
This seems too much like a "developer centric" view, and is obsoleted by the current design of the transaction list, balance overview and global notifications.

Instead, the web extension wallet should just have the same balance view and transaction history as the Android wallet.

There are many things that *could* be done there, but that is post-1.0 gold plating on the "financial literacy" aspect.

IMHO this one should be resolved as "won't fix", WDYT?


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6418 [Taler] merchant backend API (C) feature always 2020-06-26 20:11 2020-07-16 17:55
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: enable transmission of client authentication data in taler-merchant-setup-reserve
Description: A Taler merchant backend may require authentication data, especially in 'secure' deployments with a reverse proxy. We thus need to add support for client authentication (http basic, http digest, TLS client certificates) to the C API and the CLI tool.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016468)
Christian Grothoff   
2020-07-16 17:48   
Implemented in 99bf3..dabc4c25
(0016469)
Christian Grothoff   
2020-07-16 17:55   
Options documented in 319b517..4d318f9


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6011 [GNUnet] UNIX transport minor always 2019-12-25 14:03 2020-07-16 16:21
Reporter: schanzen Platform:  
Assigned To: t3sserakt OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.14.0  
Summary: [TNG] UNIX communicator performance issues
Description: A trained eye might notice the absolutely unacceptable, abysmal performance of this
communicator considering it is using UDS (we should expect ~XGB/s!).
As it is not meant to be used productively (and can't really) this is not a blocker, but
still something worth investigating. I do not know if this was also the case for the old
plugin
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015222)
schanzen   
2019-12-26 06:20   
This is also an issue for the TCP communicator. Maybe this is a general issue with MQ or TNG api.
(0015223)
schanzen   
2019-12-26 20:54   
We now also have a UDP communicator which does no flow control (additional messages with (mock)transport).
It is now likely clear that something is wrong with the MQ (in general?).
(0015347)
schanzen   
2020-02-11 17:51   
For reference, current numbers:

* Size packet test done.
* 12798/12798 packets -- avg latency: 3148434 us
* Short size packet test done.
* 5000/5000 packets in 838528 us (745 KiB/s) -- avg latency: 409401 us
* Long size packet test done.
* 5000/5000 packets in 1717230 us (88 MiB/s) -- avg latency: 1034307 us
(0015377)
schanzen   
2020-02-18 17:13   
(Last edited: 2020-02-18 17:14)
In HEAD performance is now increased drastically (but still not optimal).

Unix: Short ~5MB/s Long: ~300MB/s
TCP: Short ~4MB/s Long: ~200MB/s

UDP is currently not functional. The packet send logic is very inefficient and the test always fails.

(0015378)
schanzen   
2020-02-18 17:15   
Note that the issue might actually be test design and lack of FC and not so much the communicator itself.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5865 [Taler] merchant frontend (blog) minor have not tried 2019-08-31 11:13 2020-07-16 15:22
Reporter: Florian Dold Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: refund should only be offered when it is actually possible to refund
Description: Currently the frontend doesn't check if refund is actually still possible, or if the refund deadline for the article expired.

In /check-payment, the backend should probably tell the frontend with an easy flag if refund is still possible.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6175 [Taler] exchange feature N/A 2020-04-11 22:07 2020-07-16 15:19
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9  
Summary: implement privilege separation for access to online signing keys
Description: We should not allow the main exchange HTTP process direct access to the exchange's signing keys (RSA or EdDSA).
Instead, those keys should be kept internal to another process running under a different UID. The HTTPD should then
use IPC (per-thread UNIX DGRAM connection is the preferred right now) to send signing requests to the helper process.
The helper process setup must ensure that the UNIX socket is ONLY accessible to the HTTPD.

This will prevent the private keys from being fully disclosed to an adversary if they were able to gain RCE in the HTTPD process. Naturally, they can still use the helper as a signing oracle, but the damage will still be a bit more limited.

Also, this should facilitate transitioning to an HSM in the future.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6161 [Taler] exchange feature N/A 2020-04-06 13:19 2020-07-16 15:18
Reporter: fefe Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: assigned Product Version: 0.7.0  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9  
Summary: Suggestion: Do some more signature checks
Description: This is for exchange/src/auditor/taler-auditor-httpd_deposit-confirmation.c

 80 now = GNUNET_TIME_absolute_get ();
 81 if ( (es->ep_start.abs_value_us > now.abs_value_us) ||
 82 (es->ep_expire.abs_value_us < now.abs_value_us) )
 83 {
 84 /* Signing key expired */
 85 TALER_LOG_WARNING ("Expired exchange signing key\n");
 86 return TALER_MHD_reply_with_error (connection,
 87 MHD_HTTP_FORBIDDEN,
 88 TALER_EC_DEPOSIT_CONFIRMATION_SIGNATURE_INVALID,
 89 "master_sig (expired)");
 90 }

This looks good and is technically sufficient, but maybe we could add some additional sanity checks?
For example that the signature was not valid for longer than a year?

Maybe add a way to revoke the key?
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015535)
Christian Grothoff   
2020-04-06 13:29   
Validity lifetime: the specification doesn't set a limit on signing key lifetimes so far, hence the auditor cannot check for one. We can consider to impose an upper bound (1 year seems reasonable), but then this needs to be implemented/enforced also on the exchange side.

Revocation: I agree we probably would want to check for revocations here. That said, I think the entity that should sign off on these revocations actually probably is the auditor, and that is not implemented. Right now, 'revocation' of these keys is basically the exchange stops using them and uses something else -- usually after the auditor in the very code you are reading detected a key compromise. So yeah, I guess we should add a "revoke this key" tool/program to the auditor, and should henceforth ignore further reports involving the now revoked key. We'd also need to expand the exchange's /keys API to return the revocation signatures (possibly multiple per key, as we may have multiple auditors per exchange).


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6363 [Taler] exchange feature have not tried 2020-06-03 13:31 2020-07-16 15:16
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: confirmed Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: allow merchants to query more information about the deposit status
Description: Quoting from the e-mail discussion:

>> I'm not really sure what to do there in practice, as "Taler 1.0" doesn't
>> really have any way for the merchant to ask about the status of the
>> transfer other than trying to call the exchange.
>>
>
> Well, we do have:
>
> https://docs.taler.net/core/api-exchange.html#get--deposits-$H_WIRE-$MERCHANT_PUB-$H_CONTRACT_TERMS-$COIN_PUB
>
> We could easily add a HTTP "Failed Dependency" status code to the API
> here and define a message format to indicate details about the problem
> to the merchant (and prescribe resolution strategies).
>
> This would mostly require another taler-exchange-XXX process to _query_
> LibEuFin for R-transactions, *and* a change to the exchange DB schema.
> More messy: this will require additional changes to the merchant
> backend. Please file a bug on this on Mantis, but I think we should
> target Taler v1.5 for this, given the scope of the changes and need for
> testing -- and the hopefully reasonably low likelihood of occurrence. At
> least I think we should be fine to manually handle the first generation
> of complaints for R-transactions, and that might even be better than
> trying to automate a process where we may not fully understand all the
> ways these things fail in practice.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016388)
Christian Grothoff   
2020-07-06 11:00   
Some status information is now provided ('wired'), alas automatically querying LibEuFin for transactions as _merchant_ is missing: right now, the merchant has to manually POST incoming transactions to the backend.

Writing a taler-merchant-wirewatch tool should be done, but later.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6182 [Taler] documentation text have not tried 2020-04-15 11:11 2020-07-15 22:31
Reporter: oec Platform:  
Assigned To: Christian Grothoff OS:  
Priority: low OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Specifiy the details for FDH in GNU Taler
Description: GNU Taler requires Full Domain Hash (FDH) in the withdrawal, refresh and
deposit protocols of GNU Taler. AFAIK, the term FDH refers to a property or
requirement of the hashing+signing operation and does not define a particular
choice of hash functions and the actual algorithm to generate a value of
specific length.

GNU Taler uses a particular implementation of an FDH by eventually calling

        return GNUNET_CRYPTO_hkdf_v (result,
                                       out_len,
                                       GCRY_MD_SHA512,
                                       GCRY_MD_SHA256,
                                       xts,
                                       xts_len,
                                       skm,
                                       skm_len,
                                       argp);

I suggest to be specific about the FDH algorithm that GNU Taler uses and add it
to the protocol specification(s) and documentation.
Tags: documentation
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016462)
Christian Grothoff   
2020-07-15 22:31   
Documented in glorious detail in 9e92cc60..c6278cee


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6375 [Taler] exchange minor have not tried 2020-06-10 14:31 2020-07-15 21:47
Reporter: Florian Dold Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: methods like TALER_config_get_amount should report errors instead of the call site
Description: Right now the callees report errors, but can't distinguish between missing values and malformed values.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016461)
Christian Grothoff   
2020-07-15 21:47   
Fixed for the exchange in 8f0a4b60..9e92cc60


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6133 [Taler] auditor feature N/A 2020-03-22 21:22 2020-07-15 21:29
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: low OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: auditor helpers do not support CTRL-C
Description: Except for taler-helper-auditor-wire, the auditor helpers do not support stopping the audit on CTRL-C.

We might need to add this to have a (controlled) way to (prematurely) terminate the audit without having to loose all progress.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016454)
Christian Grothoff   
2020-07-15 14:14   
ef0eb9e5..5d4d5dca extends report lib with signal handling. Next: modify all DB iterations to 'stop' early if an abort signal has been received.
(0016460)
Christian Grothoff   
2020-07-15 21:28   
Implemented in 6cabb25d..8f0a4b60


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6166 [Taler] merchant backend API (HTTP specification) feature N/A 2020-04-07 19:12 2020-07-15 20:48
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: RESTify merchant backend API
Description: Similar to the changes done in 0.7.0 for the exchange, we should RESTify the merchant API.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016458)
Christian Grothoff   
2020-07-15 20:48   
New API exists in master now.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5351 [Taler] codeless payments feature have not tried 2018-06-14 17:35 2020-07-15 20:45
Reporter: Florian Dold Platform:  
Assigned To: Christian Grothoff OS:  
Priority: low OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: design and implement API for notifications and inventory tracking
Description: Advanced sellers might want to have API access to their inventory, and the possibility to get notified via some API on various events.

There are various options and design choices for this, and they should be discussed and documented before going ahead with an implementation.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015639)
Christian Grothoff   
2020-04-13 21:31   
4e05ec8..01bb5c7 adds an endpoint /products/ to the merchant API specification for inventory management. Please comment!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6429 [Taler] mechant backend minor always 2020-07-15 17:24 2020-07-15 20:23
Reporter: MS Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Malformed payto value causes abort.
Description: The merchant aborts when a malformed "payto" value is found in the body of a new POSTed instance.
The error should be managed without aborting the service.
Tags:
Steps To Reproduce: Launch the merchant, and request the following resource:

POST $merchant/private/instances

{
  "payto_uris": ["payto://wrong"]
   .. other fields as indicated here: file:///home/consulting/sources/docs/_build/html/core/api-merchant.html#post--private-instances
}
Additional Information:
Attached Files:
Notes
(0016457)
Christian Grothoff   
2020-07-15 20:23   
Fixed in 550ba62a..199c806e


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6373 [Taler] exchange tweak always 2020-06-10 13:45 2020-07-15 14:32
Reporter: oec Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 0.7.1  
Product Build: Resolution: won't fix  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Simplify check_for_denomination_key by making denomination_key_lookup_by_hash also return the DenominationKeyUse
Description: The current signature of TEH_KS_denomination_key_lookup_by_hash expects a TEH_KS_DenominationKeyUse parameter.
It is called in the http-handlers for withdraw, deposit, recoup, refund, melt and reveal. Except for melt, the function is called once in each handler.

However, the logic in taler-exchange-httpd_melt suffers from multiple calls to ...lookup_by_hash with different values for the ...DenominationKeyUse in nested if-statements.

I believe it would be easier to reason about this particular case when TEH_KS_denomination_key_lookup_by_hash would be called with a pointer to ...DenominationKeyUse that will be set to the correct value according to in which map it was found (if at all). Something along the lines of:

enum TEH_KS_DenominationKeyUse use;

dki = TEH_KS_denomination_key_lookup_by_hash(
  key_state,
  &rmc->refresh_session.coin.denom_pub_hash,
  &use;
  &ec,
  &hc);

if (NULL = dki) {
  // free resources;
  // return with error
}

switch(use)
{
   case TEH_KS_DKU_DEPOSIT: ...
   case TEH_KS_DKU_RECOUP: ...
   case TEH_KS_DKU_ZOMBIE: ...
   default: ...
}

IMHO, this would simplify the logic and increase readability a lot

For refactoring the other handlers, one could introduce a new function and rename the existing one to TEH_KS_denomination_key_lookup_by_hash_with_use() or so.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016455)
Christian Grothoff   
2020-07-15 14:31   
The problem with your proposal is that we cannot simply return a 'use' for a denomination key: a denomination key can be valid for deposit and withdraw (at the same time!), or just deposit, or for recoup and zombie (at the same time), or just for zombie but not recoup. Also, as you observed, we do the lookup in different data structures, depending on the use.

Thus, if we were to return a 'use' as you propose, a single enum value would have to represent a combination of uses. Messy. So for me, your proposal would not actually simplify the code.
(0016456)
Christian Grothoff   
2020-07-15 14:32   
I've looked over the code, and for now decided that this will not result in code that is definitively more readable. At best, it seems likely to shift complexity and at worst it may make the 'use' field more difficult to understand.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6114 [GNUnet] integration tests minor always 2020-03-06 10:05 2020-07-15 13:53
Reporter: tanguy Platform: x86_64  
Assigned To: OS: Guix System  
Priority: normal OS Version: 944b1d0  
Status: assigned Product Version: Git master  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.14.0  
Summary: Tests failing on Guix System
Description: To update GNUnet package from 0.11.8 to 0.12.2, I had to disable several tests. Those are:
  - src/cadet/Makefile
    * test_cadet_2_reopen
    * test_cadet_5_forward
    * test_cadet_5_signal
    * test_cadet_5_keepalive
    * test_cadet_5_speed
    * test_cadet_5_speed_ack
    * test_cadet_5_speed_reliable
    * test_cadet_5_speed_reliable_backwards
    * test_cadet_5_speed_backwards
    * test_cadet_5_reopen
  - src/transport/Makefile
    * test_transport_api_https
    * test_transport_blacklisting_multiple_plugins
  - src/testbed/Makefile
    * test_testbed_api_2peers_1controller
    * test_testbed_api_test
    * test_testbed_api_statistics
    * test_testbed_api_topology
    * test_testbed_api_topology_clique

I'm attaching the full log.

Some tests where already disabled for version 0.11.8:
  - src/transport/Makefile
    * test_transport_api_manipulation_cfg
    * test_transport_api_udp_nat
  - src/topology/Makefile
    * test_gnunet_daemon_topology
  - src/namestore/Makefile
    * am__append_2
  - src/gns/Makefile
    * am__append_4
  - contrib/Makefile
    * check_PROGRAMS.*

Those I didn't try to re-enable, so I don't have any log.
Tags:
Steps To Reproduce: Get latest Guix sources: https://guix.gnu.org/cookbook/en/guix-cookbook.html#Direct-checkout-hacking
Apply GNUnet 0.12.2 patch: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39673
Build: $ ./pre-inst-env guix build --keep-failed gnunet
Additional Information: my_gnunet_bugreport.log

I had to complete it manually because gnunet-bugreport does not really work well on Guix System.


    INFO: gnunet-bugreport 0.11.0
    INFO:
    INFO: Please submit the following
    INFO: information with your bug report:
    =========================================
    INFO: OS : Linux
    INFO: OS RELEASE : 5.4.18-gnu
    INFO: HARDWARE : x86_64
    INFO: awk : 5.0.1
    INFO: gcc :
    INFO: cc :
    INFO: c++ :
    INFO: clang :
    INFO: clang++ :
    WARNING: make : Not Found (unexpected result)
    INFO: gmake :
    INFO: automake : 1.16.1
    INFO: libtool : 2.4.6
    WARNING: libextractor : 1.9
    INFO: GNUnet 0.8 : Not Found (good)
    INFO: GNUnet : 0.12.2
    INFO: git commit :
    INFO: libgcrypt : 1.8.4
    INFO: mysql :
    INFO: pkg-config : 0.29.2
    INFO: glib2 : 2.60.6
    INFO: GTK2 : 2.24.32
    INFO: GTK3 : 3.24.13
    INFO: GTK4 :
    ERROR: GMP : 6.1.2
    ERROR: libunistring : 0.9.10
    INFO: GNU gettext : 0.20.1
    INFO: gettext : 0.20.1
    INFO: libcurl : 7.65.3
    INFO: libgnurl : 7.67.0
    INFO: libmicrohttpd : 0.9.70
    INFO: GNU GLPK : 4.65
    INFO: GnuTLS : 3.6.9
    =========================================
    INFO: Bug report saved in ./my_gnunet_bugreport.log
Attached Files: 4z2sv4v1a6h8092160sq61arwyy68r-gnunet-0.12.3.drv.bz2 (40,527 bytes) 2020-03-06 10:05
https://bugs.gnunet.org/file_download.php?file_id=876&type=bug
Notes
(0015778)
schanzen   
2020-04-24 10:32   
Is this patch applied yet?
(0015800)
nikita   
2020-04-29 11:59   
(Last edited: 2020-04-29 12:09)
What patch? I only see a build directory or what I assume is one (.drv.bz is a compressed log file (thanks Ricardo)) and the guix bugtracker holds no patch to fix this.
tanguy: It is very likely you run into a similar problem like Nix: https://github.com/NixOS/nixpkgs/issues/73793

(0015801)
nikita   
2020-04-29 12:06   
> I had to complete it manually because gnunet-bugreport does not really work well on Guix System.

Can we get a separate bugreport for this? gnunet-bugreport is a bit basic and I was meaning to abstract it more, but this takes time
(0015803)
tanguy   
2020-04-29 12:28   
Hi guys!

Thanks for following up on this! Unfortunately, I haven't done anything since my last note! Sorry!

I'll try to spend some time on it at the week end, and I'll open a separate report for gnunet-bugreport.
(0015806)
nikita   
2020-04-29 12:59   
Hi,

I think the resolution lies within the 2 bugreports I linked to this one, I'd advice to just skip the tests again for now.
(0015853)
tanguy   
2020-05-05 09:14   
Hi,

@schanzen, yes, my patch has been merged into guix master with all the failing tests deactivated. We now have a "working" gnunet 12.2 on Guix and I'm trying to validate (slowly, slowly) all the components to make sure it works.

@nikita, I reported the gnunet-bugreport problem (which looks like a guix system "problem") in 0006233. I'll read the 2 linked bugs and the Nix issue. Thanks for the links! So for the time being, I'll keep the failing test deactivated.
(0015862)
schanzen   
2020-05-06 17:41   
Seems to be a TNG related issue. Moving to 0.14
(0016439)
tanguy   
2020-07-10 15:41   
Hi @schanzen,

I've uprgaded Guix package to 0.13.0 (I'll send the patch for integration today) and I had to disable another test to make the tests pass:
* test_testbed_api_test_timeout

Kind regards
(0016440)
schanzen   
2020-07-10 16:07   
(Last edited: 2020-07-10 16:08)
Did you have to disable thsi test in addition or only this test this time? Di you have a log to look at?
Testbed is not critical, so I guess this is fine.

I am asking because there is a chance that the cadet tests now work.

(0016451)
tanguy   
2020-07-15 11:10   
Hi @schanzen

> Did you have to disable thsi test in addition or only this test this time?

In addition of all the tests mentioned above.

> Di you have a log to look at?

Yes…

```
FAIL: test_testbed_api_test_timeout
===================================

Jul 15 08:32:56-017816 test_testbed_api_test-19106 ERROR Assertion failed at test_testbed_api_test_timeout.c:98.
Jul 15 08:32:56-018321 ats-19248 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018322 ats-19274 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018321 ats-19236 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018345 ats-19247 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018940 ats-19271 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018943 ats-19198 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019071 ats-19228 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019124 ats-19227 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019137 ats-19194 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019456 ats-19312 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019508 ats-19221 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019651 ats-19230 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019992 ats-19181 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-021692 ats-19278 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-022023 util-client-19189 WARNING Error during sending message of type 344: Broken pipe
Jul 15 08:32:56-022196 ats-19255 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-022500 ats-19211 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-022795 ats-19205 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-023986 ats-19290 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-023998 ats-19294 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-024072 ats-19261 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-024544 transport-api-core-19213 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-024672 ats-19269 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-025049 util-client-19200 WARNING Error during sending message of type 344: Broken pipe
Jul 15 08:32:56-025494 transport-api-core-19219 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-025964 ats-19286 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-026422 ats-19324 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-026638 ats-19212 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-026765 ats-19316 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-027634 util-client-19241 WARNING Error during sending message of type 344: Broken pipe
Jul 15 08:32:56-028315 util-client-19265 WARNING Error during sending message of type 344: Broken pipe
Jul 15 08:32:56-031411 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031510 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031556 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031575 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031901 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031924 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031935 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031954 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032872 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032885 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032896 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032907 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032918 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032929 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032939 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032950 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032963 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
FAIL test_testbed_api_test_timeout (exit status: 1)
```

> I am asking because there is a chance that the cadet tests now work.

I'll try to run the cadet tests and I'll re-enable them in the next package update if they pass.
(0016452)
schanzen   
2020-07-15 11:24   
Interesting. If you have the time, it would be great if you could execute the failing tests by themselves (not in make check) so that we can see what is happening.
There should also be .log files for each test with potentially more info.
I currently do not have a guix system and on my platforms the tests pass atm so I cannot reproduce.
(0016453)
tanguy   
2020-07-15 13:53   
>> I am asking because there is a chance that the cadet tests now work.
> I'll try to run the cadet tests and I'll re-enable them in the next package update if they pass.

They pass ! I'll submit a patch to update the package definition.


> There should also be .log files for each test with potentially more info.

It looks the same to me:

```
cat /tmp/guix-build-gnunet-0.13.1.drv-0/gnunet-0.13.1/src/testbed/test_testbed_api_test_timeout.log
Jul 15 08:32:56-017816 test_testbed_api_test-19106 ERROR Assertion failed at test_testbed_api_test_timeout.c:98.
Jul 15 08:32:56-018321 ats-19248 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018322 ats-19274 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018321 ats-19236 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018345 ats-19247 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018940 ats-19271 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-018943 ats-19198 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019071 ats-19228 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019124 ats-19227 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019137 ats-19194 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019456 ats-19312 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019508 ats-19221 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019651 ats-19230 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-019992 ats-19181 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-021692 ats-19278 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-022023 util-client-19189 WARNING Error during sending message of type 344: Broken pipe
Jul 15 08:32:56-022196 ats-19255 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-022500 ats-19211 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-022795 ats-19205 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-023986 ats-19290 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-023998 ats-19294 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-024072 ats-19261 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-024544 transport-api-core-19213 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-024672 ats-19269 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-025049 util-client-19200 WARNING Error during sending message of type 344: Broken pipe
Jul 15 08:32:56-025494 transport-api-core-19219 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-025964 ats-19286 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-026422 ats-19324 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-026638 ats-19212 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-026765 ats-19316 ERROR Assertion failed at container_multipeermap.c:347. Aborting.
Jul 15 08:32:56-027634 util-client-19241 WARNING Error during sending message of type 344: Broken pipe
Jul 15 08:32:56-028315 util-client-19265 WARNING Error during sending message of type 344: Broken pipe
Jul 15 08:32:56-031411 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031510 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031556 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031575 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031901 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031924 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031935 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-031954 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032872 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032885 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032896 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032907 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032918 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032929 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032939 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032950 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
Jul 15 08:32:56-032963 transport-api-core-19121 ERROR Error receiving from transport service (1), disconnecting temporarily.
FAIL test_testbed_api_test_timeout (exit status: 1)
```

The failure might come from the Guix test environment, but I'm not knowledgeable enough to investigate. Sorry!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6414 [Taler] auditor text always 2020-06-24 13:45 2020-07-14 21:18
Reporter: oec Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: No checks for refunds table implemented
Description: There is no implementation for checks of the refunds table. (There are however some checks related to refunds in aggregation and coins)

I suggest to add a timestamp to the refunds table and add a check that a refund happened before a `refund_deadline` passed.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016352)
Christian Grothoff   
2020-06-24 13:49   
Adding such a timestamp would not be very meaningful, as either side can easily lie (or disagree) about it. I also think you are misunderstanding the 'refund_deadline' semantic. The 'refund_deadline' is NOT the last time when a refund may happen, but the EARLIEST time by which the exchange may wire the funds. Refunds are allowed until the time when the exchange wires the funds to the merchant.

The checks for refunds are included in the coin consistency check (if coins are refunded, their balance goes back up _allowing_ further refreshes against those coins), and of course in the aggregate wire transfer calculations (how much we should wire to the merchant).

So what _else_ would you want to check that is meaningful?
(0016363)
oec   
2020-06-25 10:59   
see my note on 6413, what about verifying that a signature for a refund comes from a merchant whose merchant_pub has been used in a deposit before?
(0016364)
Christian Grothoff   
2020-06-25 11:01   
It must be the "right" merchant_pub, not just seen in "a deposit before", but seen in THE deposit for that coin.
This IS checked in taler-helper-auditor-coins.c::refund_cb().
(0016449)
Christian Grothoff   
2020-07-14 21:17   
62d5aae1..ef0eb9e5 improves the 'system' manual, documenting for each auditor helper what checks it performs (at a moderate level of detail).
(0016450)
Christian Grothoff   
2020-07-14 21:18   
I believe the new documentation makes it obvious why the refunds table does not have its own helper.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6413 [Taler] auditor tweak always 2020-06-24 13:39 2020-07-14 21:16
Reporter: oec Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Not enough checks for deposits
Description: taler-helper-auditor-deposits.c implements only one check for the deposits: "deposit confirmation missing".

I suggest to add at least the following checks:
- compare `refund_deadline` and `wire_deadline` to timestamp
- `coin_pub` and `coin_sig` are sound
- `amount_with_fee_val` and `amount_with_fee_frac` are sound
- `merchant_pub` points to an existing merchant

In comparison, for aggregation, coins, wire and reserves there is an average of 20 distict checks implemented.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016351)
Christian Grothoff   
2020-06-24 13:45   
- refund_deadline and wire_deadline and timestamp are (almost) orthogonal. You can
  legitimately set them far in the future, and I think the only constraint that applies is
  that refund_deadline <= wire_deadline here. What exactly would you want to check
  that certainly holds?
- coin_pub and coin_sig: these are checked in taler-helper-auditor-coins. It seems
  strictly wasteful to check the same signature multiple times, and would also result
  in duplicate reports (which doesn't make them easier to understand).
- amount_with_fee*-soundness: not sure which check you want to see, but again
  I wonder if this is not checked elsewhere.
- merchant_pub: what do you mean by "existing merchant"? We don't have a
  merchant registry. So very unclear to me what you would want to check here.
  The merchant_pub primarily is of interest because that is the key used to sign
  _refunds_. So we don't really care if it 'exists', the merchant could use a different
   merchant_pub for every deposit.
(0016362)
oec   
2020-06-25 10:55   
- As long as the tests for coin_pub/con_sig is implemented _somewhere_, that's fine. However, I suggest to put a description and reference to the location in taler-helper-auditor-deposits.c

- amount_with_fee*, same: If there are checks implemented somewhere, a note in the taler-*-deposits.c would be helpful

- merchant_pub: I see. For some reason I thought that merchants are registered. Would it make sense to check the refunds to be signed with a merchant_pub that is listed in a deposit?
(0016365)
Christian Grothoff   
2020-06-25 11:05   
Merchants are not registered, we want the system to be open. Forcing registration on merchants is not desirable, as that would allow an exchange to refuse to do business with some merchants, and we do not believe that this is socially desirable.

HOWEVER, it _may_ be that in the future _SOME_ form of registration is required from a regulatory perspective (KYC). *IF* this registration _WERE_ tied to a merchant_pub, then it may make sense to add some check here. But, that is only if regulatory requirements are articulated in such a way, and that remains very much unclear. So for the currently specified and implemented protocol, such a check makes no sense.
(0016448)
Christian Grothoff   
2020-07-14 21:15   
A check that the refund deadline should not be after the wire deadline was added to the taler-helper-auditor-coins in Git 2570b21d..62d5aae1.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6425 [libextractor] build system major always 2020-07-07 08:31 2020-07-10 23:41
Reporter: beberking Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 1.10  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: current SVN  
    Target Version: 1.11  
Summary: Soname was demoted from libextractor.so.3 to libextractor.so.2
Description: I am not sure which commit introduced the change, but the soname of libextractor was demoted from libextractor.so.3 to libextractor.so.2 between 1.9 and 1.10.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016443)
Christian Grothoff   
2020-07-10 23:39   
Woops. Backport: apply this to configure.ac:

-LIB_VERSION_REVISION=7
-LIB_VERSION_AGE=2
+LIB_VERSION_REVISION=8
+LIB_VERSION_AGE=1
(0016444)
Christian Grothoff   
2020-07-10 23:40   
Fixed in Git in 1d085a9..e51f24b


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6042 [Taler] wallet (WebExtensions) tweak N/A 2020-01-12 22:28 2020-07-10 23:29
Reporter: Christian Grothoff Platform: i7  
Assigned To: Florian Dold OS: Debian GNU/Linux  
Priority: high OS Version: squeeze  
Status: assigned Product Version: git (master)  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.9  
Summary: test webextensions wallet on GNU IceCat
Description: We should test the Webex wallet on IceCat, fix it if it is broken, and if it works list GNU IceCat on the wallet Website. Requested by RMS. Marcos should do this.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0015268)
gmarcos87   
2020-01-14 17:20   
After debugging the serviceworker I found that the problem with the Taler web extension is that it uses BigInt, which is not available in the current version of IceCat.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt#Browser_compatibility

There may be other probelamas, but this is the first blocker I found.
(0015270)
Florian Dold   
2020-01-14 23:26   
The wallet uses the big-integer package (https://www.npmjs.com/package/big-integer), which should only use native BigInt in the environments that support it, and fall back to a custom implementation of big integers on older browsers.

Could you please check where exactly the native BigInt is used and why the big-integer library doesn't fall back to its custom implementation?
(0015271)
gmarcos87   
2020-01-15 12:36   
(Last edited: 2020-01-15 12:49)
BigInt is used in wallet-core/src/crypto/workers/cryptoImplementation.ts (amountToBuffer and timestampToBuffer). The funny thing is that the lines used seem to affect what the functions return, but it does! If I remove these lines the signature validation process fails.

These two functions also uses DataView.prototype.setBigUint64() which has existed since Firefox version 68 and has no external library to replace it.

(0015272)
Florian Dold   
2020-01-15 12:52   
I don't quite understand what you wrote. Of course these functions all affect signature generation and verification!

What is surprising (and what I would like you to debug) is why the big-integer library (used in the functions you were referring to above) tries to use the native JavaScript big integer, even though the browser doesn't support it.

The big-integer *should* have a fallback implementation for older browsers, and we need to find out why it is not used!
(0015273)
gmarcos87   
2020-01-15 16:20   
The library is the one that is not being used in that file. It's called the native method, the big-integer package is not even imported at https://git.taler.net/wallet-core.git/tree/src/crypto/workers/cryptoImplementation.ts

I can do some debugging but solving this part of the implementation is beyond my current understanding of the code.
(0015518)
buckE   
2020-04-04 03:23   
Note to self: in an e-mail Christian says:
Ok, maybe you can take care of 0006043/0006042 and _really_ test the various
features of the demo with Brave and IceCat and _also_ look at the
console (F12) to see if any issues/failures are reported there while you
do it?
(0015519)
buckE   
2020-04-04 03:34   
Firefox Extension downloads but failed on initial install with error:

GNU Taler Wallet could not be installed because it is not compatible with IceCat 60.7.0.

F12 (Console and Debugger) show blank screens. No further testing done at this time. My first plan will be to look at the code to see if it is expecting a specific user agent string with the word "Firefox."
(0015521)
buckE   
2020-04-04 04:29   
Maybe? src/webex/compat.ts Line 22:

export function isFirefox(): boolean {
  const rt = chrome.runtime as any;
  if (typeof rt.getBrowserInfo === "function") {
    return true;
  }
  return false;
}

I tried editing this and rebuilding the add-on but the INSTALL file in wallet-core reads "Run `./configure && make' to create an archive
containing the extension in ./build/taler-wallet-$VERSION.zip" However `./configure` does not exist and this was not my highest priority at this time.
(0015524)
Christian Grothoff   
2020-04-04 10:21   
I'm pretty sure we're not checking for "Firefox", but we do have a version requirement (but it says 57+ on the Web site). Not sure 57+ is still accurate though.
(0015525)
Christian Grothoff   
2020-04-04 10:26   
Anyway, with a sufficiently recent IceCat, the issue _should_ be the BigInt stuff. It could be that the BigInt issue is explicitly detected now, or that IceCat 60 is 'too low' because of BitInt, not sure. The real question is why the BigInt library doesn't fall back to a software implementation as it should when the Browser doesn't support it natively. So debugging that _should_ be the hard part here.
(0015541)
buckE   
2020-04-07 06:24   
Florian: I don't know much about javascript but why are you surprised that "the big-integer library (used in the functions you were referring to above) tries to use the native JavaScript big integer, even though the browser doesn't support it?" It seems to me (knowing nothing) that one of these must be true:

1 - BigInt is a generic term, so a browser will treat it however its libraries tell it to.

2 - BigInt is a specific term, so any browser that does not support it will break unless you have some sort of `case` statement that includes specific alternatives. But even with a case statement, this might something that is really either in or out. Because the workaround to generate a number larger than Number supports depends on the specific use case (do you use a String, or Number.MAX_SAFE_INTEGER, etc.?)

I think the situation is closer to (2) so my question now is, maybe the choice is between (a) using numbers supported by BigInt, or (b) supporting IceCat, or (c) a substantial rewrite that uses some workaround. I do not see any reason to expect graceful fallback?

IF this analysis is correct (and it may not be) then the next question is: does IceCat have a plan to support the BigInt function? If it is as useful for strong crypto as I think then it may be on the release map.
(0015544)
Florian Dold   
2020-04-07 08:30   
The big-integer library is supposed to detect whether the browser natively supports big integers. Big integers were introduced into browser only relatively recently. If the browser *doesn't* support big integers natively, the library has a fallback implementation.

See this table (that unfortunately doesn't include icecat directly): https://caniuse.com/#feat=bigint

Maybe the code in big-integer that detects native support doesn't work correctly. Or it specifically doesn't detect it correctly on icecat.

I do think that icecat will eventually support is, once it merges in the latest gecko / other components from Firefox.
(0015546)
Christian Grothoff   
2020-04-07 10:13   
Currency-symbol for kudos: ク (ku in Katakana).
(0015570)
buckE   
2020-04-08 10:01   
> The big-integer library is supposed to detect whether the browser natively supports big integers. Big integers were introduced into browser only relatively recently. If the browser *doesn't* support big integers natively, the library has a fallback implementation.

So a call to BigInt will read a library that has another set of instructions to run instead, after sensing if the browser has this support. That seems weird to me, because it is very complicated to manually write an alternative to a BigInt function(I think?), so it seems very difficult to automate a fallback. (It is not a set procedure; BigInt solves a number of different approaches, not simply replaces one other method.)

> I do think that icecat will eventually support is, once it merges in the latest gecko / other components from Firefox.

In that case, my little recommendation is I think we should put this issue on hold until BigInt support is merged. We have a reason to use BigInt, and (I assume?) we have other priorities than learning why BigInt does not do what you expect. If RMS requests this maybe we can ask him what the timetable is for this merging? (And if it is a long time, then maybe this becomes higher priority.) But if you want me to investigate, and you have no more experienced js devs to do it, I will.
(0015573)
Florian Dold   
2020-04-08 11:04   
The big-integer library provides a superset of the functionality of the browser-native BigInt. Stuff like computing the GCD or modpow of big integers isn't available in the browser. Thus can't just get rid of the library, even if all browsers we use support native BigInt ;-)

I think we should first find out *why* this library doesn't properly detect it's running in a browser without native BigInt support.

Once some more high-priority issues are out of the way, I can look at the detection code myself.
(0015704)
buckE   
2020-04-20 11:12   
> The big-integer library provides a superset of the functionality of the browser-native BigInt

Maybe I am really confused but the extension specifically uses "BigInt," not some generic big integer call. So it's not using a generic term and the browser can use the best native library available; the browser would have to know that BigInt is a modern subset of big-integer. And how would it know that if it was built before BigInt?
(0015814)
buckE   
2020-04-30 10:01   
Assigning to Florian re: we are waiting for his feedback.

My synopsis:

Error is due to explicit use of "BigInt," instead of a generic call that could be interpreted differently by different browsers. I know nothing, but the docs I've read do not support the statement that "The big-integer library is supposed to detect whether the browser natively supports big integers." Also I am not a javascript programmer, I am just reading the docs I find.

Suggested solutions:

- may be moot point if GNU Icecat and all browsers we intend to support will move to BigInt ( From https://www.npmjs.com/package/big-integer: "Update (December 2, 2018): BigInt is being added as a native feature of JavaScript.")

- Or maybe we can include the library in all pages required by the wallet with <script src="https://peterolson.github.io/BigInteger.js/BigInteger.min.js"></script>

I remain available to test/verify/etc.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6416 [Taler] auditor minor always 2020-06-25 11:15 2020-07-10 23:24
Reporter: oec Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 0.7.0  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: Same coin_pub with multiple denom_sigs - a problem?
Description: Taler uses a cache for fast lookups of coin_pub -> (denom_pub, denom_sig), the table known_coins. The table is populated via TEH_DB_know_coin_transaction before deposit, melt and recoup operations, i.e. independent of the outcome of those operations.

Consider the scenario where the same coin_pub is signed with different denomination keys. The first usage of one of those coins would lock the denomination value in the known_coins table. However, it is not clear (to me) what would happen if the same coin_pub than is used later with a _different_ (but also validly signed) denomination for any of the operations.

I have not come up with a particular attack to the advantage of a customer (i.e. gain profit). But maybe leaving the exchange in a confused state that the auditor might notice and complain about could lead to DoS?

Would it make sense to have (coin_pub, denom_pub) as an index for the known_coins and allow multiple entries with the same coin_pub in it?
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: db.patch (85,066 bytes) 2020-07-07 00:28
https://bugs.gnunet.org/file_download.php?file_id=919&type=bug
Notes
(0016366)
Christian Grothoff   
2020-06-25 11:48   
Great find. Discussed with Florian, we will change the index to span coin_pub and denom_h and make sure all SELECT()s also select on both columns.
(0016391)
Christian Grothoff   
2020-07-07 00:28   
Proposed DB change attached. May be incomplete (see FIXMEs), also requires modest changes to the rest of the exchange code.
(0016396)
Christian Grothoff   
2020-07-08 12:49   
After some internal discussion, we decided that it would be too messy (and brittle) to really allow coin_priv/pub re-use. Reasons include that having 'coin_pub' serve as a primary key is nice and simple, and that tolerating wallets that non-sensicaly re-use coin keys should really not be encouraged by allowing it per spec. So instead we will change the code/protocol to
a) detect that the coin_pub has been used before, and
b) return an error proving this.

This requires the following changes:
1) Various request signatures (deposit, melt, recoup) must include the h_denom_pub so we can prove that the same coin key was involved in different denominations, and it's not the exchange's fault.
2) The transactions must be changed to only commit to known_coins table if the rest of the transaction that includes the above signature also ends up in the DB. So we merge the two transactions (add known coin + actual work) into one transaction.
3) The transactions must be changed to detect the coin_pub re-use with a different denom_pub and then return the coin's transaction history, proving that it was previously used with the same denom_pub_h.
4) We _may_ need to return h_denom_pub with some other replies to enable verification of the signatures with the changes from (1). (not sure).
(0016398)
Christian Grothoff   
2020-07-08 15:36   
95f5de1..8236b1e updates the docs.git specification to add the h_denom_pub where needed (deposit and melt, already present for recoup).
(0016399)
Christian Grothoff   
2020-07-08 17:22   
1ca062fc..97dfbec0 adds the h_denom_pub to the deposit request.
(0016400)
Christian Grothoff   
2020-07-08 18:05   
97dfbec0..8e03498a adds the h_denom_pub to the melt request.
(0016401)
Christian Grothoff   
2020-07-08 18:27   
8e03498a..c93f6471 combines the known_coin transactions with the respective deposit/melt/recoup transactions, so that there will never be a known coin entry if the 'rest' of the transaction failed.
(0016402)
Christian Grothoff   
2020-07-08 19:45   
c93f6471..8a1402a5 implements generation of proper errors for conflicting denomination keys on the server-side.
(0016403)
Christian Grothoff   
2020-07-08 20:05   
8236b1e..31e66d9 documents the API change in docs.git
(0016404)
Christian Grothoff   
2020-07-08 21:09   
31e66d9..2f705cc further updates the specs to update the coin history response to include the coin's signatures on the RECOUP request and not merely exchange signatures on the confirmation(s) where applicable.
(0016405)
Christian Grothoff   
2020-07-08 21:30   
8a1402a5..92ac6dd1 should implement the new logic client-side as well. However, I noticed the client did previously _NOT_ handle some of the corner cases properly! Needs more testing!
(0016441)
Christian Grothoff   
2020-07-10 23:16   
ddf95c49..7085cfef adds test cases specifically creating coins with private key re-use and checking that deposit/melt fail with this.
*recoup* handling is NOT yet tested.
(0016442)
Christian Grothoff   
2020-07-10 23:24   
7085cfef..8ea4e50a adds a test with recoup as well. Declaring victory.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5960 [libeufin] sandbox minor have not tried 2019-11-06 11:12 2020-07-09 19:38
Reporter: Marcello Stanisci Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: All exceptions lead to "500 Internal server error"
Description: Even if the client POSTs invalid data, they get a "500 Internal server error" response.

This happens since now there is only one default exceptions handler that catches *all*
exceptions and respond with 500. Eventually, any exception should be managed according
to the context it originates from.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015919)
Marcello Stanisci   
2020-05-15 19:28   
(Last edited: 2020-07-09 19:31)
Exceptions are now organized into three sections: NexusError, SandboxError, and UtilError.
Each type accepts a HTTP status code and a hint message to show to the user. Therefore this
bug doesn't really apply anymore.

BUT: because Gson doesn't throw any exception when a non-nullable field was missing in the
uploaded JSON and such field *are* tried to be accessed along the code, then an invalid JSON
POSTed can *still* lead the Sandbox (and the Nexus) to respond with 500 (due to the NPEs that
inevitably result by accessing those problematic null-fields.)

(0015921)
Marcello Stanisci   
2020-05-15 19:35   
Must correct the previous comment: although the SandboxError type was created, the Sandbox never
picks the details from it and rather respond with "500 Internal Server Error".
(0016438)
MS   
2020-07-09 19:38   
Such commit aee99f3e1a4fd38dbf2fa352b112ce9d08c0566c defines responses with a error description, and a HTTP status code (other than the default 500). *if* 500 is returned, then either the server reached a fatal state or one less severe error wasn't properly caught and managed.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6423 [libeufin] nexus minor have not tried 2020-07-03 01:09 2020-07-09 19:29
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Need a endpoint returning the list of prepared payments.
Description: Nexus needs the following endpoint:

GET {nexusBase}/bank-accounts/{my-acct}/payment-initiations

to return the list of all the prepared payments belonging to 'my-acct'. Some filter might be needed, though, as
it would be very inconvenient to return always ALL the records.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6420 [libeufin] nexus minor have not tried 2020-06-30 13:36 2020-07-09 19:29
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Return always JSON in responses, and make HTTP response code match the EBICS error.
Description: Nexus should return *always* JSON, and if one EBICS response has errors, then the response's status code should represent the problem that happened in the EBICS communication.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016397)
MS   
2020-07-08 13:03   
Nexus seems to already respond always with JSON. Another review is needed to check if such responses reflect always what the bank says.
(0016437)
MS   
2020-07-09 19:28   
The current style tends to throw EbicsProtocolError() when the bank sends a invalid response, or some technical error happens. This object is then caught and its message gets included in the NexusErrorJson() response object.

This can be closed, but error management needs to be tested!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6409 [libeufin] other minor have not tried 2020-06-22 17:05 2020-07-08 22:13
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Try to validate documents against their schema before communication.
Description: Nexus and sandbox always communicate their XML documents without validating them first.
Instead, they should tend to validate any document before communicating it over the network.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016406)
MS   
2020-07-08 22:13   
Implemented in d3fba053fe2e0e4a1268da726f62aa96f0e12fdd (both sandbox and nexus)


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6386 [libeufin] sandbox minor always 2020-06-16 11:08 2020-07-06 16:45
Reporter: tanhengyeow Platform:  
Assigned To: MS OS:  
Priority: urgent OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: One EBICS subscriber with multiple bank accounts
Description: Currently, each "EBICS Subscriber" can only "own" one bank account. We can consider allowing one EBICS subscriber to own multiple bank accounts.
Tags:
Steps To Reproduce: 1. Run the script libeufin/integration-tests/start-testenv.py
2. One "EBICS Subscriber" should own multiple bank accounts but it only shows one bank account in the NexusBankAccounts database table (nexus-testenv.sqlite3)
Additional Information:
Attached Files:
Notes
(0016389)
MS   
2020-07-06 16:21   
By looking at the code, it seems that sandbox does allow to create more than one bank accounts under the same one EBICS subscriber.

That is the API:
https://docs.taler.net/libeufin/api-sandbox.html#post--admin-ebics-bank-accounts

And that is the implementation:
https://git.taler.net/libeufin.git/tree/sandbox/src/main/kotlin/tech/libeufin/sandbox/Main.kt#n238

As you can see, such "BankAccountEntity" object from the sandbox DB doesn't forbid multiple
accounts with the same subscriber. Some test is needed, but this one might be a non bug.
(0016390)
MS   
2020-07-06 16:45   
Tested. The sandbox already allows this. If Nexus has problems to *retrieve* multiple bank accounts per subscriber, then that should get a new bug assigned.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5930 [Taler] mechant backend feature N/A 2019-10-16 15:46 2020-07-06 10:58
Reporter: Florian Dold Platform:  
Assigned To: Christian Grothoff OS:  
Priority: low OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: control of instances over contract terms should be restricted
Description: Currently, instance A can "pretend" to be instance B, by supplying the information of B in the contract terms.

Unless *explicitly* configured, an instance should not be able to set certain fields of the contract terms, such as the merchant field. This field should be taken from the instance configuration instead.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015640)
Christian Grothoff   
2020-04-13 21:32   
Is there a reason why you say this should be allowable by configuration? I'd simply 400 bad request such orders, unless you have a very good reason to allow it.
(0016387)
Christian Grothoff   
2020-07-06 10:58   
Fixed in d37e16a..85a0221: require 'merchant' field to be provided by backend only.
We already force the merchant_pub being set by the backend.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6408 [Taler] exchange crash always 2020-06-22 16:01 2020-07-05 22:07
Reporter: Florian Dold Platform:  
Assigned To: Christian Grothoff OS:  
Priority: high OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: exchange wirewatch crashes when it gets a non-rounded timestamp
Description: When the Taler Wire Gateway API returns a non-rounded timestamp, wirewatch just crashes and burns:

eurint@gv:~$ taler-exchange-wirewatch -r
Jun 22 15:53:36-967726 taler-exchange-wirewatch-13646 ERROR Assertion failed at pq_query_helper.c:252. Aborting.
Aborted (core dumped)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016386)
Christian Grothoff   
2020-07-05 22:06   
6de49ea2..54e63f01 fixes in exchange.
f829235..d37e16a fixes in merchant.

The fixes replace the JSON parsing functions from GNUnet with Taler-specific variants that insist on the time being rounded.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5487 [Taler] deployment and operations feature have not tried 2018-11-27 10:53 2020-07-05 00:07
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: confirmed Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: set up system monitoring on gv
Description: On tripwire, we previously ran into the issue of a full /tmp directory.

With gv, we should have monitoring in place to inform us of abnormal states, such as full disks or excessive memory use.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0014680)
nikita   
2019-07-15 18:55   
ack, will get to this after GSoC.
(0015634)
buckE   
2020-04-13 03:37   
Please list all things to monitor here. They do not have to be in one note, but as you think of them, add them so we have a full picture when it's time to start this.
(0016320)
buckE   
2020-06-19 08:54   
Please create a list of things we should monitor. The list will help determine the best solution. If it's just RAM and disk space, we could have a simple BASH script and cron job. If the list is larger, we do something else.

Also Christian should be involved because I think he will probably use it (as root) most often. Maybe he likes something that he doesn't have to check, but notifies by e-mail when there is a problem. Maybe he prefers a GUI like Zabbix. Etc. But let's start with a list of things to monitor.
(0016356)
Florian Dold   
2020-06-24 18:18   
We need two things:

1. General system monitoring, such as disks, memory, CPU usage.
2. Application-specific monitoring, which will go over log files and extract "stuff" from it. What this stuff is depends to some degree on the application, but we want to at least collect log lines that are at the ERROR log level and alert if they occur with a too high frequency.

NetData (https://github.com/netdata/netdata) might be a good start for system monitoring.
(0016358)
buckE   
2020-06-25 07:52   
Ah. Well if you already have a project in mind that you want to use, it's now a matter of how Christian feels.

This is the quickstart: https://github.com/netdata/netdata#quickstart

For the .deb: https://github.com/netdata/netdata/blob/master/packaging/installer/README.md

And a good overview: https://github.com/netdata/netdata/blob/master/docs/guides/step-by-step/step-00.md#netdata-fundamentals

Here is information on opting out of the analytics: https://github.com/netdata/netdata/blob/master/docs/anonymous-statistics.md

From the website it looks to have some nice features, including integration with PostgreSQL, nginx, syslog.

But I really can't imagine doing this on a live system without testing on a stage server first. But the next step is gettin Christian's opinion. Assigning to him.
(0016361)
Christian Grothoff   
2020-06-25 09:16   
netdata seems fine. I do see the point that we need to sort out server staging. Don't have a good plan for that yet.
(0016369)
buckE   
2020-06-26 11:27   
> do see the point that we need to sort out server staging. Don't have a good plan for that yet.

I recommend a cheap VPS. They can be *very* cheap. We don't need anything fast, or even very secure just to do staging (except deployment.git stuff maybe). All we need to do is to duplicate the taler.net environment reasonably well.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6411 [Taler] deployment and operations minor have not tried 2020-06-24 02:22 2020-07-05 00:06
Reporter: buckE Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: confirmed Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Change doc-builder (sphinx ) log location
Description: > Well, ~ is not ideal either, as it'll put stuff into people's $HOME
> directories. $TMPDIR or $PWD would be acceptable IMO.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016359)
buckE   
2020-06-25 07:56   
How about `/tmp/file.log` ?
(0016360)
Christian Grothoff   
2020-06-25 09:09   
Well, 'file.log' is a bit, eh, unspecific. "${TMP:-/tmp/}/sphinx-build.log" should be fine.
(0016368)
buckE   
2020-06-26 05:54   
What's the issue with using /tmp/example.log? The point is to use an absolute path because the same path is being referenced in a Makefile and also a shell script.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6378 [libeufin] nexus minor have not tried 2020-06-12 17:59 2020-06-30 22:28
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: implement "two step" bank account import
Description: In the first step, the user downloads the bank account information offered by the
bank and nexus stores it in a temporary place (a database table storing the raw XML).

In the second step, nexus lists all the downloaded bank accounts and lets the user
import some/all of them by giving them the way of assigning a personal label to the
imported account.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016378)
Florian Dold   
2020-06-30 13:52   
We also need integration test cases for this!
(0016380)
MS   
2020-06-30 22:28   
Tests (and related fixes) were added here:
245b3ea998a9e095c4ca685a019364afb8b9f06a


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6417 [libeufin] nexus minor have not tried 2020-06-25 16:32 2020-06-30 17:58
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: nexus should support bank connection deletion
Description: Nexus should let users delete their bank connections.
One use case is when a bank connection had errors and didn't get confirmed by the bank.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016377)
Florian Dold   
2020-06-30 13:21   
Also the documentation should be updated and there should be a test case ;-)
(0016379)
MS   
2020-06-30 17:58   
Implemented and tested here:
98ab97b89a4b7f56efe1ecbff61bb29c118fa675

Also documentation was extended accordingly.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6421 [libeufin] sandbox minor have not tried 2020-06-30 16:57 2020-06-30 16:57
Reporter: MS Platform:  
Assigned To: OS:  
Priority: low OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: CLI should fail when the called logic does.
Description: Basically, CLI returns always a successful status code, especially when Nexus gives some bad response. It should instead return some status code other than zero when something unexpected happens.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6269 [libeufin] sandbox minor have not tried 2020-05-29 08:48 2020-06-30 13:55
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: sandbox should emit c52/c53 more like real banks do
Description: Right now we generate a fresh c53 statement for every request.

Banks would return the same message on every request. They basically emit c5x messages regularly, and EBICS just allows you to download them.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015980)
Florian Dold   
2020-05-29 09:25   
A major problem right now is that the MsgId of messages emitted by the sandbox is always zero. This doesn't mesh well with the assumptions made in the nexus (and that are presumably true for all real banks).
(0015981)
Florian Dold   
2020-05-29 09:26   
Setting this to higher prio, as Heng Yeow needs this for testing his UI stuff
(0015983)
MS   
2020-05-29 13:33   
bf5ae27..01c8396 is a quick fix that gives more entropy to the message id.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6402 [libeufin] other minor have not tried 2020-06-22 07:01 2020-06-30 13:51
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: handle disrupted EBICS transactions
Description: We should either explicitly cancel such a transaction or re-try if the EBICS host supports this feature.

Currently we might run into waiting for a timeout (for re-doing the same order type) until the bank cancels the active transaction.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6405 [libeufin] nexus minor have not tried 2020-06-22 14:37 2020-06-30 13:51
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Define convenient data structure to store raw bank accounts information
Description: Nexus stores the HTD response as a raw XML string into database. This information instead should be parsed and stored either using appropriate DB columns or serialized in a simpler JSON structure.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016345)
MS   
2020-06-23 15:36   
addressed here: 5f9ad837016c66fe974e413432b2785b58268820.

Values are stored in a dedicated table's columns.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6395 [libeufin] nexus minor have not tried 2020-06-18 21:54 2020-06-30 13:23
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: implement bank protocol-specific logic in a less ad-hoc way via interfaces and a registry
Description: Instead of having some ad-hoc methods and switch statements, we should define an interface and use polymorphism.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6415 [libeufin] nexus minor have not tried 2020-06-24 21:06 2020-06-30 13:20
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: split transaction list into physical transactions (entries) and logical transactions
Description: This would allow the UI to always show logical transactions, but we would still have the information available to reconcile data from different sources (camt.052/53/54, FinTS / EBICS).
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016376)
Florian Dold   
2020-06-30 13:20   
We're actually not gonna do this, as tracking the state of logical transactions is a big mess.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6236 [Taler] exchange API (C) feature N/A 2020-05-13 17:59 2020-06-27 18:37
Reporter: Christian Grothoff Platform: i7  
Assigned To: Christian Grothoff OS: Debian GNU/Linux  
Priority: normal OS Version: squeeze  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: exchange deposit testing command should expose deposit fee
Description: This is helpful/needed for properly testing POST /transfers in the merchant.
Suggestion: use index 0 for the deposit amount, and index 1 for the fee. The deposit amount is already exposed.

Complication: the fee will need to be obtained via the coin's withdraw command, the current deposit command does not really know it.
Tags:
Steps To Reproduce:
Additional Information:
System Description
Attached Files:
Notes
(0016372)
Christian Grothoff   
2020-06-27 18:36   
b3411dc6..d0d71dab implements this in the exchange.
cde317f..4b820e2 makes use of it in the merchant.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6391 [libextractor] plugins minor always 2020-06-16 15:13 2020-06-26 01:13
Reporter: beberking Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 1.9  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 1.10  
    Target Version: 1.10  
Summary: FAIL: test_rpm
Description: While building libextractor on Debian unstable (with librpm8 4.14.2.1), the test suite fails one test with the following message:

FAIL: test_rpm
==============

Got additional meta data of type 58 and format 1 with value `Thu Oct 2 09:44:33 2003' from plugin `rpm'
Did not get expected meta data of type 58 and format 1 with value `Thu Oct 2 11:44:33 2003' from plugin `rpm'
FAIL test_rpm (exit status: 1)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016357)
Christian Grothoff   
2020-06-24 21:06   
Hmm. I suspect that's a time zone (conversion) issue.
(0016367)
Christian Grothoff   
2020-06-26 01:13   
Fixed in Git.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5853 [Taler] merchant backend API (C) minor have not tried 2019-08-26 23:32 2020-06-25 23:44
Reporter: Florian Dold Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: sessions should be limited in number and duration per order_id
Description: Currently we keep session-based payments around in the database forever.

We should delete old sessions after a configurable duration.

Furthermore, we should limit the number of simultaneous sessions per order id.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015647)
Christian Grothoff   
2020-04-14 14:44   
Florian says: simply don't support multiple session IDs. When a second one is created, drop information about the first.
(0015842)
Christian Grothoff   
2020-05-03 01:31   
The protocolv1 branch now stores one session per order.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6412 [Taler] deployment and operations minor have not tried 2020-06-24 06:54 2020-06-24 12:34
Reporter: buckE Platform:  
Assigned To: Christian Grothoff OS:  
Priority: normal OS Version:  
Status: resolved Product Version: git (master)  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 0.8  
    Target Version: 0.8  
Summary: New GPG key for Weblate
Description: Weblate will use a new GPG key. I don't think there is anything you need to do because the SSH key is the same but in case there is, please find it at:

https://weblate.taler.net/keys
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016346)
buckE   
2020-06-24 07:27   
(Last edited: 2020-06-24 07:36)
*deleted*

(0016347)
buckE   
2020-06-24 08:01   
it was a weblate bug


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6407 [libeufin] other minor have not tried 2020-06-22 15:59 2020-06-24 12:18
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Move logs to appropriate location
Description: nexus and sandbox log everything to /tmp/nexus.log and /tmp/sandbox.log. Instead, they should log to STDOUT and/or give some command line option to specify to which file logs should be saved.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016350)
MS   
2020-06-24 12:18   
3c50e81 implements a "--log-file" option for nexus and sandbox where users can pass the
path to the wanted logfile. If such option is not given, then only stdout will show logs.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6372 [Taler] Web site(s) minor have not tried 2020-06-10 12:36 2020-06-24 08:16
Reporter: Stefan Platform:  
Assigned To: Stefan OS:  
Priority: normal OS Version:  
Status: assigned Product Version: git (master)  
Product Build: Resolution: reopened  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: git (master)  
Summary: Need for fixing the German Taler website https://taler.net/de/principles.html
Description: Dear Buck,

On the German Taler website /principles.html, there are some new paragraphs which appear in English language. I tried to fix this via Weblate and tried alternatively Poedit, too. But everything I see displayed there, are only those paragraphs that have been used formerly to generate the said website. So, no new paragraphs there...

Weblate displays the former text strings only. The same applies to Git. See https://git.taler.net/www.git/tree/locale/en/LC_MESSAGES/messages.po and analogously the German .po file https://git.taler.net/www.git/tree/locale/de/LC_MESSAGES/messages.po

Interestingly, Weblate shows a link to the sourcecode, from which the website is generated. There are on display the new paragraphs:
https://git.taler.net/www.git/tree/template/principles.html.j2?h=master#n36).

So there must be a logic and a place for the new strings to be handled. I have been waiting for an update that might be effective on Weblate, but so far nothing has changed. Could you be so kind to check for a solution?

Thank you very much for your work in advance.

Kind regards
Stefan
Tags: translation
Steps To Reproduce:
Additional Information:
Attached Files: Screenshot_2020-06-11 GNU Taler Taler Website.png (32,275 bytes) 2020-06-11 06:29
https://bugs.gnunet.org/file_download.php?file_id=914&type=bug
Screenshot_20200616_074500.png (329,678 bytes) 2020-06-16 08:09
https://bugs.gnunet.org/file_download.php?file_id=915&type=bug
messages.po-DE-14JuneBackup (94,155 bytes) 2020-06-17 02:23
https://bugs.gnunet.org/file_download.php?file_id=916&type=bug
Screenshot_20200616_080729.png (229,823 bytes) 2020-06-17 08:50
https://bugs.gnunet.org/file_download.php?file_id=917&type=bug
Notes
(0016255)
buckE   
2020-06-11 06:29   
I'm not sure I understand the problem. Please understand I do not know weblate any better than anyone else. Probably less as I have never used it.

Let me try to understand the issue:

> On the German Taler website /principles.html,

Link please?

> there are some new paragraphs which appear in English language.

I assume I will see these when I get the link.

> I tried to fix this via Weblate

Exactly how did you try to fix this in Weblate? This is where a bug report is useful. What did you try, what was the result, and what was the desired result?

> and tried alternatively Poedit, too

What's Poedit, and - if you think it is important that I know - what did you try, what was the result, and what was the desired result?


What I can tell you is that there is a problem with the repo. Do you have access to this page: https://weblate.taler.net/projects/gnu-taler/taler-website/#repository

If not, I attached a screenshot of the error.

The problem is a conflict with the files in the French repository. I wonder how this happened. Is it possible that someone is manually pushing changes to the french repository? Doing a simple "rebase" does not fix the problem. I am not sure how to manually handle this fix but I will investigate if when I get your reply it seems like this is the source of the problem.
(0016256)
Stefan   
2020-06-11 09:08   
Hi,

I apologize that I posted the link to the German website in the headline of my bug commit only and not within my explanation of the bug, so here it is: https://taler.net/de/principles.html

Weblate is new for us all, and I recently found out that this web-based service is lacking some advantages in comparison to Poedit, a tiny little program we used beforehand to manage translations for the website. Admitted, I am not a big fan of Weblate anymore.

With Poedit, at least I can scroll through all of the strings at a glance. I did that yesterday in order to resolve this bug to know where the new strings in English language are in relation to the German strings which are unfortunately replaced by the new English ones on the said website.

As far as I know, websites are generated by a logic that seeks for strings and not for string IDs. So it all depends on the English original source strings that we find stored in the file https://git.taler.net/www.git/tree/template/principles.html.j2?h=master#n36 (absolute path in Git: /www/locale/en/LC_MESSAGES/messages.po).

For resolving this bug, I tried to find the strings in /www/locale/de/LC_MESSAGES/messages.po and expected there the new English strings. No clue about how they did not even appear in the English messages.po file.

Yes, you might have a good reason to assume that there is a problem with the repository.

I have access to https://weblate.taler.net/projects/gnu-taler/taler-website/#repository

And yes, before I was spammed by mails from Weblate notifications regarding the project "DO NOT USE - See "GNU Taler" for Main Website project" for which I am designated as the administrator somehow. When I turned off all notifications from Weblate the spam ceased, but I am still the administrator of both projects "GNU Taler" and the mentioned project "DO NOT USE". :-)

Kind regards
Stefan
(0016269)
buckE   
2020-06-15 10:00   
Stefan, I have a bug report out on this with weblate. I will update you when I get a reply.

Question: did you do any French translations? Maybe even accidentally?

I think it will be helpful if you use the weblate user interface in the future. See https://docs.weblate.org/en/latest/faq.html#how-to-fix-merge-conflicts-in-translations

But I do not think that is what caused this problem. (Maybe, maybe not.)

I will update you when I know more.
(0016271)
buckE   
2020-06-15 10:12   
Stefan, I had a realization:

You were not using weblate to add translations.

The French person's translations were already committed.

So there was nothing to lose from a "reset" of the local/weblate changes, and that is what I did.

In the future it is best to use the weblate interface for translations. And if you can not do this, at least make sure to do a pull in the weblate interface frequently.

It looks like there are no more issues. Please check and reopen if you still have problems.
(0016272)
buckE   
2020-06-16 04:37   
Stefan, I got information from you in an e-mail but I don't see it here. Anyway I will cut and paste from the e-mail:

> I would not consider the issue as resolved. The German website https://taler.net/de/principles.html still contains English phrases, the staged version is fully in English now.

I don't know what you mean by "staged version." I thought we only have a live site. Do you mean https://taler.net/de/index.html? Because I see that in German.

Could it be that you added changes but they did not appear immediately and this worried you? Weblate was set to push every 24 hours. I just changed that to 1 hour.

If there are specific translations that you made that still do not appear, that is certainly a problem. Were these ones that you added in poedit, or weblate? Can you add your local copy to weblate again? Are changes you add to weblate showing up on the site? I need more information to offer suggestions.

> My last commit via Poedit was in March. In Mai, I used Weblate as the one and only tool to contribute to our website translation. Moreover, I just did not touch the French strings at all.

Okay good. Mystery solved, if not the actual problem. And it means I may have overwritten your changes. I was under the impression your translations were all checked in outside of weblate, so my mistake may have cost us some changes.

> In one of Android component, I translated one string.

Okay. We'll see if there are issues there eventually, but not now.

> Sure, I will use Weblate in the future, even though it seems to be a service that distinguishes with its complexity.

Maybe. I have never used it. I am not in charge of this in any way.

> As a suggestion and also to keep my restricted time effective for other jobs I have to fulfill for Taler, I would like you to change my status in Weblate to no longer be the administrator of any project,Not the end result

I don't see that you are an administrator or superuser. Maybe I don't know your e-mail address? But you are "Users,Viewers,Managers" and I can delete Managers and see what happens?

> I was never designated to take over the role of administrating a project, but to translate strings and set up components.

It looks like to set up components, you have to be Manager level: https://docs.taler.net/developers-manual.html#how-to-create-a-component

> So please... do something about the German website

That's really quite vague. I need actual bug reports to help you. I do not know what is wrong, or about what you've done, and I am not tech support. However I do have admin access to weblate and I will use that power to investigate any bug reports you choose to file. I'll need to know the exact problem, the context, how I reproduce it, and what you expect to happen instead. A complaint about the end result does not give me anything to try to debug.

> at least you could be so kind to tell me where in the repo to search for the file that keeps the English-German translation strings.

No I could not because I do not know anything about the repo structure, such as how translations are handled. But I can tell you that if you know a phrase that you are searching for, you could get into the parent directory of the repo and type this:

grep -R 'text to search for'

and that command should output the filenames that contain that string. Then you can find the location with:

find . -name 'filename.txt'

You can also give me an *exact* string to search for and I can do this locally. Then I will send you whatever files I find, if you want. Would that help? I do not know what the issue is
(0016275)
Stefan   
2020-06-16 08:09   
Dear Buck,

Please take a look at https://stage.taler.net/de/principles.html (the staged version) and https://taler.net/de/principles.html (the live version of the website).

Obviously, the are paragraphs in English and in German language mixed together. Before the change to the English master text happened (in the first days of June), all the passages on the said website were in German only. After that, some paragraphs are in English, some are in German.

I can see the English master on https://git.taler.net/www.git/tree/template/principles.html.j2?h=master#n36. There are the new paragraphs in English.

I can also download the translation file for German (see screenshot). But there are the old texts only displayed.

What I cannot do is find the file from where the German website has its text generated. It is definitely not anymore in the git tree. I pulled the ~/www repository and have its actual version here as like it resides on the Git tree and its repos.

So, take it with a smile ;-) - and let's try to find out about the 'mystery' of the German website ... I'm sure that we will unveil the secret.

Cheers

Stefan
(0016282)
buckE   
2020-06-17 02:23   
Thank you for showing me a link to a stage site. I did not know this existed, although I am not sure what this has to do with the original issue. Anyway....

> What I cannot do is find the file from where the German website has its text generated. It is definitely not anymore in the git tree.

Well, I don't know "the file from where the German website has its text generated" and you're really not telling me much. But I have a backup from 14 June of the DE messages.po. It's attached. Does it help anything? I don't know what messages.po means but it was under a de/ directory and it contained a string `Die Nutzer des Bezahlsystems` from https://taler.net/de/principles.html. That's good, right?

Some good news is that the source of the problem, Christian thinks and I think sounds plausible, is because I migrated this project from another location inside weblate. So this error is not something that should happen often. But it does mean a good best practices is to do a pull in weblate before committing local weblate translations.

> and let's try to find out about the 'mystery' of the German website ... I'm sure that we will unveil the secret

I do not know what this means, or who the "us" is who is working on that mystery. But if there is information I can provide, feel free to ask me.
(0016283)
buckE   
2020-06-17 02:24   
Marking resolved as we know what caused the problem, Stefan has a backup copy from 14 June, and there is nothing else to do without more information.
(0016284)
Stefan   
2020-06-17 08:44   
Hello Buck,

My latest commit of June 14 did not change anything on the website. The German strings remained the same.

The German website (staged and live version) is still a puzzle of German and English phrases. It seems that all the strings of that website respectively the string IDs are mixed up together as if there was an incident which killed the relation between the original English strings and the German ones.

What more information do you need to solve this issue?

> I migrated this project from another location inside weblate
This is exactly what I want to know in order to find a solution on my own. Where is that location inside Weblate?

Moreover, in Weblate I am still displayed as the "administrator" of the project GNU Taler and of 5 components. I want to administrate the Terms of Services component only. Maybe you could be so kind to solve that issue, too.

Thanks for your work on both topics.

Kind regards
Stefan
(0016285)
Stefan   
2020-06-17 08:49   
Attached you will find a screenshot of the "administration" issue (project GNU Taler and 5 components I don't want to administrate).

With the 'manager' status in Weblate I want to administrate the 'Terms of Services' component only, which I am going to open on my own.

Thanks for your work in advance.

Kind regards

Stefan
(0016286)
buckE   
2020-06-17 10:15   
> My latest commit of June 14 did not change anything on the website. The German strings remained the same.

I don't know what you are communicating here. Are you saying that the file that I sent you does not have the lost information you need? Honestly I'm not sure at this point if there *is* lost information, or what it is you need?

> The German website (staged and live version) is still a puzzle of German and English phrases. It seems that all the strings of that website respectively the string IDs are mixed up together
> as if there was an incident which killed the relation between the original English strings and the German ones.

Maybe this is so. I do not know anything about it, and I don't know what "string IDs" refer to. I can not think of any incident that killed this relationship though. The only thing I did (recently) was to reset the local weblate repository because the local weblate repository was conflicting with the remote repository, and the alternative fix required installing a lot of potentially buggy weblate subsystems, and opening commit access via SSH. Also as you know I was not aware that there were important translations in weblate at that time.

This might have been an error, but it is an error in the past. I can not un-do it. Anyway I do not see how it relates to this new issue you describe about "all the strings of that website respectively the string IDs are mixed up together?" Maybe it does, but (a) I'm not sure how and (b) I wouldn't be able to go back and un-do that decision anyway.

Some suggestions: If you think the repository is good, but weblate is bad, you could create a new "website 2" project, do a fresh checkout from git, and see if that is any better.

If you think the information in weblate is good, but the website is bad, then please explain this and if you can't find the problem in weblate I will try looking.

If you think there is an old version of the website that is good, but it is not in current master or in weblate, you could look through Git (I can help you if you want) for older versions of the committed files. I would have to know exactly what you are looking for though.

Or I could provide more files from the 14 June backup, but maybe you are saying that backup does not have information you need?

> What more information do you need to solve this issue?

Well I'd need to know all of the things I keep asking you for. I still haven't gotten an initial bug report.

> > I migrated this project from another location inside weblate
> This is exactly what I want to know in order to find a solution on my own. Where is that location inside Weblate?

The new project is: https://weblate.taler.net/projects/gnu-taler/taler-website/. I think it is the only one you used? The old one was for testing only, and was labeled "DO NOT USE" and the migration was really a fresh setup and fresh check-out from git.taler.net, not an internal weblate migration.

> Moreover, in Weblate I am still displayed as the "administrator" of the project GNU Taler and of 5 components. I want to administrate the Terms of Services component only. Maybe you
> could be so kind to solve that issue, too.

Sorry I think I was not clear earlier. I do not know of user levels that are this specific, and you said you need "Manager" status. I do not know weblate so it's possible I will discover something in the future that allows for this level of control, but at the moment I think a person is either a Manager for a Project (GNU Taler) and therefore all of the Components under that project, or one is not a Manager for any Components of a Project. Maybe this is a feature request you could make from Weblate if you wish it.
(0016287)
buckE   
2020-06-17 10:16   
Also we upgraded Weblate to 4.1 yesterday and we can expect bugs. I already found one, and we will find more I think. But I need specifics to help find them.

What's the context?
What did you do?
What happened?
What did you want/expect to happen instead?
(0016288)
buckE   
2020-06-17 10:36   
Stefan I did some research for you and I saw that there is a problem with weblate committing: https://weblate.taler.net/projects/gnu-taler/taler-website/#repository

I can't always manage your commit process so this is something you should check if you have local changes that do not appear on the website. But this time I thought of it.

This looks like a weblate 4.1 bug, so I reported it. (I wonder if weblate 4.1 is not reading our GPG key. I don't know.) Of course, I do not know if the problem you have is that there is code in weblate that you do not see on the website? But if that is the case, this might be a solution. I'll let you know when I get information back.
(0016289)
Stefan   
2020-06-17 10:56   
My answers inline.

>> My latest commit of June 14 did not change anything on the website. The German strings remained the same.

> I don't know what you are communicating here. Are you saying that the file that I sent you does not have the lost information you need? Honestly I'm not sure at this point if there *is* lost information, or what it is you need?

We do need that the German website displays German text. There is no lost information, because I have all the information backuped here at my place. But the German website https://taler.net/de/principles.html is just a puzzle of English and German phrases now. I want this mixture to be fixed urgently. But I cannot find the file where the strings for the website are stemming from.

> The German website (staged and live version) is still a puzzle of German and English phrases. It seems that all the strings of that website respectively the string IDs are mixed up together
> as if there was an incident which killed the relation between the original English strings and the German ones.

> Maybe this is so. I do not know anything about it, and I don't know what "string IDs" refer to. I can not think of any incident that killed this relationship though. The only thing I did (recently) was to reset the local weblate repository because the local weblate repository was conflicting with the remote repository, and the alternative fix required installing a lot of potentially buggy weblate subsystems, and opening commit access via SSH. Also as you know I was not aware that there were important translations in weblate at that time.

This is of course not anyone's fault, especially not yours.

> This might have been an error, but it is an error in the past. I can not un-do it. Anyway I do not see how it relates to this new issue you describe about "all the strings of that website respectively the string IDs are mixed up together?" Maybe it does, but (a) I'm not sure how and (b) I wouldn't be able to go back and un-do that decision anyway.

There is NO new issue. Point.

> Some suggestions: If you think the repository is good, but weblate is bad, you could create a new "website 2" project, do a fresh checkout from git, and see if that is any better.

> If you think the information in weblate is good, but the website is bad, then please explain this and if you can't find the problem in weblate I will try looking.

> If you think there is an old version of the website that is good, but it is not in current master or in weblate, you could look through Git (I can help you if you want) for older versions of the committed files. I would have to know exactly what you are looking for though.

> Or I could provide more files from the 14 June backup, but maybe you are saying that backup does not have information you need?

> What more information do you need to solve this issue?

> Well I'd need to know all of the things I keep asking you for. I still haven't gotten an initial bug report.

>>> I migrated this project from another location inside weblate
>> This is exactly what I want to know in order to find a solution on my own. Where is that location inside Weblate?

> The new project is: https://weblate.taler.net/projects/gnu-taler/taler-website/. I think it is the only one you used? The old one was for testing only, and was labeled "DO NOT USE" and the migration was really a fresh setup and fresh check-out from git.taler.net, not an internal weblate migration.

>> Moreover, in Weblate I am still displayed as the "administrator" of the project GNU Taler and of 5 components. I want to administrate the Terms of Services component only. Maybe you could be so kind to solve that issue, too.

> Sorry I think I was not clear earlier. I do not know of user levels that are this specific, and you said you need "Manager" status. I do not know weblate so it's possible I will discover something in the future that allows for this level of control, but at the moment I think a person is either a Manager for a Project (GNU Taler) and therefore all of the Components under that project, or one is not a Manager for any Components of a Project. Maybe this is a feature request you could make from Weblate if you wish it.

I have a 'Manager' status already. The 'Manager' status is eligible for opening and deploying components. I don't want to be an 'Administrator' of any project nor component. The persons to change a status in Weblate are: You, Christian, and Florian. One of these three person had to apply the status change for my account from 'user' to 'Manager'.

Now, please take a look at https://weblate.taler.net/projects/gnu-taler/taler-website/de/ and search for the "original translation file" to download.
Then take a look at the English original file on https://weblate.taler.net/projects/gnu-taler/taler-website/en/.

Both files contain a string like: GNU Taler must be https://www.gnu.org/philosophy/free-sw.html. For merchants, our Free Software reference implementation prevents vendor lock-in. As the software of the payment provider itself is free, countries can deploy the payment system without compromising sovereignty.

This paragraph was displayed on the former websites.

Now, we read on https://taler.net/de/principles.html instead: GNU Taler must be Free/Libre Software. For merchants, Free/Libre Software prevents vendor lock-in meaning merchants can easily choose another service provider to process their payments. For countries, Free/Libre software means GNU Taler can not compromise sovereignty by imposing restrictions or requirements. And for exchange operators, transparency is crucial to satisfy Kerckhoff's principle and to establish public confidence.

Where the heck is the file that contains that paragraph?

Kind regards
Stefan
(0016290)
Stefan   
2020-06-17 11:20   
The whole issue started at the time when the English phrases on the English Taler website where changed. Not my case, I did not touch anything. Does this help a bit?

Kind regards
Stefan
(0016294)
buckE   
2020-06-18 05:36   
> I cannot find the file where the strings for the website are stemming from.

But you haven't (as of this part of your message; but see below) told me where you've searched, or what strings you're searching for so I can't really help (again, see below).

> This is of course not anyone's fault, especially not yours.

Thanks, though even if it *was* my fault, that is okay. Sometimes things are my fault. I just want to know how I can help you solve the problem.

> I have a 'Manager' status already. The 'Manager' status is eligible for opening and deploying
> components.

Also, I think for administering project.

> I don't want to be an 'Administrator' of any project nor component.

Then as I say again and again, I must remove your "Manager" status. If my understanding of weblate is correct, you must choose between these two options.

> The persons to change a status in Weblate are: You, Christian, and Florian. One of these three
> person had to apply the status change for my account from 'user' to 'Manager'.

Yes, and any one of us can remove it if you wish. You will then not be able to open and deploy components. Do you want me to do this? Or do you want to continue to be an Administrator (a.k.a "Manager"?)

Note that you are not listed as a "Superuser," which Christian and I are. (Maybe Florian too, I don't remember.)

> Where the heck is the file that contains that paragraph?

Ah! With this part of your message, I have the first indications of what you are having trouble with. It's not as good as if you'd followed my template/questions but it's great! Okay I found a phrase from that paragraph in:

www/locale/fr/LC_MESSAGES/messages.po:"Taler can not compromise sovereignty by imposing restrictions or "
www/locale/en/LC_MESSAGES/messages.po:"Taler can not compromise sovereignty by imposing restrictions or "
www/locale/pt/LC_MESSAGES/messages.po:"Taler can not compromise sovereignty by imposing restrictions or "
www/locale/it/LC_MESSAGES/messages.po:"Taler can not compromise sovereignty by imposing restrictions or "
www/locale/es/LC_MESSAGES/messages.po:"Taler can not compromise sovereignty by imposing restrictions or "
www/locale/de/LC_MESSAGES/messages.po:"Taler can not compromise sovereignty by imposing restrictions or "
www/locale/ru/LC_MESSAGES/messages.po:"Taler can not compromise sovereignty by imposing restrictions or "

Do you have a local copy of the repository that is up to date? I think you will find this paragraph there.

Christian told me in an e-mail today that he checked in English translations to the repository manually. I don't know how they got to the other languages though.

> The whole issue started at the time when the English phrases on the English Taler website where
> changed. Not my case, I did not touch anything. Does this help a bit?

It would really help if you could be more specific, like a link to a git checkin or a date and who did this, but like I said above, I think I happen to know the answer this time. So it's all Christian's fault. :) Okay, joking. Obviously adding translations should not cause a problem.

But basically this all points to the problem being that weblate is not interacting with the repository correctly. Maybe since the upgrade to 4.1, maybe something from before then. We are not sure, but I am working on that problem already.

Also note that we had a prior issue with weblate interacting with the repository, which I solved (I thought) by doing a local reset. I don't *think* the two issues are related but really I'm guessing. And investigating.

I assume you have seen that weblate has had trouble communicating with the repository, right?
https://weblate.taler.net/projects/gnu-taler/taler-website/#repository

So until that is resolved, anything added to the master repo will not be in weblate (like Christian's manually-checked-in English changes), and anything in weblate will not be in the master repo (like your German ones).

One suggestion: If we are in a hurry, we could temporarily stop using weblate and only do manual check-ins to the repository. I would ask you to ask Christian, and then I will send an e-mail out to everyone with this note. Then when we find a fix, we reset and re-update weblate, YOU verify it is up to date, and then we tell everyone it is time to try weblate again.

But that's complicated and I prefer to just wait for the weblate dev to get us through some new 4.1 bugs that are preventing me from even debugging this error.

Small note: until this most recent message from you, I had basically no indication what you were having trouble with. If you can use the template/questions I provided for future reports, I think we will get to solutions much faster.
(0016299)
Stefan   
2020-06-18 10:03   
Dear Buck,

Christian's latest commit on www.git for Taler's website yesterday around noon brought the solution for us, I assume. Now I know the place in the tree structure where the data should have been - as I expected. I pulled the messages.po files to my local repo as I always did before and could investigate all the strings. Fine.

To my mind's eyes, we are not so much in a hurry to update the German website. So I will keep my fingers off the translation topic until Weblate will function as it should. Please be patient with yourself, this application is pretty complex ;-)

As soon as I will see the Taler's website project in Weblate again, I will dedicate some time for restoring the German strings. Now it seems that everything is working fine. No data has been lost, I have the missing strings in my backups here.

I am happy that we found a way solving our bug. At least, I can see light at the end of the tunnel. Thank you very much for your appreciated work!

Kind regards
Stefan
(0016315)
buckE   
2020-06-19 08:27   
Stefan, that's great. I'm glad nothing was lost.

Weblate 4.1.1 should be out today. I will upgrade tomorrow. *Maybe* that will have a fix. Maybe it will make things worse. We will find out.
(0016327)
buckE   
2020-06-20 02:30   
Stefan, they fixed one bug but there is still another that prevents me from debugging. I think maybe that since 4.1 upgrade, weblate is not correctly signing commits with the GPG key. But I don't know.
(0016332)
buckE   
2020-06-22 03:25   
I am going to reset the local weblate repository again. Please make sure you have your translations backed up.
(0016348)
buckE   
2020-06-24 08:10   
I think this is fixed. There was a weblate bug where it was not signing commits with the GPG key. Please go back to using Weblate, but please keep backups of all translations in case I have to reset again. Assigning to you so you see it. Please mark resolved if you agree it is working.
(0016349)
Stefan   
2020-06-24 08:16   
Hi Buck,

Thank you very much for your work. I am going to check the translations and Weblate's behavior as soon as I get back to it. Meanwhile there are some other topics to be solved on my side...

Have a good time!
Kind regards
Stefan


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6406 [libeufin] nexus minor have not tried 2020-06-22 15:56 2020-06-23 15:34
Reporter: MS Platform:  
Assigned To: MS OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Check facade type upon facade creation
Description: Nexus now allows any value to be specified as the facade's type. Instead, it should check that the given type belongs to a allowed types set.

For the moment, only "taler-wire-gateway" is allowed as the facade type.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016344)
MS   
2020-06-23 15:34   
fixed here: 838143b..1040812


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6404 [libeufin] other minor have not tried 2020-06-22 12:03 2020-06-22 12:03
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: ISO20022 camt mapping is not documented, incomplete and incorrect in some places
Description: See summary.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6401 [GNUnet] GNS feature always 2020-06-21 22:18 2020-06-21 22:18
Reporter: schanzen Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version: Git master  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version: 0.14.0  
Summary: Filter DNS2GNS answers in some cases
Description: Currently, dn2gns does not filter by queried record type.
We should probably filter *unless* the client is _not_ on loopback and the resulting message would be too big for a single UDP packet.

Here, we should ideally _also_ check the (E)DNS settings and if (E)DNS is not specified, stick to the 512-byte maximum (and first filter only DNSSEC Records if (E)DNS is not specified _and_ the reply gets too big).
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6382 [libeufin] nexus minor have not tried 2020-06-15 08:47 2020-06-21 16:06
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: support transaction fetching since last successful bank-side creation date/time
Description: Thus we can request transaction history without any gaps.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016330)
Florian Dold   
2020-06-21 16:06   
A rudimentary version of this is implemented, however the EBICS-related timezone mess still needs to be sorted out.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6400 [libeufin] other minor have not tried 2020-06-21 14:18 2020-06-21 14:18
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: consider doing faster re-tries for some scheduled operations
Description: When the daily bank statement download fails, should LibEuFin retry before the next scheduled execution?

For most tasks, a faster re-try doesn't make sense though.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6256 [libeufin] other minor have not tried 2020-05-24 09:50 2020-06-20 08:52
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: automated scheduling of operations
Description: Certain operations (fetching bank account history, balance, payment status reports) should be done regularly and automatically.

We need to think about how to integrate this into LibEuFin conceptually and technically.

As a first step, maybe the TWG facade can do this scheduling? But we might also want to have this for bank accounts directly.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016329)
Florian Dold   
2020-06-20 08:52   
Implemented now, with a generic scheduling mechanism that's currently only available for accounts.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6389 [libeufin] nexus minor have not tried 2020-06-16 13:30 2020-06-20 08:46
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: TWG ingestion should not be triggered separately, but automatically follow account transaction ingestion
Description: See summary.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0016328)
Florian Dold   
2020-06-20 08:46   
Done, it's now automatically triggered after ingestion.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6399 [libeufin] other minor have not tried 2020-06-19 14:19 2020-06-19 14:19
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: review and document date/time/timezone handling
Description: Especially for EBICS order params.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5695 [Taler] other minor have not tried 2019-04-19 15:43 2020-06-19 09:42
Reporter: Marcello Stanisci Platform:  
Assigned To: Marcello Stanisci OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: talerconfig.py too much susceptible to missed files / directories
Description: The Python talerconfig "copylib" module uses to crash when it fails to find the directory for defaults.
Instead it should just move on and look - maybe print a WARNING - for the next "element".

  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/marcello/prog/taler/bank/talerbank/settings.py", line 24, in <module>
    TC = TalerConfig.from_file(os.environ.get("TALER_CONFIG_FILE"))
  File "/home/marcello/prog/taler/bank/talerbank/talerconfig.py", line 348, in from_file
    cfg.load_defaults()
  File "/home/marcello/prog/taler/bank/talerbank/talerconfig.py", line 409, in load_defaults
    self.load_dir(os.path.join(prefix, "share/taler/config.d"))
  File "/home/marcello/prog/taler/bank/talerbank/talerconfig.py", line 436, in load_dir
    files = os.listdir(dirname)
NotADirectoryError: [Errno 20] Not a directory: '/dev/null/share/taler/config.d'
Makefile:882: recipe for target 'check' failed
make: *** [check] Error 1
Tags:
Steps To Reproduce: Checkout the bank.git, then

./bootstrap

./configuure --prefix=/dev/null

make check
Additional Information:
Attached Files:
Notes
(0015879)
buckE   
2020-05-11 09:09   
Actually what happens is:

$ ./configure --prefix=/dev/null
./configure: unrecognized option '--prefix=/dev/null'
(0016322)
buckE   
2020-06-19 09:42   
Please confirm this is a current bug. I have this:


~/code/bank$ ./configure --prefix=/dev/null
./configure: unrecognized option '--prefix=/dev/null'


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5984 [Taler] deployment and operations minor have not tried 2019-12-03 13:30 2020-06-19 08:36
Reporter: Florian Dold Platform:  
Assigned To: Florian Dold OS:  
Priority: normal OS Version:  
Status: assigned Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: usability of deployment.git must be improved
Description: In order to upgrade an environment, currently we have to run an arcane set of commands in the right order.

* When taler-deployment-sign is run before taler-deplyoment-keyup, a new and different key is created for the exchange. That's not really acceptable.

* We frequently run info permissions problems on the demo-blue/demo-green split. The corresponding deployment scripts should make sure that permissions are correct.

* The following "tools" should probably be folded into one command:
  (1) taler-deployment-config-generate
  (2) taler-deployment-keyup
  (3) taler-deployment-sign
  (4) taler-deployment-hier (what does this one even do?)
  (And what does taler-deployment-prepare even do? It looks like some unholy amalgamation of the previous commands *plus* running a re-build)

* scripts for taler-deployment-start should probably *also* run some checks after it finishes and print the output of "taler-deployment-arm -I", so it is harder to forget checking manually
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0015382)
Christian Grothoff   
2020-02-18 23:27   
It seems the folding of the commands has been partially done with the creation of taler-deployment-prepare. However, more needs to be done here.

I also find it confusing that some commands to be used are taler-deployment COMMAND and others are taler-deployment-COMMAND. If all commands to be actually (primarily) used were of the form "taler-deployment COMMAND" and all helper-scripts were "taler-deployment-helper-COMMAND", that might reduce confusion and improve consistency. And we should document each command and each helper script in the developer manual.
(0015383)
Christian Grothoff   
2020-02-18 23:33   
> However it's not entirely clear what should happen or what should be
> done if, say, you change the envcfg.py. Do you have to re-boostrap? Or
> should "taler-deployment build" handle this? What if I want to not lose
> my local modifications? Should "taler-deployment build" some option to
> either fetch repos or not? Or update from envcfg.py? Etc ...

Right now, the docs on 'updating' say that you have to
$ rm -rf ~/sources ~/local.

That seems both very expensive (lots of compile time) and prevents any kind of "not lose my local modifications".

In general, I would argue that local modifications are just bad and should simply not be done: if the deployment.git is used, ALL modifications belong into the scripts, we do have cases for the different deployments in the scripts, so users MUST update the scripts (otherwise we have no versioning and just chaos).

However, doing incremental builds (git pull, make install over the existing installation) and preserving existing *KEYS* and databases should be the default and supported during upgrades, so no more "$ rm -rf" and while the configuration is re-generated, the keys should be kept/migrated *unless* a specific version jump breaks something badly, and then that has to be in the instructions for updating to that version. Doing the full
bootstrap, activate (missing in docs!), build, prepare, restart sequence seems fine, as deployment.git obviously needs to be updated (bootstrap), code needs to be build (gcc) and possibly databases/key material needs to be updated/migrated/etc.
(0016318)
buckE   
2020-06-19 08:36   
Florian, when you're ready please respond to Christian's comments. Also I would like to know:

- what do you mean by "upgrade an environment" in general? What sort of environment?

- What is the full and exact current process?

- What is the general process you would like to see?

 - which scripts are you referring to? Are these in some private script repo?

- Is this something that should be done by buildbot? Or unrelated?


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6397 [libeufin] other minor have not tried 2020-06-18 22:13 2020-06-18 22:13
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: write a good README(.md) file for LibEuFin
Description: See summary.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6396 [libeufin] other minor have not tried 2020-06-18 22:09 2020-06-18 22:09
Reporter: Florian Dold Platform:  
Assigned To: OS:  
Priority: l