View Issue Details

IDProjectCategoryView StatusLast Update
0009154Talerexchange KYC SPApublic2024-11-01 18:34
ReporterChristian Grothoff Assigned ToFlorian Dold  
PrioritynormalSeverityfeatureReproducibilityN/A
Status assignedResolutionopen 
Platformi7OSDebian GNU/LinuxOS Versionsqueeze
Product Versiongit (master) 
Target Version0.14 
Summary0009154: generic form
DescriptionWe need a generic form in the KYC SPA where the AML officer can provide basically arbitrary instructions via the context.
I'm thinking of something where the context can specify:

* The instructions to show ("instructions":"TEXT")
* The fields to show ("$KEY": { "type" : "radiobutton", "choices":{"cat","dog","elephant"}, "default":"dog"})

Initially I'd go for some of the very common HTML form fields, like "text" (line of text), "radiobutton", "checkbox" and "upload", plus maybe some super-common bank-specific types like "password" (with blind double-entry), "date", "address" (simple spec for complex form) and "country" (searchable drop-down list). Maybe "text" should have options to constrain the input ("numeric").

This will be useful for any deployment (definitively needed for TOPS) as it is the ultimate fall-back for AML staff.
TagsNo tags attached.

Relationships

parent of 0009252 assignedFlorian Dold fail to trigger second kyc, wallet core should move tx to kyc state 
Not all the children of this issue are yet resolved or closed.

Activities

Christian Grothoff

2024-10-06 23:58

manager   ~0023458

What is the estimate on how long this will still take?

sebasjm

2024-10-07 05:47

developer   ~0023460

it's getting hard to test https://bugs.gnunet.org/view.php?id=9252

right now i'm trying this scenario
1.- trigger kyc
2.- officer mark as good for now
3.- wallet complete withdraw
4.- officer ask for more information
5.- wallet tries to withdraw more
6.- wallet is redirected to the custom form
7.- complete custom form
8.- officer allows or freeze account based on values

I can't reach step 6 due to 0009252

in step (4) officer is using measure "ask" but unable to trigger the custom form in the kyc spa yet

[kyc-measure-ask]
CHECK_NAME = askContext
CONTEXT = {}
PROGRAM = customForm

[kyc-check-askContext]
TYPE = FORM
VOLUNTARY = NO
FORM_NAME = from_context
DESCRIPTION = "Use the next form to answer what officer ask"
OUTPUTS = all_info
FALLBACK = gatheringProblem

[aml-program-customForm]
COMMAND = aml-custom-form.sh
DESCRIPTION = "officer gatther info"
ENABLED = YES
FALLBACK = programProblem

Christian Grothoff

2024-10-08 00:03

manager   ~0023477

Sebastian: are you now unblocked? What is the ETA on this one?

sebasjm

2024-10-08 13:59

developer   ~0023478

No, i'm still blocked by 0009252. I need an environment where I can run this scenario.

In the AML Spa the officer is setting the custom form in the properties of the new measure.
This needs to be picked up by the KYC SPA as the form that is going to be shown.
For that I have a generic dynamic form which i'm not able to test (step 7).

If I have an working environment where I can make all the 8 steps (but without the custom form) I think I can make this in one day of two.
I should be easy but I'm still having some doubts on how the properties of the measure (set by the officer who wants to ask for more information) is going to communicate the form to the KYC spa.
This is more testing intensive.

Christian Grothoff

2024-10-08 16:53

manager   ~0023479

Why can't you do this much simpler scenario:
1.- trigger kyc (say wallet-balance too high, or withdraw limit, or whatever)
2.- officer ask for more information
3.- wallet is redirected to the custom form
4.- complete custom form
5.- officer allows or freeze account based on values

sebasjm

2024-10-08 18:02

developer   ~0023480

AFAIK the officer don't see the account in the AML spa in step (2) but I will try to simplify this

sebasjm

2024-10-08 18:51

developer   ~0023482

update: I can make this by using the "search account" feature after step (1)... that is making a decision for an account that the exchange doesn't yet knows. It is useful as a workaround.

sebasjm

2024-10-10 17:17

developer   ~0023504

Last edited: 2024-10-10 17:23

1.- can't read the context in AML programs https://bugs.gnunet.org/view.php?id=9254

while that's being fixed (i can workaround it by creating a static form instead a officer defined one) i'm blocked by this

2.- context if static based on the measure configuration

after update docs.git :: f1fa1193..9ab882ea the `kyc-info` can returns the `context` value but there is no way to update it from the AML Program

so:
- aml officer can set the form configuration in `properties: AccountProperties` part of `aml/$OFFICER_PUB/decision`
- aml program can read context and properties part of AmlProgramInput
- KYC SPA customer can read `context?: Object` part of `kyc-info/$ACCESS_TOKEN`

but there is no way for the aml program to use the input to build the form that the KYC SPA will use.

maybe we can have a new_context in AmlOutcome and if it's not undefined then replace it all

Christian Grothoff

2024-10-11 22:28

manager   ~0023523

Use custom measures, define context as part of the measure.

sebasjm

2024-10-15 03:40

developer   ~0023528

Measure with CHECK_NAME = SKIP fail, I made a integration test

1. creates withdrawal which triggers measure m1
2. check kyc-info endpoint (ok)
3. officer makes decision with measure m3 (which have check = skip)
4. check kyc-info endpoint and fail (http status 500)

expected `kyc-info` to return the result of the program associated with measure m3

can be reproduce with

# taler-harness run-integrationtests kyc-skip-expiration

sebasjm

2024-10-15 16:52

developer   ~0023529

I think that the problems is that when a officer post a decision with a Measure that contains a CHECK_NAME = SKIP I expect the program to run instantly and as far as i'm reading the code legitimization_check_run is only called when an KYC event is triggered

so it's also need to be called when the officer makes a decision (to handle the case when the measure being posted is an instant measure)

Christian Grothoff

2024-10-15 19:10

manager   ~0023532

Yes, it *should* run instantly, likely something the code simply doesn't check for yet on AML decisions.

Christian Grothoff

2024-10-15 19:11

manager   ~0023533

@florian: it would be great if you could look into this one...

Florian Dold

2024-10-21 15:29

manager   ~0023550

commit a7cb67227df85d63c9433981358dea406fd0346c
Author: Florian Dold <florian@dold.me>
Date: Mon Oct 21 15:06:28 2024 +0200

    run immediate measures on POSTing AML decision


=> Part of this is now fixed, what's missing is that /kyc-info should block while the instant measures are running.

Florian Dold

2024-10-21 16:26

manager   ~0023551

We don't do long-polling for that yet, but return 202 Accepted when the measure program is still running.

sebasjm

2024-10-23 15:30

developer   ~0023571

i'm getting http 500 with this log (below) it may be my script but checking

Oct 23 10:23:12 taler-exchange-httpd[2367016]: }(MG8BFTEGTGHE4XT2Z9G36EG88G) WARNING External protocol violation detected at json_helper.c:443.
Oct 23 10:23:12 taler-exchange-httpd[2367016]: (MG8BFTEGTGHE4XT2Z9G36EG88G) WARNING Parsing failed for field `context:2`
Oct 23 10:23:12 taler-exchange-httpd[2367016]: (MG8BFTEGTGHE4XT2Z9G36EG88G) WARNING External protocol violation detected at kyclogic_api.c:681.
Oct 23 10:23:12 taler-exchange-httpd[2367016]: (MG8BFTEGTGHE4XT2Z9G36EG88G) ERROR AML program output is malformed at `new_rules'
Oct 23 10:23:12 taler-exchange-httpd[2367016]: (MG8BFTEGTGHE4XT2Z9G36EG88G) ERROR Assertion failed at taler-exchange-httpd_common_kyc.c:962.
Oct 23 10:23:12 taler-exchange-httpd[2367016]: (MG8BFTEGTGHE4XT2Z9G36EG88G) ERROR Assertion failed at taler-exchange-httpd_common_kyc.c:216.

sebasjm

2024-10-23 16:52

developer   ~0023572

the problem seems to be that exchange expect an array instead of an object for the context field.
I tried both
1.- modifying my check script to produce an array in context (despite the specs says context is Object)
2.- modifying the exchange to accept an Object instead of an array

in both cases I get the same error below.

So I'm going to push a patch for (2) and try to fix this new error


Oct 23 11:49:59 taler-exchange-httpd[2415855]: (CRM15RAP62N3JQK36RXZEV6YGR) ERROR Query `insert_kyc_attributes' failed with result: insert or update on table "kyc_attributes" violates foreign key constraint "kyc_attributes_foreign_legitimization_processes"/Key (legitimization_serial)=(0) is not present in table "legitimization_processes"./ERROR: insert or update on table "kyc_attributes" violates foreign key constraint "kyc_attributes_foreign_legitimization_processes"
Oct 23 11:49:59 taler-exchange-httpd[2415855]: DETAIL: Key (legitimization_serial)=(0) is not present in table "legitimization_processes".
Oct 23 11:49:59 taler-exchange-httpd[2415855]: CONTEXT: SQL statement "INSERT INTO kyc_attributes
Oct 23 11:49:59 taler-exchange-httpd[2415855]: (h_payto
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,collection_time
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,expiration_time
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,encrypted_attributes
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,legitimization_serial
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,trigger_outcome_serial
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ) VALUES
Oct 23 11:49:59 taler-exchange-httpd[2415855]: (in_h_payto
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,in_collection_time_ts
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,in_expiration_time_ts
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,in_enc_attributes
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,in_process_row
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,my_trigger_outcome_serial)"
Oct 23 11:49:59 taler-exchange-httpd[2415855]: PL/pgSQL function exchange_do_insert_kyc_attributes(bigint,bytea,integer,text,text,text,bigint,bigint,bigint,text,text,text[],bytea,boolean,text) line 30 at SQL statement
Oct 23 11:49:59 taler-exchange-httpd[2415855]: /PGRES_FATAL_ERROR/ERROR: insert or update on table "kyc_attributes" violates foreign key constraint "kyc_attributes_foreign_legitimization_processes"
Oct 23 11:49:59 taler-exchange-httpd[2415855]: DETAIL: Key (legitimization_serial)=(0) is not present in table "legitimization_processes".
Oct 23 11:49:59 taler-exchange-httpd[2415855]: CONTEXT: SQL statement "INSERT INTO kyc_attributes
Oct 23 11:49:59 taler-exchange-httpd[2415855]: (h_payto
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,collection_time
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,expiration_time
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,encrypted_attributes
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,legitimization_serial
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,trigger_outcome_serial
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ) VALUES
Oct 23 11:49:59 taler-exchange-httpd[2415855]: (in_h_payto
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,in_collection_time_ts
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,in_expiration_time_ts
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,in_enc_attributes
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,in_process_row
Oct 23 11:49:59 taler-exchange-httpd[2415855]: ,my_trigger_outcome_serial)"
Oct 23 11:49:59 taler-exchange-httpd[2415855]: PL/pgSQL function exchange_do_insert_kyc_attributes(bigint,bytea,integer,text,text,text,bigint,bigint,bigint,text,text,text[],bytea,boolean,text) line 30 at SQL statement
Oct 23 11:49:59 taler-exchange-httpd[2415855]: (CRM15RAP62N3JQK36RXZEV6YGR) INFO PG poll job active
Oct 23 11:49:59 taler-exchange-httpd[2415855]: (CRM15RAP62N3JQK36RXZEV6YGR) INFO PG poll job finishes after 0 events
Oct 23 11:49:59 taler-exchange-httpd[2415855]: (CRM15RAP62N3JQK36RXZEV6YGR) INFO Stored encrypted KYC process #0 attributes: -2
Oct 23 11:49:59 taler-exchange-httpd[2415855]: (CRM15RAP62N3JQK36RXZEV6YGR) ERROR Assertion failed at taler-exchange-httpd_common_kyc.c:327.

Florian Dold

2024-10-28 12:22

manager   ~0023607

@sebasjm: Please provide a way to reproduce this, then I can look into fixing it.

(Cause: Running instant measures is wrongly trying to insert attributes when we have no legitimization process.)

sebasjm

2024-11-01 18:34

developer   ~0023639

Now that we have implemented aml programs in integration test we can track this issue with

taler-harness run-integrationtests kyc-skip-expiration

currently the exchange replies 202, i'm investigating why it doesn't progress to 200

Issue History

Date Modified Username Field Change
2024-09-02 19:42 Christian Grothoff New Issue
2024-09-02 19:42 Christian Grothoff Status new => assigned
2024-09-02 19:42 Christian Grothoff Assigned To => sebasjm
2024-10-06 23:58 Christian Grothoff Note Added: 0023458
2024-10-07 05:47 sebasjm Note Added: 0023460
2024-10-07 10:41 Christian Grothoff Relationship added parent of 0009252
2024-10-08 00:03 Christian Grothoff Note Added: 0023477
2024-10-08 13:59 sebasjm Note Added: 0023478
2024-10-08 16:53 Christian Grothoff Note Added: 0023479
2024-10-08 18:02 sebasjm Note Added: 0023480
2024-10-08 18:51 sebasjm Note Added: 0023482
2024-10-10 17:17 sebasjm Note Added: 0023504
2024-10-10 17:22 sebasjm Note Edited: 0023504
2024-10-10 17:22 sebasjm Note Edited: 0023504
2024-10-10 17:23 sebasjm Note Edited: 0023504
2024-10-10 17:23 sebasjm Note Edited: 0023504
2024-10-10 17:23 sebasjm Note Edited: 0023504
2024-10-11 22:28 Christian Grothoff Note Added: 0023523
2024-10-15 03:40 sebasjm Note Added: 0023528
2024-10-15 03:40 sebasjm Assigned To sebasjm => Christian Grothoff
2024-10-15 16:52 sebasjm Note Added: 0023529
2024-10-15 19:10 Christian Grothoff Note Added: 0023532
2024-10-15 19:10 Christian Grothoff Assigned To Christian Grothoff => Florian Dold
2024-10-15 19:11 Christian Grothoff Note Added: 0023533
2024-10-21 15:29 Florian Dold Note Added: 0023550
2024-10-21 16:25 Florian Dold Assigned To Florian Dold => sebasjm
2024-10-21 16:26 Florian Dold Note Added: 0023551
2024-10-23 15:30 sebasjm Note Added: 0023571
2024-10-23 16:52 sebasjm Note Added: 0023572
2024-10-28 12:22 Florian Dold Note Added: 0023607
2024-10-28 15:18 Florian Dold Assigned To sebasjm => Florian Dold
2024-11-01 18:34 sebasjm Note Added: 0023639