View Issue Details

IDProjectCategoryView StatusLast Update
0008205Talerlibeufin-nexuspublic2024-02-12 18:09
ReporterAntoine A Assigned ToAntoine A  
PrioritynormalSeverityminorReproducibilityN/A
Status resolvedResolutionfixed 
Target Version0.9.4 
Summary0008205: Consider merging the fetch and submit commands
DescriptionCurrently the submit command submits pending transactions without knowing if they succeed, and the fetch command just fetches the final states of the transactions without tracking the status of the payments.

We need to fetch incoming transactions if we're going to bounce malformed payments and we need to fetch to check if the submit was successful, so I don't see any reason to separate the two commands. Having a single command will simplify the deployment and administration of libeufin-nexus.
Additional InformationI plan to use the following logic:
- Retrieve incoming transactions and the final state of outgoing transactions (camt.054 and camt053?)
    - Fail fast if EBICS download failed due to HTTP or protocol failure
    - Ingest incoming transactions, boucing malformed transactions
    - Update the status of outgoing transactions, defining their final state
- Retrieve CustomerPaymentStatusReport and HAC logs (pain.002)
    - Fail fast if EBICS download failed due to HTTP or protocol failure
    - Update the status of outgoing transactions, defining their final state in the event of failure
- Submit pending transactions in batch in a single file (pain.001)
    - Fail fast if EBICS download failed due to HTTP or protocol failure
- Retrieve HAC logs in a loop for fast feedback, wait until all pending orders have failed or been confirmed, or until a delay has elapsed
TagsNo tags attached.

Relationships

related to 0006258 assignedAntoine A Track outgoing transaction status 
child of 0008273 assignedChristian Grothoff package and upload libeufin 0.9.4 to ftp and stable Debian/Ubuntu server 

Activities

Christian Grothoff

2024-01-22 16:45

manager   ~0020956

One reason to separate them is that in some emergencies, you may want one but not the other. For example, when going out of business, you might disable 'fetch'. Or when there is a security issue, you may want to disable 'submit' but not 'fetch'. But, I guess this could be done with --disable-submit or something similar, that hacks into your main event loop.

Another reason might eventually be performance: running multiple services (possibly on multiple hosts) to do the respective jobs (possibly in parallel).

A third reason might be credentials and minimization of attack surface. Given separate credentials for submit and fetch, the code of the fetch job would have no access to the submit credentials.

More critically, if one direction fails, that probably shouldn't block processing the other direction. Like if I cannot wire funds to merchants right now, that should NOT block customers from withdrawing funds! (And vice versa.).

Finally, it is usually good to simply do one EBICS interaction, then one matching DB interaction, or one DB interaction and then one matching EBICS interaction (and ideally no subsequent DB interaction!). Doing a long sequence to me just suggests many more things that could go wrong. In fact, I'd rather see many independent different fetch jobs (camt53, camt.54, pain.02, HAC logs ...) and possibly even different submit jobs (pain.01, ???) each with very simple logic and very simple failure scenarios (and clear logs) than one complex state machine. Using DB notify/subscribe it should also be easy to trigger just the right one(s), so for example after submitting the pain01, you could do a DB notify to trigger the job that fetches the pain02.

None of these are "hard" reasons not to do this, but please consider each when redesigning the system.

Antoine A

2024-01-30 13:55

developer   ~0021092

We have chosen not to merge the commands.
We will keep `libeufin-nexus ebics-submit` to submit all current transactions and `libeufin-nexus ebics-fetch` to fetch all documents. Additional subcommands will be added to `ebics-fetch` to allow you to choose which document to fetch.
We can close this issue when libeufin-nexus has its own debian target so that we can add new services for fetch sub-commands in the future.

Issue History

Date Modified Username Field Change
2024-01-22 16:15 Antoine A New Issue
2024-01-22 16:15 Antoine A Status new => assigned
2024-01-22 16:15 Antoine A Assigned To => Antoine A
2024-01-22 16:15 Antoine A Relationship added related to 0006258
2024-01-22 16:18 Antoine A Additional Information Updated
2024-01-22 16:45 Christian Grothoff Note Added: 0020956
2024-01-22 17:55 Christian Grothoff Target Version 0.9.4 => 0.9.5
2024-01-24 15:23 Antoine A Target Version 0.9.5 => 0.9.4
2024-01-27 01:09 Christian Grothoff Relationship added child of 0008273
2024-01-30 13:55 Antoine A Note Added: 0021092
2024-02-12 18:09 Antoine A Status assigned => resolved
2024-02-12 18:09 Antoine A Resolution open => fixed