View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0005855Talerotherpublic2019-08-27 15:162021-08-24 16:23
ReporterFlorian Dold Assigned ToMarcello Stanisci  
PrioritynormalSeverityminorReproducibilityhave not tried
Status closedResolutionfixed 
Target Version0.8Fixed in Version0.8 
Summary0005855: get rid of copylib in favor of using real python packaging
DescriptionWe can simply upload a taler helper package to the python package archive. There is no need to copy around code ...
TagsNo tags attached.

Activities

nikita

2019-09-18 15:25

reporter   ~0014913

this is basically copylib -> packaging -> pypi; adjust code which now copies around code to pypi install that pypi file.
correct me if I missed a detail, I can do this.

Florian Dold

2019-09-19 11:03

manager   ~0014922

Yes. The library should probably be named something like "taler-util".

Then I should be able to do something like

  import taler.util.Amount as Amount

(For a technical explanation of namespaces, see this <https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages>. We also have other taler python packages, but we might wanna have them under the same "taler" namespace prefix.)

nikita

2019-09-19 17:42

reporter   ~0014926

Okay, thanks.

nikita

2019-09-24 09:47

reporter   ~0014931

So about the kleingedrucktes TODO item point:

is the identification for author and copyright right?
see https://git.taler.net/taler-util.git/tree/setup.py

Florian Dold

2019-09-24 09:58

manager   ~0014932

I'd put "Utility library for GNU Taler" as a description, as it's not really related to any build process!

Generally we want to keep version numbers roughly consistent between components. Thus the version should be 0.6.0rc1 (a pre-release version, see https://www.python.org/dev/peps/pep-0440/#pre-releases). When we release 0.6, it should be bumped to "0.6.0".

When you upload to PyPI, could you please also add Marcello and me as a maintainer?

nikita

2019-09-24 10:39

reporter   ~0014933

Last edited: 2019-09-24 10:41

 > When you upload to PyPI, could you please also add Marcello and me as a maintainer?

if it's not clear from within pypi (I have an account), can you tell me your names on pypi?
I also assume from this statement that you want to handle the uploading of wheel files, correct?

Florian Dold

2019-09-24 10:48

manager   ~0014934

Ah, I don't have an account there yet, neither does Marcello.

I don't want to handle the upload myself, I assumed you'd put yourself as the owner of the package. If you don't want to do this, we can just use an @taler.net email alias as a "team account" to handle PyPI uploads. But I guess they require a person and not an org to sign up.

I just want Marcello and me to have upload access too to increase the "bus factor", in case you're not able to do uploads for a longer stretch of time, somebody else should be able to do it too.

nikita

2019-09-24 11:02

reporter   ~0014935

Okay, that's more clear and I agree.
I assume pypi is easy, I only signed up ahead of time because I intended to publish some packages I work on myself, I just have to read into the whole process. Looked easy so far, guess we just got terminology mixed up.

Thanks!

nikita

2019-09-24 11:20

reporter   ~0014936

if we have LGPL2.1 code with the "and later", and now LGPL3 is out, some systems consider this
LGPL3 code. should we go with this, or how should we handle these lgpl2.1+ files?

Florian Dold

2019-09-24 11:28

manager   ~0014937

It should indeed be just LGPL3 then, good catch!

nikita

2019-09-24 12:04

reporter   ~0014938

is the distributed wheel then "AGPL3+ AND LGPL3+" or "AGPL3+"? So far I have it as a list ["AGPL3", "LGPL2.1"].
I'm no fan of "or later" license definitions, but if we still want to put it as LGPL3+ it seem lined up with what
Marcello did with LGPL2.1+

Florian Dold

2019-09-24 12:07

manager   ~0014939

Since this is a GNU project, we're going with LGPL3+.

I don't really know how the AGPL got into this. At some point we decided that the helper libraries should be LGPL, not AGPL.

Only components that are services (exchange, merchant, ...) have AGPL.

nikita

2019-09-24 12:07

reporter   ~0014940

I think I have to be more precise: distributors choose version 3 when presented with a choice, but 2.1+ still means as a developer you got to pick any of 2.1 or 3. so unconditionally bumping is not necessary.

nikita

2019-09-24 12:09

reporter   ~0014941

Last edited: 2019-09-24 12:11

 > I don't really know how the AGPL got into this. At some point we decided that the helper libraries should be LGPL, not AGPL.

Okay, can you then do the relicensing, or is it enough for the record if I point to this ticket and your okay? Formally only the copyright holders can relicense, and the agpl3 files are (c) INRIA.
better safe than sorry, even though I know it's minimal and probably no one will care, but I learned to nitpick when it comes to licenses.

nikita

2019-09-24 14:07

reporter   ~0014942

okay, so what I have is ready for release and would push to pypi.org once you commit the license change. assign back to me when you're done.

Florian Dold

2019-09-24 14:10

manager   ~0014943

Could you please just change the license headers yourself? Christian and me both approved the change, and it reflects wrong information (as copyright was assigned to Taler Systems SA) anyway.

nikita

2019-09-24 14:16

reporter   ~0014944

Ah, okay. I thought we needed this additional step because nothing of this was mentioned. I'll do it. Thanks.

nikita

2019-09-24 14:38

reporter   ~0014945

On both NetBSD and Linux I get a reliable test failure. Before I push to pypi, is this something I am supposed to fix (as in: can we ignore this or not?) or will anyone of you get to it?

python3.7 setup.py test
running test
running egg_info
writing taler_util.egg-info/PKG-INFO
writing dependency_links to taler_util.egg-info/dependency_links.txt
writing top-level names to taler_util.egg-info/top_level.txt
writing manifest file 'taler_util.egg-info/SOURCES.txt'
running build_ext
test_force_logfile (tests.log_test.TestGnunetLog) ... ok
test_forced_env_AND_nonforced_env (tests.log_test.TestGnunetLog) ... ok
test_manual_loglevel_AND_forced_env (tests.log_test.TestGnunetLog) ... ok
test_manual_loglevel_AND_nonforced_env (tests.log_test.TestGnunetLog) ... ok
test_no_env_and_no_setup (tests.log_test.TestGnunetLog) ... ok
test_non_forced_env (tests.log_test.TestGnunetLog) ... FAIL
test_only_forced_env (tests.log_test.TestGnunetLog) ... ok
test_only_manual_loglevel_setup (tests.log_test.TestGnunetLog) ... ok
test_add_and_dump (tests.test_amount.TestAmount) ... ok
test_bad_stringification (tests.test_amount.TestAmount) ... ok
test_negative_value (tests.test_amount.TestAmount) ... ok
test_parse_and_cmp (tests.test_amount.TestAmount) ... ok
test_stringify (tests.test_amount.TestAmount) ... ok
test_subtraction (tests.test_amount.TestAmount) ... ok
test_very_big_number (tests.test_amount.TestAmount) ... ok

======================================================================
FAIL: test_non_forced_env (tests.log_test.TestGnunetLog)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/pkg/lib/python3.7/site-packages/mock/mock.py", line 1330, in patched
    return func(*args, **keywargs)
  File "/home/ng0/src/taler/taler-util/tests/log_test.py", line 131, in test_non_forced_env
    mocked_setLevel.assert_called_with(level=logging.ERROR)
  File "/usr/pkg/lib/python3.7/site-packages/mock/mock.py", line 944, in assert_called_with
    six.raise_from(AssertionError(_error_message(cause)), cause)
  File "<string>", line 3, in raise_from
AssertionError: expected call not found.
Expected: setLevel(level=40)
Actual: setLevel(level=20)

----------------------------------------------------------------------
Ran 15 tests in 0.032s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=15 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=15 errors=0 failures=1>

Florian Dold

2019-09-24 14:40

manager   ~0014946

Yes, the tests should all pass before we upload the package.

nikita

2019-09-24 14:55

reporter   ~0014947

Okay, I'm looking into it again.

nikita

2019-09-24 16:06

reporter   ~0014948

It's not recent, this fails in copylib for me as well.

Florian Dold

2019-09-24 16:07

manager   ~0014949

That's possible. If you can't figure out why the test case fails yourself or if you're unsure about the functionality it should provide, please ask Marcello, as he originally wrote the code in copylib.git

nikita

2019-09-24 16:13

reporter   ~0014950

I have been trying to wrap my head around it, but marcello would likely be faster - I found out that this is a part I have to learn about mock.

nikita

2019-09-24 16:16

reporter   ~0014951

Last edited: 2019-09-24 16:16

 Essentially I don't know where this magic(?) linenumber comes from (the 99) or how it relates to the test (which call is not found? in gnunet_logger? do I need any prerequisites running, even though the mock test does not read like I do need more than the python modules and a /tmp?)

nikita

2019-09-24 19:51

reporter   ~0014953

is the expected assert maybe wrong?

git commit 2f9384498f26f382452e41440dd0fa1f45935ea1 and this diff:

diff --git a/tests/log_test.py b/tests/log_test.py
index 268e176..9758476 100755
--- a/tests/log_test.py
+++ b/tests/log_test.py
@@ -128,7 +128,8 @@ class TestGnunetLog(TestCase):
         os.environ["GNUNET_LOG"] = "gnunet-pylog;log_test.py;test_non_forced_env;99;ERROR" # lineno is not 100% accurate.
         gl = GL("gnunet-pylog")
         gl.log("msg", gl.DEBUG)
- mocked_setLevel.assert_called_with(level=logging.ERROR)
+ # mocked_setLevel.assert_called_with(level=logging.ERROR)
+ mocked_setLevel.assert_called_with(level=logging.INFO)
 
     ##
     # This function tests the case where *only* the GNUNET_FORCE_LOG


lead to this test output:

running test
running egg_info
writing taler_util.egg-info/PKG-INFO
writing dependency_links to taler_util.egg-info/dependency_links.txt
writing top-level names to taler_util.egg-info/top_level.txt
writing manifest file 'taler_util.egg-info/SOURCES.txt'
running build_ext
test_force_logfile (tests.log_test.TestGnunetLog) ... ok
test_forced_env_AND_nonforced_env (tests.log_test.TestGnunetLog) ... ok
test_manual_loglevel_AND_forced_env (tests.log_test.TestGnunetLog) ... ok
test_manual_loglevel_AND_nonforced_env (tests.log_test.TestGnunetLog) ... ok
test_no_env_and_no_setup (tests.log_test.TestGnunetLog) ... ok
test_non_forced_env (tests.log_test.TestGnunetLog) ... ok
test_only_forced_env (tests.log_test.TestGnunetLog) ... ok
test_only_manual_loglevel_setup (tests.log_test.TestGnunetLog) ... ok
test_add_and_dump (tests.test_amount.TestAmount) ... ok
test_bad_stringification (tests.test_amount.TestAmount) ... ok
test_negative_value (tests.test_amount.TestAmount) ... ok
test_parse_and_cmp (tests.test_amount.TestAmount) ... ok
test_stringify (tests.test_amount.TestAmount) ... ok
test_subtraction (tests.test_amount.TestAmount) ... ok
test_very_big_number (tests.test_amount.TestAmount) ... ok

----------------------------------------------------------------------
Ran 15 tests in 0.030s

OK

Florian Dold

2019-09-24 21:39

manager   ~0014954

Yeah looks like the assert was wrong.

nikita

2019-09-24 22:36

reporter   ~0014955

Okay, committing the diff.
Thanks.
At least I learned more about mock today :)

nikita

2019-09-25 13:48

reporter   ~0014960

Okay, I thought I was done, but I just checked bank.git.

Next step I'm working on now is syncing the individual copylib forks back into taler-util.
Then I'll do a new release of taler-util (rc2 I guess...),
and then move all copylib users to just use taler-util.

nikita

2019-09-25 13:58

reporter   ~0014961

Last edited: 2019-09-25 16:21

 Not trying to be too chatty about this work, but on further looks I'll just merge what I know can not backfire, then compare the individual changes and get back to you if I see any large changes.

nikita

2019-09-25 14:39

reporter   ~0014962

Last edited: 2019-09-27 18:24

 For myself as reference, remaining:
./django-payments-taler/payments/taler/amount.py
./playground/talerplayground/talerconfig.py [where did this repo disappear to in the last months?]

nikita

2019-09-27 18:33

reporter   ~0014967

what is playground and where did it move to?
I don't have all the repos but this one turned up to contain talerconfig.

Florian Dold

2020-08-21 18:04

manager   ~0016690

We do not seem to be using the copylib anymore, and we've archived the repo.

The playground repo also doesn't exist anymore, it was just used as an experiment at some point.

Issue History

Date Modified Username Field Change
2019-08-27 15:16 Florian Dold New Issue
2019-09-18 15:25 nikita Note Added: 0014913
2019-09-18 15:25 nikita Assigned To => nikita
2019-09-18 15:25 nikita Status new => assigned
2019-09-19 11:03 Florian Dold Note Added: 0014922
2019-09-19 17:42 nikita Note Added: 0014926
2019-09-24 09:47 nikita Note Added: 0014931
2019-09-24 09:47 nikita Assigned To nikita => Florian Dold
2019-09-24 09:47 nikita Status assigned => feedback
2019-09-24 09:58 Florian Dold Note Added: 0014932
2019-09-24 09:58 Florian Dold Status feedback => assigned
2019-09-24 10:39 nikita Note Added: 0014933
2019-09-24 10:41 nikita Note Edited: 0014933
2019-09-24 10:48 Florian Dold Note Added: 0014934
2019-09-24 11:02 nikita Note Added: 0014935
2019-09-24 11:02 nikita Assigned To Florian Dold => nikita
2019-09-24 11:20 nikita Note Added: 0014936
2019-09-24 11:28 Florian Dold Note Added: 0014937
2019-09-24 12:04 nikita Note Added: 0014938
2019-09-24 12:07 Florian Dold Note Added: 0014939
2019-09-24 12:07 nikita Note Added: 0014940
2019-09-24 12:09 nikita Note Added: 0014941
2019-09-24 12:10 nikita Note Edited: 0014941
2019-09-24 12:11 nikita Note Edited: 0014941
2019-09-24 14:07 nikita Note Added: 0014942
2019-09-24 14:07 nikita Assigned To nikita => Florian Dold
2019-09-24 14:10 Florian Dold Note Added: 0014943
2019-09-24 14:10 Florian Dold Assigned To Florian Dold => nikita
2019-09-24 14:16 nikita Note Added: 0014944
2019-09-24 14:38 nikita Note Added: 0014945
2019-09-24 14:40 Florian Dold Note Added: 0014946
2019-09-24 14:55 nikita Note Added: 0014947
2019-09-24 16:06 nikita Note Added: 0014948
2019-09-24 16:07 Florian Dold Note Added: 0014949
2019-09-24 16:13 nikita Note Added: 0014950
2019-09-24 16:13 nikita Assigned To nikita => Marcello Stanisci
2019-09-24 16:16 nikita Note Added: 0014951
2019-09-24 16:16 nikita Note Edited: 0014951
2019-09-24 19:51 nikita Note Added: 0014953
2019-09-24 21:39 Florian Dold Note Added: 0014954
2019-09-24 22:36 nikita Note Added: 0014955
2019-09-25 13:34 nikita Assigned To Marcello Stanisci => nikita
2019-09-25 13:48 nikita Note Added: 0014960
2019-09-25 13:58 nikita Note Added: 0014961
2019-09-25 14:39 nikita Note Added: 0014962
2019-09-25 15:54 nikita Note Edited: 0014962
2019-09-25 16:21 nikita Note Edited: 0014961
2019-09-25 21:34 nikita Note Edited: 0014962
2019-09-25 21:47 nikita Note Edited: 0014962
2019-09-26 10:09 nikita Note Edited: 0014962
2019-09-26 11:35 nikita Note Edited: 0014962
2019-09-27 13:12 nikita Note Edited: 0014962
2019-09-27 18:24 nikita Note Edited: 0014962
2019-09-27 18:33 nikita Note Added: 0014967
2019-09-27 18:33 nikita Assigned To nikita => Marcello Stanisci
2019-09-27 18:33 nikita Status assigned => feedback
2020-08-21 18:04 Florian Dold Status feedback => resolved
2020-08-21 18:04 Florian Dold Resolution open => fixed
2020-08-21 18:04 Florian Dold Note Added: 0016690
2020-09-11 22:24 Christian Grothoff Fixed in Version => 0.8
2020-09-11 22:24 Christian Grothoff Target Version => 0.8
2021-08-24 16:23 Christian Grothoff Status resolved => closed