View Issue Details

IDProjectCategoryView StatusLast Update
0005723Talerdeployment and operationspublic2019-09-16 09:22
ReporterLUGAssigned ToLUG 
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionreopened 
Platformvirtual machineOSxUbuntuOS Version18.04
Product VersionSVN HEAD 
Target Version0.6Fixed in Version0.6 
Summary0005723: Undocumented environment dependencies for bootstrap-standalone deployment
DescriptionSince setting up the repos individually didn't work out for me, I wanted to follow the developers' advice to run a system wide setup from the official deployment repo, following the onboarding guide:
https://docs.taler.net/onboarding/pdf/onboarding.pdf

When running "./deployment/bootstrap-standalone" I get the following error:
  fatal: repository '/var/git/twister.git' does not exist

Looking into the bootstrap-standalone script it looks like numerous repos from git.taler.net are required to be present in /var/git/
I'd like to suggest adding this information as a requirement to the onboarding documentation.

Also, the package libpq-dev seemed to be needed as well to support the pg_config call in bootstrap-standalone:14
Steps To Reproduce- Install clean xUbuntu 18.04
- clone deployment repo into ~/deployment
- ./deployment/bootstrap-standalone
TagsNo tags attached.

Activities

LUG

2019-05-13 21:30

developer   ~0014392

Last edited: 2019-05-13 21:33

View 2 revisions

Alternatively we could adjust the standalone deployment script to clone the relevant repos from git.taler.net instead of /var/git
Let me know what you prefer and I'm happy to help however I can.

LUG

2019-05-13 21:37

developer   ~0014393

And "libpq-dev" doesn't seem to be enough. Installing the "postgresql" server package as a whole seems necessary.

LUG

2019-05-13 21:42

developer   ~0014394

When I follow the setup to:
# Then we need to install GNUnet beforehand, as it provides the ’ARM’# utility that is used to start the database service.
$ cd deployment/taler-arm/
$ make gnunet-stamp

I get "make: *** No rule to make target 'gnunet-stamp'. Stop."

The content of deployment/taler-arm/ is:
-rwxrwx--- 1 user user 656 Mai 13 21:15 arm.conf
-rwxrwx--- 1 user user 630 Mai 13 21:15 defaults.conf
-rwxrwx--- 1 user user 116 Mai 13 21:15 taler-aggregator.conf
-rwxrwx--- 1 user user 104 Mai 13 21:15 taler-auditor.conf
-rwxrwx--- 1 user user 150 Mai 13 21:15 taler-backoffice.conf
-rwxrwx--- 1 user user 125 Mai 13 21:15 taler-blog.conf
-rwxrwx--- 1 user user 127 Mai 13 21:15 taler-demobank.conf
-rwxrwx--- 1 user user 140 Mai 13 21:15 taler-donations.conf
-rwxrwx--- 1 user user 107 Mai 13 21:15 taler-exchange.conf
-rwxrwx--- 1 user user 130 Mai 13 21:15 taler-exchange-wirewatch.conf
-rwxrwx--- 1 user user 107 Mai 13 21:15 taler-merchant.conf
-rwxrwx--- 1 user user 134 Mai 13 21:15 taler-playground.conf
-rwxrwx--- 1 user user 192 Mai 13 21:15 taler-postgres-standalone.conf
-rwxrwx--- 1 user user 131 Mai 13 21:15 taler-survey.conf

Christian Grothoff

2019-05-19 02:00

manager   ~0014405

I fixed this:
  When running "./deployment/bootstrap-standalone" I get the following error:
  fatal: repository '/var/git/twister.git' does not exist

The old script would have only worked on git.taler.net as it assumed that the master Git repositories were one the same machine. I've now changed the script to clone from remote.

Christian Grothoff

2019-05-19 02:03

manager   ~0014406

libpq-dev is enough to build the code, of course a Postgres server is required "somewhere", but at least in theory it doesn't have to run on the same physical machine.

Christian Grothoff

2019-05-19 02:03

manager   ~0014407

On the "make gnunet-stamp" bug, I am confused: the Makefile in that directory contains that target for me. Did you clone the deployment.git from git.taler.net?

LUG

2019-05-19 02:11

developer   ~0014408

Thanks for the git repo fix.

True, the database server doesn' t need to run on the same machine as well. In the context of the bootstrap-standalone deployment I understood that it requires a Postgrees server on the same machine though, right? If I didn' t misunderstand, it might be nice to add a matching hint to the onboarding documentation.

Thanks for trying this locally yourself, I'll try to replicate the issue again and provide more detailed information.

Marcello Stanisci

2019-05-19 02:44

manager   ~0014409

LUG, that gnunet-stamp is to be mentioned inside the *taler-build* directory, and NOT inside taler-arm. Like this:

$ cd deployment/taler-build
$ make gnunet-stamp

LUG

2019-05-19 12:55

developer   ~0014410

Last edited: 2019-05-19 14:56

View 10 revisions

Thanks Marcello, there are make recipes in that directory. Would you then be so kind to correct this directory in the onboarding document?

And while that new directory enables me to run make, I needed to satisfy these additional dependencies:
autoconf autopoint libtool gnutls-dev texinfo libgcrypt20-dev libunistring-dev libjansson-dev

And run
$ cd ~/libmicrohttpd
$ autoreconf -fi

Then I can successfully run:
$ make gnunet-stamp
$ taler-deployment-arm -s
$ taler-deployment-arm -i taler-postgres-standalone

For running "taler-deployment-config-generate" I first needed to:
$ export LC_ALL=C.UTF-8
$ export LANG=C.UTF-8

LUG

2019-05-19 14:56

developer   ~0014411

Last edited: 2019-05-19 16:53

View 8 revisions

To get "taler-deployment-build" to work until the error below, the following steps were necessary:

Install missing python dependencies
$ apt install python3-pip python3-click python3-babel python3-jinja2 python3-jsmin python3-requests

Install nodejs package manager yarn (following https://yarnpkg.com/en/docs/install#debian-stable)
$ apt install apt-transport-https
$ curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
$ echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
$ apt update
$ apt install yarn

Install recent version of nodejs (following https://downloads.nodesource.com/)
$ curl -sL http://nsolid-deb.nodesource.com/nsolid_setup_3.x | bash -
$ apt -y install nsolid-dubnium nsolid-console

$ taler-deployment-build

After a lot of successful stuff, it comes to "deployment/taler-build/update_exchange.sh" where the following line causes an error
$ ./configure CFLAGS='-ggdb -O0' --with-libgnurl=$HOME/local --with-microhttpd=$HOME/local --with-twister=$HOME/local --prefix=$HOME/local --with-gnunet=$HOME/local --enable-logging=verbose
./configure: line 17352: ac_fn_c_check_decl: command not found
configure: error:
***
*** You need libmicrohttpd >= 0.9.39 to build this program.
***
Makefile:40: recipe for target 'exchange-stamp' failed
make: *** [exchange-stamp] Error 1

And I have "$HOME/local/lib/libmicrohttpd.so.12.50.0"

Marcello Stanisci

2019-05-19 23:26

manager   ~0014412

re 14410:

LUG, that directory looks already good on the manual, see here:
https://git.taler.net/deployment.git/tree/doc/onboarding.texi#n127

Then, as for your other observations: you are basically saying that the onboarding manual
doesn't tutor the user along the installation of the various components (exchange / bank / ..)
and dependencies (GNUnet / libmicrohttd / ..).

The point is that we already have such instructions for all the components, under each
component's repository; you might want to have a look under "doc/" directories, or READMEs,
or just find ".texi" files and compile them.

And to tell the whole story, the onboarding manual was meant to onboard sysadmins at
our servers, and not as a tutorial to build Taler on arbitrary machines. In fact, it forces the
machine to have a specific layout of directories, disallowing the reader to freely choose their
"--prefix"(es). That said, feel free to run it where you want!

Marcello Stanisci

2019-05-20 14:07

manager   ~0014413

Here: 4de683a..e468ab0, I put a note that invites the reader to go through each component's building instructions.

If you think this is solved, please either remove the 'block' status or just close; thanks!

LUG

2019-05-20 20:44

developer   ~0014418

Last edited: 2019-05-20 20:54

View 2 revisions

Marcello, thanks for your input and the explanations. If that is the intention of the onboarding document I missed its use case, indeed.
What would you recommend for repos like "donations" or "survey" where there seems to be neither README nor doc/ directory?

The directory on the manual source is correct but the compiled version that is linked from the website is still outdated: https://docs.taler.net/onboarding/pdf/onboarding.pdf
Is there maybe a way to build that linked document automatically to ensure that it's always up to date?

Thanks for your support with enabling me.

Marcello Stanisci

2019-05-21 00:38

manager   ~0014420

Thanks for reporting!

donations (27e5a47), blog (2fd1669), and survey (d382fb1) got all updated to contain a README file.
However, please note that those three repos are not officially part of Taler, but are just examples.

Then, the docs building automation was already put in place, except that for deployment.git was never
started! Now it SHOULD be fixed (here, at deployment.git: 82b748694d), but I can't test it right now because
Gitolite doesn't update its repository upon pushes. I filed a new bug for this: 0005729.

LUG

2019-05-21 13:51

developer   ~0014424

Much appreciated! I'll close this issue and work my way through the repos to hopefully soon provide a consistent update in terms of 0005704 to all demo relevant repos.

Marcello Stanisci

2019-05-22 10:47

manager   ~0014429

As for the docs automated building: please consider going to docs.wild.gv.taler.net now.

LUG

2019-05-22 17:44

developer   ~0014433

It seems that I can't. When opening https://docs.wild.gv.taler.net/ I get a certificate error that I can't ignore due to HTTP Strict Transport Security (HSTS).

Marcello Stanisci

2019-05-30 14:29

manager   ~0014462

Hello LUG.

The whole site got migrated to: https://docs.taler.net/

Please try there (and reopen if need be)!

Issue History

Date Modified Username Field Change
2019-05-13 21:28 LUG New Issue
2019-05-13 21:30 LUG Note Added: 0014392
2019-05-13 21:33 LUG Note Edited: 0014392 View Revisions
2019-05-13 21:37 LUG Note Added: 0014393
2019-05-13 21:42 LUG Note Added: 0014394
2019-05-19 02:00 Christian Grothoff Note Added: 0014405
2019-05-19 02:03 Christian Grothoff Note Added: 0014406
2019-05-19 02:03 Christian Grothoff Note Added: 0014407
2019-05-19 02:11 LUG Note Added: 0014408
2019-05-19 02:44 Marcello Stanisci Note Added: 0014409
2019-05-19 12:55 LUG Note Added: 0014410
2019-05-19 12:57 LUG Note Edited: 0014410 View Revisions
2019-05-19 13:21 LUG Note Edited: 0014410 View Revisions
2019-05-19 13:41 LUG Note Edited: 0014410 View Revisions
2019-05-19 13:46 LUG Note Edited: 0014410 View Revisions
2019-05-19 13:58 LUG Note Edited: 0014410 View Revisions
2019-05-19 14:02 LUG Note Edited: 0014410 View Revisions
2019-05-19 14:18 LUG Note Edited: 0014410 View Revisions
2019-05-19 14:19 LUG Note Edited: 0014410 View Revisions
2019-05-19 14:56 LUG Note Edited: 0014410 View Revisions
2019-05-19 14:56 LUG Note Added: 0014411
2019-05-19 14:56 LUG Note Edited: 0014411 View Revisions
2019-05-19 15:28 LUG Note Edited: 0014411 View Revisions
2019-05-19 16:11 LUG Note Edited: 0014411 View Revisions
2019-05-19 16:46 LUG Note Edited: 0014411 View Revisions
2019-05-19 16:46 LUG Note Edited: 0014411 View Revisions
2019-05-19 16:51 LUG Note Edited: 0014411 View Revisions
2019-05-19 16:53 LUG Note Edited: 0014411 View Revisions
2019-05-19 23:26 Marcello Stanisci Note Added: 0014412
2019-05-20 14:07 Marcello Stanisci Note Added: 0014413
2019-05-20 20:44 LUG Note Added: 0014418
2019-05-20 20:51 LUG Severity block => minor
2019-05-20 20:54 LUG Note Edited: 0014418 View Revisions
2019-05-21 00:38 Marcello Stanisci Note Added: 0014420
2019-05-21 13:51 LUG Note Added: 0014424
2019-05-21 13:52 LUG Assigned To => LUG
2019-05-21 13:52 LUG Status new => resolved
2019-05-21 13:52 LUG Resolution open => fixed
2019-05-22 10:47 Marcello Stanisci Note Added: 0014429
2019-05-22 17:44 LUG Status resolved => feedback
2019-05-22 17:44 LUG Resolution fixed => reopened
2019-05-22 17:44 LUG Note Added: 0014433
2019-05-30 14:29 Marcello Stanisci Note Added: 0014462
2019-05-30 14:30 Marcello Stanisci Status feedback => resolved
2019-09-16 09:22 Christian Grothoff Product Version => SVN HEAD
2019-09-16 09:22 Christian Grothoff Fixed in Version => 0.6
2019-09-16 09:22 Christian Grothoff Target Version => 0.6