View Issue Details

IDProjectCategoryView StatusLast Update
0005353GNUnetotherpublic2021-03-03 18:48
Reporternikita Assigned Toschanzen  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionwon't fix 
OSGuix and Nix 
Product VersionGit master 
Summary0005353: libcurl/libgnurl certificate location needs to be handled in GNUnet code
DescriptionWe need to add the ability to use well-known environment variables to define and/or override system provide certificate stores.

Ee need this to make it to work on Operating Systems and package managers which expose certain flaws.

In my case, I need this fixed in GNUnet for proper Guix and GuixSD integration. gnURL does not seem like the right place to do this. I've tried this a couple of times in the past in Guix and the result was always bad. There's a certain variable we need to be able to accept, at least that is one way to do it.
You could fetch the guix source code (https://git.savannah.gnu.org/cgit/guix.git) and grep for 'curl' and 'CURL_'. There's a couple of places (if I remember correctly) where not only the search-path is adjusted but also the source code of the application gets patched.
Additional InformationIn my case it looks like this, build against most recent gnurl etc (ignore all the other errors, it's not properly integrated yet):

abyayala$ sudo -E gnunet-arm -s
Password:
abyayala$ Jun 15 09:35:28-961830 util-os-installation-6653 WARNING `access' failed on file `/gnu/store/7axjz524qbi4fx95zackdm36b13a0wi3-gnunet-git1/lib//gnunet/libexec/gnunet-auto-share' at os_installation.c:888 with error: No such file or directory
Jun 15 09:35:28-961889 arm-6652 ERROR Failed to start service `gnunet-auto-share'
Fatal: executable iptables not found in approved directories: No such file or directory
Fatal: executable iptables not found in approved directories: No such file or directory
Jun 15 09:35:29-049837 transport-tcp-6670 WARNING `bind' failed for port 2086 (IPv6): address already in use
Jun 15 09:35:29-049975 transport-tcp-6670 WARNING `bind' failed for port 2086 (IPv4): address already in use
Jun 15 09:35:29-050004 util-connection-6670 ERROR `bind' failed at tcp_server_legacy.c:605 with error: Address already in use
Jun 15 09:35:29-444090 hostlist-6658 WARNING Download of hostlist from `http://v10.gnunet.org/hostlist' failed: `Peer certificate cannot be authenticated with given CA certificates'
Fatal: executable iptables not found in approved directories: No such file or directory
TagsNo tags attached.

Activities

nikita

2018-09-05 08:35

developer   ~0013224

lurchi: As I sometimes forget how things were solved in Nix.. is this reproducible on NixOS? Or do they handle curl differently?

nikita

2018-10-21 12:12

developer   ~0013276

I forgot to add this a while back:

https://curl.haxx.se/libcurl/c/CURLOPT_CAPATH.html

https://curl.haxx.se/mail/lib-2016-08/0118.html


This affects both curl and gnurl.

From experience I know that our curl code should pick up the environment variable mentioned there. In Guix they use:

env | grep CURL
CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

nikita

2018-10-26 13:12

developer   ~0013290

dvn and lurchi asked me to expand on the information I have on this apparently Guix and Guix-on-GuixSD specific issue.

> why do we experience this CA error on guix and nix but not on eg. Debian?

Aside: Debian and Guix(SD) follow different hierarchies. No one ever bothered to document the hierarchies, but it's not that hard.
The build chroots (of Debian, Nix, and Guix) are also different.
Notable Guix has just the bare minimum, less than Nix even. So Nix != Guix, when you
read the code relating to how the environments are constructed.

Anyway, the situation. I will quote and reference where possible, because a) I've been annoyed enough as a packager by this problem back on Guix and discussions around it, and b) because you will get more than one perspective.


In early 2015 Guix started migrating away from having a built-in ca-path location:
https://lists.gnu.org/archive/html/guix-devel/2015-03/msg00615.html

After the first code for this was implemented, the differences between curl linked against OpenSSL and curl linked against GnuTLS are explained here: https://lists.gnu.org/archive/html/guix-devel/2015-03/msg00127.html

--quote--
> However there’s the complication that all the files of ‘nss-certs’ would
> still be there in addition to the bundle. Hmm.

That's a feature, not a bug. It is more efficient to look up the
individual files by their hash-named symlinks than to read the entire
certificate bundle as one file. The only problem is that some
combinations of software don't yet support this mode.

For example, libcurl (used by git) only supports the single-file when it
is linked with GnuTLS. When linked with OpenSSL it supports both modes.
(This is a limitation of libcurl's backend for GnuTLS, not an inherent
problem with GnuTLS.)
--end quote--

https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00429.html

It has been discussed that p11kit should be used instead of the *deprecated* single file or directory location for systemwide certificates: https://lists.gnupg.org/pipermail/gnutls-devel/2015-February/007442.html
http://gnutls.org/manual/html_node/Verification-using-PKCS11.html


In essence it can be broken down to the reply Ludovic gave here (in this bug report from december 2016, still open):
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25240

How do we deal with this as gnunet?
Do we have to deal with this?
If you want to discuss with guix, go ahead. I'm too annoyed by the problem and outcome of past discussions.

Christian Grothoff

2018-11-22 10:36

manager   ~0013360

Well, ultimately we indeed have to deal with it, but if Guix doesn't ship with a cert store that curl+gnutls expects/supports, that's just bad. For now, I guess we should hope that GnuTLS will start to support the new format, and then things should start to work again, right?

nikita

2019-01-27 15:08

developer   ~0013478

Last edited: 2019-01-27 15:12

Here's the take we had in pkgsrc on this:

# Fallback to gnutls preferred CA certificates
CONFIGURE_ARGS+= --without-ca-bundle
CONFIGURE_ARGS+= --without-ca-path
CONFIGURE_ARGS+= --with-ca-fallback

and this makes gnurl pick up gnutls certificates, which are build in with the gnutls package. However, this won't work for guix and nix (I'm 99% sure by experience and exhausting discussions).
This works for pkgsrc building GnuTLS version 3.6.5


In my opinion someone in Nix or Guix should work on this if we don't fix it in gnunet, I'm done with the issue for now. The status quo works for everyone else. I'm considering to look at p11kit again, but that's it.

I hope you can understand that after several packages, many bugs, and countless discussions (and non-functional packages because of this) sometimes over months that I'm done with supporting the idea of how Guix wants to solve their curl problem.

dvn

2019-01-28 04:13

developer   ~0013484

With ng0's last comment in consideration, I think we should move this out of the 0.11.0 roadmap.

dvn

2019-01-28 04:16

developer   ~0013485

This issue is currently difficult to fix from our perspective - It affects some Nix derived Linux distributions, and we seek feedback on solutions, and/or collaboration downstream maintainers.

schanzen

2019-08-08 19:44

administrator   ~0014777

Not a blocker. Not urgent. Probably a downstream issue.

schanzen

2021-03-03 18:48

administrator   ~0017584

Closing this as wontfix. As mentioned this is obviously a downstream issue in those distros.

Issue History

Date Modified Username Field Change
2018-06-15 19:55 nikita New Issue
2018-06-15 19:55 nikita Issue generated from: 0004632
2018-08-23 20:34 dvn Priority normal => urgent
2018-08-23 20:34 dvn Severity feature => block
2018-08-23 20:34 dvn OS => Guix and Nix
2018-09-05 08:35 nikita Note Added: 0013224
2018-10-21 12:12 nikita Note Added: 0013276
2018-10-26 13:12 nikita Note Added: 0013290
2018-11-22 10:36 Christian Grothoff Note Added: 0013360
2019-01-27 15:08 nikita Note Added: 0013478
2019-01-27 15:12 nikita Note Edited: 0013478
2019-01-28 04:13 dvn Note Added: 0013484
2019-01-28 04:16 dvn Assigned To => dvn
2019-01-28 04:16 dvn Status new => feedback
2019-01-28 04:16 dvn Note Added: 0013485
2019-01-28 04:29 dvn Assigned To dvn =>
2019-01-28 04:29 dvn Target Version 0.11.0 =>
2019-08-08 19:44 schanzen Priority urgent => normal
2019-08-08 19:44 schanzen Severity block => major
2019-08-08 19:44 schanzen Note Added: 0014777
2021-03-03 18:48 schanzen Assigned To => schanzen
2021-03-03 18:48 schanzen Status feedback => closed
2021-03-03 18:48 schanzen Resolution open => won't fix
2021-03-03 18:48 schanzen Note Added: 0017584