View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0004386 | Taler | exchange | public | 2016-04-07 23:20 | 2016-04-10 17:12 |
| Reporter | Christian Grothoff | Assigned To | Christian Grothoff | ||
| Priority | low | Severity | tweak | Reproducibility | N/A |
| Status | closed | Resolution | fixed | ||
| Platform | i7 | OS | Debian GNU/Linux | OS Version | squeeze |
| Product Version | 0.0 | ||||
| Target Version | 0.0 | Fixed in Version | 0.0 | ||
| Summary | 0004386: configuration file and data-file structure is awkward with the "datadir" setup we have | ||||
| Description | Right now, we have the "base directory" passed with the "-d" option to various Taler-exchange tools, which is very awkward in various ways: * does not make use of the config.d/-default configuration file mechanism * usually results in two configuration files (denomination keys + normal config) * unusual to need a separate directory, where usually -c CONFIGFILE suffices * violates XDG guidelines * mixes different types of data (private keys, auditor lists, configuration files, JSON-wire files) in a hard-coded sub-structure that users must understand (!) What might we want to have instead? My current ideas: * defaults in ~/share/taler/config.d/ (already there, just change code to use it!) * custom configuration under ~/.config/taler/taler.conf (or override via -c) * auditor list under ~/.config/taler/auditors/ (per default, ultimate path in config) * JSON wire config under ~/.config/taler/wire-PLUGINNAME.json (per default, ultimate paths in config) * denomination and signing keys under ~/.local/taler/share/KEYTYPE/? (re-read XDG; again, this is just the default, ultimate path in config) taler-exchange-wire could then place the resulting JSON file by default in the right location. taler-exchange-keyup (and similar key-tools) could again by default create the key structure in the right location. In fact, in the end the entire structure may be manipulated pretty much _only_ via tools (to avoid fat-fingering errors!). Ideas/suggestions/comments welcome. | ||||
| Tags | No tags attached. | ||||
|
|
I completely support your suggestion of having a default place for the key structure, and also agree with your remark that it would be great if users never have to touch it manually. A few questions / remarks: 1. You wrote "~/share/taler/config.d/". Do you really mean $HOME, or was that meant to be "/share/taler/config.d/"? 2. My original motivation for having separate directories was that the merchant configuration and exchange configuration should be separate. Why should they access the same defaults directory? Why would they share the same config file? 3. We do not have good tooling for the configuration overloading mechanism: a) It's not possible to find out easily where a value came from b) It's not easy to modify values with gnunet-config (it just dumps the global config in the resulting config file ...) c) We load the config.d first and THEN the config given via -c, since we then have to set the location of config.d via an environment variable ... so there's no way to control the full set of configuration entries when we want to version them separately (e.g. in the deployment git) This is why I avoided the overloading. See git [1] for an example of much saner tooling with overloading support. It also shows where values came from (file/line). So, in summary, I agree with all the changes you proposed, except defaults overloading and putting all components in the same configuration. [1] https://git-scm.com/docs/git-config |
|
|
1) ~ was supposed to indicate whatever the installation prefix for Taler is. 2) First of all, there is no problem as the options will be disjoint. Having share/taler/ is natural, having share/taler-{merchant,exchange}/ is slightly more confusing, but worse is terrible for the code as the merchant needs to load plugins from the exchange (wire!), and libtalerutil can setup one set of paths for libgnunetutil, but not multiple. Because this gets unnecessarily messy and introduces great potential for problems -- without a good reason -- I decided to force all taler-*-subprojects into the same directory structure. 3) a) gnunet-config does that, for Taler you'll have to LD_PRELOAD libtalerutil to apply gnunet-config to Taler b) maybe you want gnunet-config-diff? (not installed, but exists in GNUnet) Alternatively, just make gnunet-config more powerful. c) Yes, by dumping full config with gnunet-config you can override "everything" from defaults. But really, if deployment specifies the base installation (i.e. which Taler revision), then that implies a particular set of values for config.d/. So I'm not sure your point is valid here. As for showing where values come from => Just hack GNUnet! Same for 'git-config' is more sane (didn't study it yet, but if there is really some dramatic improvement we can easily get, we might want to have it for GNUnet as well). |
|
|
I also really don't see a reason for having a config defaults *directory* when we have an include mechanism. Wouldn't it be more orthogonal if both the -c and default config would be just a file? |
|
|
The reason to have a config.d/ is the same we have this for many /etc/*/-services: appyling a diff to a config file is messy, it is way easier (if it works) for the package manager to dump a file into a particular directory. So as long as our defaults are truly sane and not over-specifying, an installer could, for example, dump the DB access data or other things into a package-specific file (i.e. config.d/debian.conf), and then it is clear what comes from upstream and what Debian added -- and what the sysadmin added himself manually. |
|
|
I agree that a lot of these issues go away when we make the tooling more powerful. But: No, you can't really override the config.d/ settings, since there's no sane way to remove a section / option, other than setting all the values to the empty string. This was the problem I had with gnunet-arm for Taler test/demo deployments, and why I introduced the GNUNET_BASE_CONFIG variable. |
|
|
I guess having a section [config] # This should be the default value include = /usr/share/taler/conf.d/* in the configuration would make everything a bit saner, since we JUST have to deal with the -c and not also the environment variable (would we otherwise also need a TALER_BASE_CONF?). It also addresses your concerns about package managers -- they can still just drop a file. Any objections to this? IMHO it makes things simpler, while retaining everything we can do now ... |
|
|
The include mechanism can also be used by users to split up complex configurations. Why should just the system config be allowed to do that? (The include idea is stolen from git, but I guess they don't allow globs ...) EDIT: erm ... I forgot that GNUnet already supports some form of includes I guess ... |
|
|
The issue with having such a section is that it implies that you MUST have _some_ configuration file. Manually creating a config file with just that one line can be a major problem for users. In fact, once we had config.d/ for GNUnet it was still an issue that people had to create an empty config file, so we fixed that by not requiring one to exist _at all_. With your change, you're basically saying "but one must exist". Now, in the past I would have said that I can live with that, _if_ you modify the code so that it automatically writes such a config file if none exists. But that's also no longer OK: with stuff like Apparmor, you might want to say that various processes must NEVER write to disk, or at least never write a file in ~/.config/. So that leaves you with the Git approach of having all GNUnet tools complain if the config does not exist, and interactively suggest to the user to run 'gnunet-config --create-default'. But this _still_ does not work, as you do want in the end users to be able to use GNUnet without _ever_ going to command line, so you only have gnunet-gtk/gnunet-setup. So now you'd also need to modify gnunet-gtk's tools to ask the user to run gnunet-setup first. That might be fine, except that furthermore gnunet-arm ought to have been configured to run automatically by the package manager. So now you're asking the package manager to create your 3-line config file for each existing user and in /etc/skel/, so that services like gnunet-arm that run in the background always have a configuration. Note that unlike my concern with package managers, this is no longer something they could do optionally, but which becomes mandatory. As for your environment variable: yes, we probably should add that to the OS_ProjectData variables; we already have the env_varname for GNUNET_PREFIX in there. |
|
|
With what I suggested there doesn't have to be a pre-existing config file for the user. As mentioned above, in GNUnet/Taler, we could make that include glob from above is the default. That is, when no configuration file is present, it pretends the "[config]include = $PREFIX/share/taler/conf.d/*" is there. Only if we want a *different* glob (or no base config files at all, or just single file(s)) we'd specify that config value. Again, this way we have two environment variables less to deal with, and we don't have the strange distinction between files for -c and directories for *_BASE_CONFIG. |
|
|
After some offline discussion I'm in favour of keeping the *_BASE_CONFIG, since it keeps things simple / doesn't require lots of changes and still works for everything we need to do. |
|
|
Resolved in 5fb65f5..30d66bc |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2016-04-07 23:20 | Christian Grothoff | New Issue | |
| 2016-04-07 23:20 | Christian Grothoff | Status | new => assigned |
| 2016-04-07 23:20 | Christian Grothoff | Assigned To | => Christian Grothoff |
| 2016-04-08 00:11 | Florian Dold | Note Added: 0010404 | |
| 2016-04-08 00:12 | Florian Dold | Note Edited: 0010404 | |
| 2016-04-08 00:24 | Christian Grothoff | Note Added: 0010405 | |
| 2016-04-08 00:27 | Florian Dold | Note Added: 0010406 | |
| 2016-04-08 00:31 | Christian Grothoff | Note Added: 0010407 | |
| 2016-04-08 00:31 | Florian Dold | Note Added: 0010408 | |
| 2016-04-08 00:38 | Florian Dold | Note Added: 0010409 | |
| 2016-04-08 00:39 | Florian Dold | Note Edited: 0010409 | |
| 2016-04-08 00:40 | Florian Dold | Note Added: 0010410 | |
| 2016-04-08 00:44 | Florian Dold | Note Edited: 0010410 | |
| 2016-04-08 00:49 | Christian Grothoff | Note Added: 0010411 | |
| 2016-04-08 01:15 | Florian Dold | Note Added: 0010412 | |
| 2016-04-08 01:16 | Florian Dold | Note Edited: 0010412 | |
| 2016-04-08 01:17 | Florian Dold | Note Edited: 0010412 | |
| 2016-04-08 01:17 | Florian Dold | Note Edited: 0010412 | |
| 2016-04-08 13:09 | Florian Dold | Note Added: 0010416 | |
| 2016-04-10 17:11 | Christian Grothoff | Note Added: 0010441 | |
| 2016-04-10 17:11 | Christian Grothoff | Status | assigned => resolved |
| 2016-04-10 17:11 | Christian Grothoff | Fixed in Version | => 0.0 |
| 2016-04-10 17:11 | Christian Grothoff | Resolution | open => fixed |
| 2016-04-10 17:12 | Christian Grothoff | Status | resolved => closed |
| 2016-04-10 17:12 | Christian Grothoff | Target Version | 0.1 => 0.0 |
