View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0003803 | GNUnet | NSE service | public | 2015-05-29 13:54 | 2018-06-07 00:24 |
| Reporter | lynX | Assigned To | Christian Grothoff | ||
| Priority | high | Severity | tweak | Reproducibility | always |
| Status | closed | Resolution | no change required | ||
| Platform | x86 | OS | linux | OS Version | several |
| Product Version | 0.10.1 | ||||
| Target Version | 0.11.0pre66 | Fixed in Version | 0.11.0pre66 | ||
| Summary | 0003803: gnunet-service-nse goes CPU hungry on default installation | ||||
| Description | I have installed GNUnet on two machines, one has connectivity to the backbone, the other is isolated. In both cases the gnunet-service-nse process has gone through the roof (~70% CPU usage). The only possibly unusual thing I was to start gnunet like an average user would: mkdir /home/gnunet chown gnunet:gnunet /home/gnunet su - gnunet gnunet-arm -s [ignore some error message about missing .gnunet directory] So all I did was to expect GNUnet to have a sane default configuration if I don't make a custom .gnunet directory. | ||||
| Tags | No tags attached. | ||||
|
|
This is expected. NSE is calculating the proof of work to participate in the network. I guess we could modify the pause between PoW rounds to lower the CPU usage, but that would of course cause the calculation to take longer in Wall-Clock time. Or maybe just warn the user? I guess this could be a "usability bug"? |
|
|
Note we use 22 bits which is scaled to take 2h on a desktop machine, after this time NSE should take exactly 0% cpu. |
|
|
Well, the issue is that with a 'modern' system, the delay is set for NSE to go to like 30% CPU usage on one core. But with older systems the same delay can still mean close to 100% CPU, just like the 2h can turn into weeks if you just have a sufficiently ancient CPU. We do have this in the FAQ already, so I'm not sure there is much to be done. |
|
|
I killed the process earlier. Does it mean my participation in GNUnet is doomed? Other usability issue: The FAQ is not where the user manual is. The FAQ sounds as if the proof-of-work is only to get the network size estimate right. Your average Joe User will not give a damn about that. Is it really only for that purpose or is the proof-of-work used as sybil attack protection for all uses of the DHT, therefore absolutely essential? In that case the FAQ should make it clear. |
|
|
I'm curious as to how you killed the process. GNUnet services are very hard to kill. Are you sure it didn't immediately get resurrected by ARM? ;-) The PoW is essential only for the NSE and for the DHT to perform correctly at scale, your peer will participate normally even if it has not been completed. |
|
|
Absolutely essential? No. Really nice to have? Yes. It would be possible to make the cpu usage exactly what we want it to be, by timing the 'busy" period and then "sleeping" for the same amount of time (for 50%). Also, wouldn't hurt to show a log message (WARNING level maybe?) with: "This peer does not have an NSE proof-of-work. We are generating one and the CPU usage will rise. This is done only once per installation of GNUnet." |
|
|
Nobody will see the warning anyway, but I do like the idea of timing the busy period and making it like 25% CPU use, which most people won't get pissy about. |
|
|
Well, another crazy idea: why not offload the PoW generation to a separate process and name it "gnunet-calm-down-im-suposed-to-use-25-perc-cpu-ill-be-done-in-a-minute"? |
|
|
No need for a separate process, you can just set argv[0] to "calm-done-read-GNUnet-FAQ" or whatever for the time we do the PoW calculation. |
|
|
I used killall. No running arms or legs have tried to restart it. If the PoW isn't crucial, expect not only openwrt installations to come without (which is good) but potentially even distro packagers to sabotage the plan by preparing a GNUnet package that simply doesn't have that. You considered so many attack vectors in the design of GNUnet and didn't think of package maintainers? "calm-done-read-GNUnet-FAQ" <- lol. |
|
|
ACK |
|
|
Not having that hurts GNUnet, specifically the DHT. The performance suffers quite a bit if *most* peers don't run it. I would absolutely not advise any maintainer to leave it out. It's a first-run operation and if we make the load configurable we can let packagers set it a 5% if they are so afraid. Even maybe post a beginner-task on Mantis to integrate nse with dbus and listen to power events to avoid running the PoW on battery. |
|
|
Ok. so let's go with timing the loop to auto-tune for 25% CPU usage (maybe only if the config option is not given, and remove the option by default?), and add the argv[0]-FAQ hint. I'll look into why NSE didn't come back via autostart. Seems like another bug. |
|
|
This doesn't seem to work: int main (int argc, char *argv[]) { argv[0]="FINDME"; while(1){} } Overwriting argv[0] (argv[0][0] = 'F';) does work. However, is it safe? How big is the buffer? strlen(SERVICE_NAME) or is there a fixed value? Implementation dependant? SHould we asume strlen(NAME) as safe max? |
|
|
"KEEP-CALM-and-read-GNUnet-FAQ" :) |
|
|
Yes, AFAIk strlen(argv[0])+1 is the safe max. Fortunately, "gnunet-service-nse" is not that short, so typically the string will be long enough. So strncpy (argv[0], "KeepCalm-GNUnetFAQ", strlen (argv[0])); should fit. |
|
|
Oh, and make sure to restore the name after PoW is done. |
|
|
KeepCalmAndReadFAQ gnunet-service-nse that fits, too ;) (although when using proportional fonts it doesn't look like it does!) in the spirit of https://en.wikipedia.org/wiki/Keep_Calm_and_Carry_On |
|
|
Yes, I noticed that would fit and I'm aware of the meme. However, if we don't explicitly mention GNUnet, some users might not be able to figure out that this is a GNUnet process in the first place, so they won't know which FAQ to read. |
|
|
Oh, stupid me… you're right! |
|
|
One more FAQ… I have tried to fix the 'top' command to show me the full length of process names but to no avail… I can never see which gnunet service is running on top. Is there a fix for top or can we dare to rename all gnunet-service-something processes into srv-something? gnunet is the new internet stack anyway, it doesn't need to say so in 20% of the process names running on a computer… |
|
|
Use a wide terminal and press 'c' (to see full command-line). |
|
|
Yeah, tried that already. Doesn't help since the installation path compiled into the binaries isn't /usr/bin .. it is /usr/depot/depot/linux-x86/gnunet-lux/lib/gnunet/libexec/... so the ending part of the process name again is not being displayed. Would be cool if ./configure was able to distinguish installation paths from runtime execution paths, because the files are actually available at the runtime path which is /usr/depot/lib/gnunet/libexec/... There is no need for them to be run from where they needed to be installed. This is a separate problem from the distinction of architecture dependency we discussed in that other bug report. |
|
|
I said to use a wide terminal ;-). Anyway, most users will see '/usr/lib/gnunet/libexec/gnunet-service-nse', which is a bit more likely to fit. Naturally, users can always put arbitrarily ridiculously long install paths, but that's their issue. |
|
|
configure --help doesn't mention DESTDIR. With that I was able to tweak a solution (a bit ugly, but nevermind). Still the path is inappropriately long: /usr/local/lib/gnunet/libexec/gnunet-service-wtf It is in fact uncommon practice to put /libexec under /lib (I only see KDE doing that, you don't want to imitate KDE habits), and the word gnunet is indeed redundant as the binary will not appear in command path and the execution path includes the word gnunet already. Hence the installation path should IMHO be: /usr/local/libexec/gnunet/srv-wtf With usability greetings from Berlin. |
|
|
Please take that up with FHS. KDE is doing the right thing here, according to my reading of the relevant documents: https://lists.debian.org/debian-devel/2005/05/msg00405.html http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA http://www.linuxbase.org/betaspecs/fhs/fhs/ch04s07.html The one that needs a usability improvement here is 'top': it needs a way to hide the path and just show the filename. |
|
|
Oh, gentoo disrespects FHS. Ok, so on debian I see that "internal binaries" are in places like /usr/lib/git-core/git-pull so we could be using /usr/local/lib/gnunet/bin/srv-wtf to shorten the paths a bit… And sure, a fix for top would be nice… but apparently nobody suffered hard enough in decades of topping. |
|
|
Clearly no change is required here, other than to 'top' which is a bit out-of-scope. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2015-05-29 13:54 | lynX | New Issue | |
| 2015-05-29 13:54 | lynX | Status | new => assigned |
| 2015-05-29 13:54 | lynX | Assigned To | => Bart Polot |
| 2015-05-29 13:57 | Bart Polot | Note Added: 0009155 | |
| 2015-05-29 13:59 | Bart Polot | Severity | major => tweak |
| 2015-05-29 14:00 | Bart Polot | Note Added: 0009156 | |
| 2015-05-29 14:00 | Bart Polot | Note Edited: 0009156 | |
| 2015-05-29 14:05 | Christian Grothoff | Note Added: 0009158 | |
| 2015-05-29 14:17 | lynX | Note Added: 0009162 | |
| 2015-05-29 14:26 | Christian Grothoff | Note Added: 0009164 | |
| 2015-05-29 14:29 | Bart Polot | Note Added: 0009165 | |
| 2015-05-29 14:31 | Christian Grothoff | Note Added: 0009166 | |
| 2015-05-29 14:38 | Bart Polot | Note Added: 0009167 | |
| 2015-05-29 14:48 | Christian Grothoff | Note Added: 0009171 | |
| 2015-05-29 14:56 | lynX | Note Added: 0009173 | |
| 2015-05-29 14:58 | Bart Polot | Note Added: 0009174 | |
| 2015-05-29 14:58 | Bart Polot | Status | assigned => acknowledged |
| 2015-05-29 15:04 | Bart Polot | Note Added: 0009175 | |
| 2015-05-29 15:05 | Christian Grothoff | Note Added: 0009177 | |
| 2015-05-29 15:17 | Bart Polot | Note Added: 0009180 | |
| 2015-05-29 15:19 | Bart Polot | Note Edited: 0009180 | |
| 2015-05-29 15:23 | lynX | Note Added: 0009182 | |
| 2015-05-29 15:30 | Christian Grothoff | Note Added: 0009184 | |
| 2015-05-29 15:30 | Christian Grothoff | Note Added: 0009185 | |
| 2015-05-29 21:40 | lynX | Note Added: 0009192 | |
| 2015-05-29 21:41 | lynX | Note Edited: 0009192 | |
| 2015-05-30 11:59 | Christian Grothoff | Note Added: 0009193 | |
| 2015-05-30 13:03 | lynX | Note Added: 0009194 | |
| 2015-06-01 06:53 | lynX | Note Added: 0009215 | |
| 2015-06-01 07:24 | Christian Grothoff | View Status | private => public |
| 2015-06-01 09:56 | Christian Grothoff | Note Added: 0009216 | |
| 2015-06-01 10:18 | lynX | Note Added: 0009217 | |
| 2015-06-01 10:19 | lynX | Note Edited: 0009217 | |
| 2015-06-01 10:22 | Christian Grothoff | Note Added: 0009218 | |
| 2015-06-01 10:55 | lynX | Note Added: 0009220 | |
| 2015-06-01 11:20 | Christian Grothoff | Note Added: 0009221 | |
| 2015-06-01 12:28 | lynX | Note Added: 0009224 | |
| 2017-02-26 02:12 | Christian Grothoff | Assigned To | Bart Polot => Christian Grothoff |
| 2017-02-26 02:12 | Christian Grothoff | Status | acknowledged => resolved |
| 2017-02-26 02:12 | Christian Grothoff | Resolution | open => no change required |
| 2017-02-26 02:12 | Christian Grothoff | Fixed in Version | => 0.11.0pre66 |
| 2017-02-26 02:12 | Christian Grothoff | Note Added: 0011856 | |
| 2017-02-26 02:12 | Christian Grothoff | Target Version | => 0.11.0pre66 |
| 2018-06-07 00:24 | Christian Grothoff | Status | resolved => closed |
