View Issue Details

IDProjectCategoryView StatusLast Update
0007355GNUnetfile-sharing servicepublic2022-09-29 07:11
Reporterthejackimonster Assigned ToChristian Grothoff  
PrioritynormalSeverityminorReproducibilityalways
Status assignedResolutionreopened 
Product Version0.17.5 
Target Version0.18.0 
Summary0007355: File service does not work in a multi-user setup
DescriptionI've encountered multiple times the situation that file sharing in the messenger service didn't work as expected. From my testing the issue seems to be connected to the FS service because its callbacks for uploading a file won't get called. This might be related to the gnunet configuration on the system I experienced this issues but I assume those issues shouldn't even appear because the FS service should allow to upload to your own node as well in case there are missing connections to other nodes.

Edit:
So if a file which should be published is part of a directory or sub-directory the service has no execution permission on, the publication fails on hash mismatch. Can be reproduced with `gnunet-publish` instead of any messenger application.
Steps To ReproduceStart messenger-gtk or messenger-cli and try sending a file. In messenger-gtk the file appears to be added to the list of uploading files but the upload never finishes. In the messenger-cli application you can send a local file by typing "file:" and the following absolute path of the file in the text field and pressing ENTER. However it ends up not doing anything when the file message should appear in the chat afterwards.

Edit:
$ mkdir test
$ echo "">test/file
$ chmod o-x test
$ gnunet-publish test/file
Error publishing: Can not index file `./test/file': hash mismatch.
.
Additional InformationOne fix I already applied is to correct the error handling of the failure in index_start_failed from the FS service:
https://git.gnunet.org/gnunet.git/commit/?id=d9a0342d77f4489e56678d9a8e6afc9e25d8163d
TagsNo tags attached.

Activities

schanzen

2022-09-10 15:30

administrator   ~0019115

I suggest producing a reproducible test outside of messenger for us unless you plan on fixing this yourself.

thejackimonster

2022-09-11 00:40

developer   ~0019116

I will probably start by adding a simple test in libgnunetchat. Then figure out what's causing it from there.

thejackimonster

2022-09-19 14:19

developer   ~0019147

I think, I found the issue:
1.) I found with valgrind that `check_index_start_failed()` in "fs_publish.c" would have 'Invalid read of size 1'. I looked it up and it seemed to substract the size of the message header twice when checking for an ending zero in the error string.
2.) Even more problematic is that the handler for `GNUNET_MESSAGE_TYPE_FS_INDEX_START_FAILED` expects a `GNUNET_MQ_hd_var_size()` but in "gnunet-service-fs.c" it sends a simple `GNUNET_MQ_msg()` without any error string.

I still need to figure out what's the issue indexing my file but the error handling seems to break expected behavior. Since with some changes to fixt those two issues, my upload callback works at least. ^^'

thejackimonster

2022-09-19 14:38

developer   ~0019148

I commited a fix for the wrong handling but my file still won't indexed. So that still needs to be fixed. However valgrind seems to be happy again.

thejackimonster

2022-09-19 15:22

developer   ~0019149

gnunet-publish /home/thejackimonster/.local/share/gnunet//messenger//chat/000G0066ZANZ0C5K1N1B3YNC76DPBPCRQ9YJRN0GMETYSQH4JDR7K5B8G4/files/SY1Y2DBYXYWBVWAM518DCVC00ZB21S051DBHBQ43YJMJ3MVCX774FM6H7HERBWNGZY1HHMM7FVP2YRXS66YMEGBTG6JKGCKTZ4KXMFG
Error publishing: Can not index file `/home/thejackimonster/.local/share/gnunet/messenger/chat/000G0066ZANZ0C5K1N1B3YNC76DPBPCRQ9YJRN0GMETYSQH4JDR7K5B8G4/files/SY1Y2DBYXYWBVWAM518DCVC00ZB21S051DBHBQ43YJMJ3MVCX774FM6H7HERBWNGZY1HHMM7FVP2YRXS66YMEGBTG6JKGCKTZ4KXMFG': hash mismatch.

cp /home/thejackimonster/.local/share/gnunet//messenger//chat/000G0066ZANZ0C5K1N1B3YNC76DPBPCRQ9YJRN0GMETYSQH4JDR7K5B8G4/files/SY1Y2DBYXYWBVWAM518DCVC00ZB21S051DBHBQ43YJMJ3MVCX774FM6H7HERBWNGZY1HHMM7FVP2YRXS66YMEGBTG6JKGCKTZ4KXMFG test.txt

gnunet-publish test.txt
Publishing `/mnt/projects/thejackimonster/git/messenger-cli/test.txt' done.
URI is `gnunet://fs/chk/SY1Y2DBYXYWBVWAM518DCVC00ZB21S051DBHBQ43YJMJ3MVCX774FM6H7HERBWNGZY1HHMM7FVP2YRXS66YMEGBTG6JKGCKTZ4KXMFG.SY1Y2DBYXYWBVWAM518DCVC00ZB21S051DBHBQ43YJMJ3MVCX774FM6H7HERBWNGZY1HHMM7FVP2YRXS66YMEGBTG6JKGCKTZ4KXMFG.0'.

o.O The name seems to cause a hash mismatch of the file? How?

thejackimonster

2022-09-19 15:33

developer   ~0019150

It seems like the issue is related to my home partition. I will check whether it's an disk error or something in the fs service. Because my test file has a size of zero... ^^'

But it works fine if it comes from a different partition as far as I can tell.

thejackimonster

2022-09-19 15:43

developer   ~0019151

Oh... I think I know what's wrong. The original hash calculated on client-level differs from the hash calculated on service-level because the service user does not have read access to my home partition. ^^'

That makes sense. I just really don't think the FS service should require global file access to work. Maybe it would be possible to change file read permissions on client-side though.

thejackimonster

2022-09-19 15:54

developer   ~0019152

So if a file which should be published is part of a directory or sub-directory the service has no execution permission on, the publication fails on hash mismatch. Can be reproduced with `gnunet-publish` instead of any messenger application.

mkdir test
echo "">test/file
chmod o-x test
gnunet-publish test/file
Error publishing: Can not index file `./test/file': hash mismatch.
.

schanzen

2022-09-20 19:01

administrator   ~0019157

Hmm. This sounds to me like the fs service should run per user:

RUN_PER_USER = YES

?

thejackimonster

2022-09-20 21:49

developer   ~0019159

@schanzen I also thought this would solve it but unfortunately it does not. The only option I found was the single-user setup running system and user services.

If I only run the FS service as user with the change in the config, I get the following:
2022-09-20T21:43:17.113890+0200 fs-2536 ERROR `mkdir' failed on file `/tmp/gnunet-system-runtime//gnunet-service-fs.sock' at service.c:1140 with error: Permission denied
2022-09-20T21:43:17.113977+0200 util-service-2536 ERROR `bind' failed at service.c:1405 with error: Permission denied
2022-09-20T21:43:17.113993+0200 fs-2536 ERROR `bind' failed at service.c:1558 with error: Success
2022-09-20T21:43:17.114003+0200 fs-2536 ERROR Could not bind to any of the ports I was supposed to, refusing to run!

The FS service will then crash and restart in a loop.

schanzen

2022-09-21 12:09

administrator   ~0019160

Last edited: 2022-09-21 12:10

That sounds like a misconfiguration to me: https://git.gnunet.org/gnunet.git/tree/src/util/util.conf#n26
In case of RUN_PER_USER, the service socker directory should be in GNUNET_USER_RUNTIME_DIR.
Your error looks like GNUNET_RUNTIME_DIR for system services.

Maybe check if RUN_PER_USER is _actually_ set.

thejackimonster

2022-09-21 22:14

developer   ~0019162

I mean, the first line in my config at /usr/share/gnunet/config.d/fs.conf states "RUN_PER_USER = YES" which I even copied from other services which do so. Also the service wouldn't even start using `gnuner-arm -s` otherwise if it wouldn't be set.

Here my output from `gnunet-arm -I`:
Services (excluding stopped services):
(started: 4 / stopped: 5)
fs (binary='gnunet-service-fs', status=started)
namestore (binary='gnunet-service-namestore', status=started)
zonemaster (binary='gnunet-service-zonemaster', status=started)
zonemaster-monitor (binary='gnunet-service-zonemaster-monitor', status=started)

It starts without issues it seems. But I can't really use it.

schanzen

2022-09-22 03:29

administrator   ~0019163

I dont understand. Your logs show errors that will prevent the service from starting because it cannot bind to the socket. How would it start like this?
Maybe the errors went away?

thejackimonster

2022-09-22 08:46

developer   ~0019164

The issue is that FS only starts without crashes when I don't start system services before. So when I just start user services with `gnunet-arm -s` it starts, it runs but it doesn't work because there are no system services running.

However when I run system services before via `sudo -u gnunet gnunet-arm -s`, so all of this runs:
Services (excluding stopped services):
(started: 16 / stopped: 22)
ats (binary='gnunet-service-ats', status=started)
cadet (binary='gnunet-service-cadet', status=started)
core (binary='gnunet-service-core', status=started)
dht (binary='gnunet-service-dht', status=started)
gns (binary='gnunet-service-gns', status=started)
hostlist (binary='gnunet-daemon-hostlist', status=started)
namecache (binary='gnunet-service-namecache', status=started)
nat (binary='gnunet-service-nat', status=started)
nse (binary='gnunet-service-nse', status=started)
peerinfo (binary='gnunet-service-peerinfo', status=started)
rest (binary='gnunet-rest-server', status=started)
revocation (binary='gnunet-service-revocation', status=started)
setu (binary='gnunet-service-setu', status=started)
statistics (binary='gnunet-service-statistics', status=started)
topology (binary='gnunet-daemon-topology', status=started)
transport (binary='gnunet-service-transport', status=started)

...and now I try to start user services via `gnunet-arm -s`, I get this (with the issue in my logs I mentioned before):
Services (excluding stopped services):
(started: 3 / failed: 1 / stopped: 5)
fs (binary='gnunet-service-fs', status=failed, exit_status=255, restart_delay='771 ms')
namestore (binary='gnunet-service-namestore', status=started)
zonemaster (binary='gnunet-service-zonemaster', status=started)
zonemaster-monitor (binary='gnunet-service-zonemaster-monitor', status=started)

So any other service from the system services seems to prevent the FS service can be run as user service as long as you don't run all services as user (single-user setup).

thejackimonster

2022-09-22 08:52

developer   ~0019165

By the way the FS service seems to use "/tmp/gnunet-system-runtime/gnunet-service-fs.sock" rather than "/tmp/gnunet-thejackimonster-runtime/gnunet-service-fs.sock" on my system, even if I run it as user service. So my assumption is that this is causing the problem. When the directory is created previously by the other user for the actual system services, the permission mismatch causes the error. I will investigate why it uses the wrong directory.

thejackimonster

2022-09-22 09:10

developer   ~0019166

The issue seems to be that the config uses:
UNIXPATH = $GNUNET_RUNTIME_DIR/gnunet-service-fs.sock

instead of:
UNIXPATH = $GNUNET_USER_RUNTIME_DIR/gnunet-service-fs.sock

I will commit a fix for this.

thejackimonster

2022-09-22 09:13

developer   ~0019167

https://git.gnunet.org/gnunet.git/commit/?id=1672c85ad702146521dc830298dcb3f802533539
This should be solved now.

thejackimonster

2022-09-22 09:15

developer   ~0019168

FS has to run as user service to have the same permissions as the running user to publish files as intended.

thejackimonster

2022-09-23 15:47

developer   ~0019171

Sorry but I had to reopen the issue. I really thought running FS as user service would solve it but it's more complicated. The FS service seems to require being run on the same peer as the CORE service. I don't know why that is necessary but now I'm unsure whether the CORE service should also run as user service.

Changing the CORE service to run as user service seems to work without any more crashes of services so far. But I haven't tested all other services yet. So I would need feedback how this should be solved before changing anything.

Otherwise the FS service won't start anymore in a multi-user setup but given that it didn't have proper file access before... it didn't work properly either. But we could also revert my latest commit (1672c85ad702146521dc830298dcb3f802533539) if that makes more sense. Then the file access issue needs to be addressed differently.

schanzen

2022-09-26 05:33

administrator   ~0019182

Core is certainly a system service. There should only be 1 "peer". Which is why core, transport etc are system services.
Other services (specifically those that require files per-user such as identity or namestore) are user services.
FS strikes me as a user service as it requires access to the user's files. What exactly is the issue?

thejackimonster

2022-09-26 12:10

developer   ~0019183

In `src/fs/gnunet-service-fs.c` the callback `peer_init_handler` (line 1234) fails because the peer identities mismatch between the CORE service and the FS service when only FS is running as user service.

Maybe this can just be removed but I'm not sure if this verification is important for security reasons.

schanzen

2022-09-26 13:45

administrator   ~0019184

To me this seems like a bug in fs. Or in gerneral the multiuser concept in GNUnet.
The $GNUNET_DATA_DIR location for the peer private key is not necessarily readable by the process.
The callback function also indicates this.
Further, I do not see where the private key is actually used.
Maybe the public key should be taken from the callback only.

schanzen

2022-09-26 13:46

administrator   ~0019185

EDIT: I see that FS _needs_ the private key. I do not know how this is supposed to work in the multiuser context, then.
Pinging grothoff.

Christian Grothoff

2022-09-26 14:13

manager   ~0019187

FS was written before we had the multi-user support and was never fixed to properly support it. In fact, the entire service would need to be split into two parts, one running on the peer-side and one running on the user-side (as we need both the per-user access to files, as well as the system-wide/peer-wide aspects). So yes, the (current) FS implementation doesn't work nicely with multi-user, and the only way to fix it is to grant additional permissions in one way or other that would ideally not be required.

schanzen

2022-09-26 14:32

administrator   ~0019188

I guess in this case you have the option to leave this bug open and possibly rename it to reflect the last comment or close it and open a new one addressing this issue.

Issue History

Date Modified Username Field Change
2022-09-09 21:27 thejackimonster New Issue
2022-09-10 15:30 schanzen Note Added: 0019115
2022-09-10 15:31 schanzen Target Version => 0.17.6
2022-09-11 00:40 thejackimonster Note Added: 0019116
2022-09-19 14:19 thejackimonster Note Added: 0019147
2022-09-19 14:38 thejackimonster Note Added: 0019148
2022-09-19 15:22 thejackimonster Note Added: 0019149
2022-09-19 15:33 thejackimonster Note Added: 0019150
2022-09-19 15:43 thejackimonster Note Added: 0019151
2022-09-19 15:54 thejackimonster Note Added: 0019152
2022-09-19 15:57 thejackimonster Reproducibility sometimes => always
2022-09-19 15:57 thejackimonster Category messenger service => file-sharing service
2022-09-19 15:57 thejackimonster Description Updated
2022-09-19 15:57 thejackimonster Steps to Reproduce Updated
2022-09-19 15:57 thejackimonster Additional Information Updated
2022-09-20 19:01 schanzen Note Added: 0019157
2022-09-20 19:01 schanzen Assigned To => schanzen
2022-09-20 19:01 schanzen Status new => feedback
2022-09-20 19:01 schanzen Assigned To schanzen => Christian Grothoff
2022-09-20 21:49 thejackimonster Note Added: 0019159
2022-09-20 21:49 thejackimonster Status feedback => assigned
2022-09-21 12:09 schanzen Note Added: 0019160
2022-09-21 12:10 schanzen Note Edited: 0019160
2022-09-21 22:14 thejackimonster Note Added: 0019162
2022-09-22 03:29 schanzen Note Added: 0019163
2022-09-22 08:46 thejackimonster Note Added: 0019164
2022-09-22 08:52 thejackimonster Note Added: 0019165
2022-09-22 09:10 thejackimonster Note Added: 0019166
2022-09-22 09:13 thejackimonster Note Added: 0019167
2022-09-22 09:15 thejackimonster Status assigned => resolved
2022-09-22 09:15 thejackimonster Resolution open => fixed
2022-09-22 09:15 thejackimonster Fixed in Version => 0.17.6
2022-09-22 09:15 thejackimonster Note Added: 0019168
2022-09-23 15:47 thejackimonster Status resolved => feedback
2022-09-23 15:47 thejackimonster Resolution fixed => reopened
2022-09-23 15:47 thejackimonster Note Added: 0019171
2022-09-26 05:02 schanzen Target Version 0.17.6 => 0.17.7
2022-09-26 05:33 schanzen Note Added: 0019182
2022-09-26 12:10 thejackimonster Note Added: 0019183
2022-09-26 12:10 thejackimonster Status feedback => assigned
2022-09-26 13:45 schanzen Note Added: 0019184
2022-09-26 13:46 schanzen Note Added: 0019185
2022-09-26 14:13 Christian Grothoff Note Added: 0019187
2022-09-26 14:32 schanzen Note Added: 0019188
2022-09-26 16:38 thejackimonster Summary Unreliable file sharing => File service does not work in a multi-user setup
2022-09-26 22:41 thejackimonster Fixed in Version 0.17.6 =>
2022-09-29 07:11 schanzen Target Version 0.17.7 => 0.18.0