View Issue Details

IDProjectCategoryView StatusLast Update
0004839GNUnetfile-sharing servicepublic2018-06-07 00:24
ReporterploopAssigned ToChristian Grothoff 
PrioritynormalSeveritymajorReproducibilityrandom
Status closedResolutionfixed 
Platformamd64OSParabola GNU/Linux-libreOS VersionN/A
Product Version0.10.1 
Target Version0.11.0pre66Fixed in Version0.11.0pre66 
Summary0004839: gnunet-publish or gnunet-download unpredictably ignores files
DescriptionWhen using `gnunet-publish' on a directory, then running the `gnunet-download' command produced by `gnunet-search', sometimes the resulting directory will be missing files. For example, on this directory with four JPEG images, named {1..4}.jpg:

================================================================================
moe :: ~/gnunet-testing » gnunet-publish -k a namori-images
Publishing `/home/nya/gnunet-testing/namori-images/4.jpg' done.
URI is `gnunet://fs/chk/U1ONA2M82JHKQFMNQOTKR3J1D7KURF74QPUHMNAC4DE7VB2B9NHIESDQ7TII10DQA18BL6O4RCSL6FL1PNPBSSH0R4S6P6IJ7HQU95G.0N6BL66U4RA5UK3MLD45A8V12SA5SV1KTQFHLKM8RPM58U8OMC442FKP4A365JMV1L61L1MHLC41FL6JOL5CE6R87ULD40HLN656VB0.603001'.
Publishing `/home/nya/gnunet-testing/namori-images/3.jpg' done.
URI is `gnunet://fs/chk/ASE6IOL0096BMVCASU2S98AT59BMQGM18CLN832FBH8JQBS3I81DT2A5A2BBFB7KFSLH60P1JU28NH040BJD6L7G56KB8UPTKCQKUU8.GP8IB3ND2VHIJ8S94GH25CGJ840FAPT18SI8QA5D6P8L488RU2642FFH94C1EGG5H5BJJU8JM8DHCG0JSHE16PAO1NC3SRN0V9I0IF0.248021'.
Publishing `/home/nya/gnunet-testing/namori-images/2.jpg' done.
URI is `gnunet://fs/chk/8E7L3RR5EOOEVEM45KFE0JIE8P7FM0HTFJ53JALHLK6FCRP3IK76UKEUUINIP9ATS9P4M814M9F6172DM7RRNTIAI4JDL1DILC5Q9MG.B5DTL1H6V761HKONQMR44KVVH2O60R1OOAD74F46RJSR9SO4A3G90NFNFDSE3F97AEQLMDIBBL6E4KT71QAOMINEKRQ4HGCAUUVNNQ8.23789'.
Publishing `/home/nya/gnunet-testing/namori-images/1.jpg' done.
URI is `gnunet://fs/chk/PP87PUOQITHNDEST040US05LJM3NN81J7JAI6UNJT6J5B5LG5VJGEH7FQ8NACRLCFDKV97UCK6DKKHKHVPS9D1FHE95H3T3QS3129C8.QMIG7UF1HBPJBUM4OPFHRNFBIAHS3687L6C3L9G1DHRUSF9UUSNMQCCM4PILVC181QUTF63O745A7G015F4A5BQ8ED8DGECQTQOD8JO.32193'.
Publishing `/home/nya/gnunet-testing/namori-images' done.
URI is `gnunet://fs/chk/PJP48NQJ1APCJN3V72J3Q90HR6IPMTEUGBR38FSIQ3JN7NF8LGO51TH4TN3UN7HN8I34N8NJUPCGTV89GCNUB9SUD74PB87UAHDP5S8.C12M54BK4PQLOE5NGOQ9OJEBLKEIB73HVT2MNNC1U8BB7R23K5M0LK3F3B9DAKCH3FVKIKQ1UREQ8D3UD9EA06LLPFC5M65NEQVI5A8.63124'.
moe :: ~/gnunet-testing » gnunet-search a
#0:
gnunet-download -o "namori-images_.gnd" -R gnunet://fs/chk/PJP48NQJ1APCJN3V72J3Q90HR6IPMTEUGBR38FSIQ3JN7NF8LGO51TH4TN3UN7HN8I34N8NJUPCGTV89GCNUB9SUD74PB87UAHDP5S8.C12M54BK4PQLOE5NGOQ9OJEBLKEIB73HVT2MNNC1U8BB7R23K5M0LK3F3B9DAKCH3FVKIKQ1UREQ8D3UD9EA06LLPFC5M65NEQVI5A8.63124

^C% moe :: ~/gnunet-testing » mkdir a
moe :: ~/gnunet-testing » cd a
moe :: ~/gnunet-testing/a » gnunet-download -o "namori-images_.gnd" -R gnunet://fs/chk/PJP48NQJ1APCJN3V72J3Q90HR6IPMTEUGBR38FSIQ3JN7NF8LGO51TH4TN3UN7HN8I34N8NJUPCGTV89GCNUB9SUD74PB87UAHDP5S8.C12M54BK4PQLOE5NGOQ9OJEBLKEIB73HVT2MNNC1U8BB7R23K5M0LK3F3B9DAKCH3FVKIKQ1UREQ8D3UD9EA06LLPFC5M65NEQVI5A8.63124
100% [============================================================]
Downloading `namori-images_/1.jpg' done (7 KiB/s).
100% [============================================================]
Downloading `namori-images_/4.jpg' done (12 KiB/s).

Downloading `namori-images_.gnd' done (1284 b/s).
moe :: ~/gnunet-testing/a » ls
namori-images_ namori-images_.gnd
moe :: ~/gnunet-testing/a » cd namori-images_
moe :: gnunet-testing/a/namori-images_ » ls
1.jpg 4.jpg
================================================================================

Out of curiosity, I tried this on another directory, with the same filenames each containing the same number of bytes, but with data from /dev/urandom instead of valid JPEG data, and something similar happened, except {2..4}.jpg downloaded successfully and 1.jpg was missed.

In case it's necessary, I've attached both the JPEG and random files.
TagsNo tags attached.

Activities

ploop

2016-12-14 17:57

reporter  

gnunet-files.tar.gz (1,780,699 bytes)

Christian Grothoff

2017-03-03 22:20

manager   ~0011884

22:15 < grothoff> ruunyan: I was able to reproduce, and the bug is with gnunet-download.
22:18 < grothoff> Yes, somehow it thinks a directory was completely downloaded, even though it never started on some files.
22:19 < grothoff> gnunet-directory does show all the files properly, so it's really somewhere in the logic that schedules
                  files for download.
22:19 < grothoff> It also appears identically in gnunet-gtk, so it must be somewhere in libgnunetfs.
22:19 < grothoff> Also, if you download _again_, the missing files do appear.
22:19 < grothoff> (i.e. by calling gnunet-download twice)
22:20 < grothoff> So it likely relates to limits on parallel downloads (which causes downloads to be deferred), and the code
                  not properly checking for deferred downloads when deciding upon completion.

Christian Grothoff

2017-03-05 15:57

manager   ~0011892

Fixed with commit 8a32cac23d7289cfb7d1d356603a523cf6effd13.

Basically, in fs_download.c::check_completed(), remove the "(NULL == dc->child_head) &&" clause.

Issue History

Date Modified Username Field Change
2016-12-14 17:57 ploop New Issue
2016-12-14 17:57 ploop File Added: gnunet-files.tar.gz
2017-02-21 18:46 Christian Grothoff Assigned To => Christian Grothoff
2017-02-21 18:46 Christian Grothoff Status new => assigned
2017-02-21 18:46 Christian Grothoff Target Version => 0.11.0pre66
2017-03-03 22:20 Christian Grothoff Note Added: 0011884
2017-03-05 15:57 Christian Grothoff Note Added: 0011892
2017-03-05 15:57 Christian Grothoff Status assigned => resolved
2017-03-05 15:57 Christian Grothoff Resolution open => fixed
2017-03-05 15:57 Christian Grothoff Fixed in Version => 0.11.0
2017-03-05 15:57 Christian Grothoff Fixed in Version 0.11.0 => 0.11.0pre66
2018-06-07 00:24 Christian Grothoff Status resolved => closed