View Issue Details

IDProjectCategoryView StatusLast Update
0003878GNUnetfile-sharing servicepublic2018-06-07 00:25
Reporterremibd Assigned ToChristian Grothoff  
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionno change required 
Platformx86_64OSGNU/LinuxOS Version4.0.7-2
Product Version0.10.1 
Target Version0.11.0pre66Fixed in Version0.11.0pre66 
Summary0003878: GNUNET_FS_download_stop fails to stop every task associated with a download
DescriptionAfter a download is completed and GNUNET_FS_download_stop is called, at least one pending task remains and is never run.

See the attachment.
Steps To Reproducegcc -std=c99 -Wall -Werror -I/usr/include/gnunet -lgnunetutil -lgnunetfs -o download download.c

then call `./download <filename> <uri>` on a chk uri.
Additional InformationHere’s an example of execution:

---------------
$ GNUNET_LOG="util-scheduler;;;;DEBUG" ./download "trek.txt" gnunet://fs/chk/AHILVENCEPEHLI9B1TQQ06CTI70TA4OOJ653E9G2D7JPV57HRN528KK7I270D81PAV23GBNNPS6KKQM48CLH7FG4IJTLETPK551MRH8.74DJFOM1T999MC6K65NUIMCORGI1S8IL27JS9SVIM79QE2S6GMSQE0K87LI0D95J9HV0VDCGFG1LBK97C2E5BD2T5F6TQTAFF6KP3F0.50

scheduler: GNUNET_PROGRAM_run
juil. 02 20:46:04-362613 util-scheduler-6235 DEBUG Registering signal handlers
juil. 02 20:46:04-362725 util-scheduler-6235 DEBUG Adding continuation task: 1 / 0x7ffddeaa5f30
juil. 02 20:46:04-362757 util-scheduler-6235 DEBUG Adding task: 2 / (nil)
juil. 02 20:46:04-362796 util-scheduler-6235 DEBUG Checking readiness of task: 2 / (nil)
juil. 02 20:46:04-362819 util-scheduler-6235 DEBUG Running task: 2 / (nil)
juil. 02 20:46:04-362846 util-scheduler-6235 DEBUG Running task: 1 / 0x7ffddeaa5f30
scheduler: start first_task
scheduler: add download tasks
juil. 02 20:46:04-362948 util-scheduler-6235 DEBUG Adding task: 3 / 0x1d40af0
scheduler: add shutdown_task
juil. 02 20:46:04-362980 util-scheduler-6235 DEBUG Adding task: 4 / (nil)
scheduler: end first_task
juil. 02 20:46:04-363007 util-scheduler-6235 DEBUG Running task: 3 / 0x1d40af0
scheduler: run progress_cb (7)
progress_cb: download started
scheduler: end progress_cb (7)
juil. 02 20:46:04-363051 util-scheduler-6235 DEBUG Adding task: 5 / 0x1d40a70
juil. 02 20:46:04-363092 util-scheduler-6235 DEBUG Running task: 5 / 0x1d40a70
scheduler: run progress_cb (14)
scheduler: end progress_cb (14)
juil. 02 20:46:04-363228 util-scheduler-6235 DEBUG Adding task: 6 / 0x1d41120
juil. 02 20:46:04-363257 util-scheduler-6235 DEBUG Adding task: 7 / 0x1d40a70
juil. 02 20:46:04-363282 util-scheduler-6235 DEBUG Checking readiness of task: 6 / 0x1d41120
juil. 02 20:46:04-363303 util-scheduler-6235 DEBUG Running task: 6 / 0x1d41120
juil. 02 20:46:04-363331 util-scheduler-6235 DEBUG Adding task: 8 / 0x1d41120
juil. 02 20:46:04-364472 util-scheduler-6235 DEBUG Checking readiness of task: 8 / 0x1d41120
juil. 02 20:46:04-364521 util-scheduler-6235 DEBUG Running task: 8 / 0x1d41120
juil. 02 20:46:04-364582 util-scheduler-6235 DEBUG Adding task: 9 / 0x1d411f0
juil. 02 20:46:04-364625 util-scheduler-6235 DEBUG Running task: 9 / 0x1d411f0
scheduler: run progress_cb (10)
scheduler: end progress_cb (10)
scheduler: run progress_cb (15)
scheduler: end progress_cb (15)
juil. 02 20:46:04-365226 util-scheduler-6235 DEBUG Canceling task: 7 / 0x1d40a70
juil. 02 20:46:04-365255 util-scheduler-6235 DEBUG Adding task: 10 / 0x1d40a70
scheduler: run progress_cb (12)
progress_cb: download ended
scheduler: end progress_cb (12)
juil. 02 20:46:04-365291 util-scheduler-6235 DEBUG Running task: 10 / 0x1d40a70
juil. 02 20:46:04-365311 util-scheduler-6235 DEBUG Adding task: 11 / 0x1d40a70
juil. 02 20:46:09-368064 util-scheduler-6235 DEBUG Running task: 4 / (nil)
scheduler: run shutdown_task
scheduler: run progress_cb (13)
scheduler: end progress_cb (13)
scheduler: end shutdown_task
Killed
---------------

In this instance, of the 11 tasks added, the last (n°11) won’t be run and prevents the program from ending.
TagsNo tags attached.
Attached Files
download.c (2,822 bytes)   
#include <sys/select.h>
#include "platform.h"
#include "gnunet_fs_service.h"

static char *client_name;
static char *output_filename;
static const struct GNUNET_CONFIGURATION_Handle *config;
static struct GNUNET_FS_Handle *fs;
static struct GNUNET_FS_DownloadContext *dc;
static struct GNUNET_FS_Uri *uri;


static void
shutdown_task(void *cls, const struct GNUNET_SCHEDULER_TaskContext *tc) {
  FPRINTF(stderr, _("scheduler: run shutdown_task\n"));
  if (dc) {
    GNUNET_FS_download_stop (dc, 1);
    dc = NULL;
  }
  FPRINTF(stderr, _("scheduler: end shutdown_task\n"));
}

static void
cleanup_task(void *cls, const struct GNUNET_SCHEDULER_TaskContext *tc) {
  FPRINTF(stderr, _("scheduler: run cleanup_task\n"));
  if (fs) {
    GNUNET_FS_stop(fs);
    fs = NULL;
  }
  FPRINTF(stderr, _("scheduler: end cleanup_task\n"));
}

static void *
progress_cb(void *cls, const struct GNUNET_FS_ProgressInfo *info) {
  FPRINTF(stderr, _("scheduler: run progress_cb (%u)\n"), info->status);
  if (info->status == GNUNET_FS_STATUS_DOWNLOAD_START) {
    FPRINTF(stderr, _("progress_cb: download started\n"));
  } else if (info->status == GNUNET_FS_STATUS_DOWNLOAD_COMPLETED) {
    FPRINTF(stderr, _("progress_cb: download ended\n"));
  } else if (info->status == GNUNET_FS_STATUS_DOWNLOAD_STOPPED) {
    GNUNET_SCHEDULER_add_continuation (cleanup_task, NULL,
				       GNUNET_SCHEDULER_REASON_PREREQ_DONE);
  }
  FPRINTF(stderr, _("scheduler: end progress_cb (%u)\n"), info->status);
  return NULL;
}

static void
first_task(void *cls, char *const *args, const char *cfgfile,
	   const struct GNUNET_CONFIGURATION_Handle *c) {
  struct GNUNET_TIME_Relative fivesec = { (5 * 1000 * 1000) };
  
  FPRINTF(stderr, _("scheduler: start first_task\n"));
  config = c;
  fs = GNUNET_FS_start(config, client_name, progress_cb, NULL,
		       GNUNET_FS_FLAGS_NONE, GNUNET_FS_OPTIONS_END);
  FPRINTF(stderr, _("scheduler: add download tasks\n"));
  dc = GNUNET_FS_download_start(fs, uri, NULL, output_filename, NULL, 0UL,
				GNUNET_FS_uri_chk_get_file_size(uri),
				0, GNUNET_FS_DOWNLOAD_OPTION_NONE, NULL, NULL);
  FPRINTF(stderr, _("scheduler: add shutdown_task\n"));
  GNUNET_SCHEDULER_add_delayed(fivesec, shutdown_task, NULL);
  FPRINTF(stderr, _("scheduler: end first_task\n"));
}

int
main (int argc, char *argv[]) {
  int ret;
  char *emsg;
  struct GNUNET_GETOPT_CommandLineOption options[] = {
    GNUNET_GETOPT_OPTION_END
  };

  if (argc != 3) {
    FPRINTF(stderr, _("Invalid number of arguments"));
    return -1;
  }
  client_name = argv[0];
  output_filename = argv[1];
  uri = GNUNET_FS_uri_parse(argv[2], &emsg);
  FPRINTF(stderr, _("scheduler: GNUNET_PROGRAM_run\n"));
  ret = GNUNET_PROGRAM_run(argc, argv, "download OUTPUT URI",
			   gettext_noop("foo"), options, first_task, NULL);
  return !(ret == GNUNET_OK);
}
download.c (2,822 bytes)   

Activities

Christian Grothoff

2015-07-03 11:51

manager   ~0009394

Looking at your code, it seems to me like you're simply not using the API correctly.

Right now, you call "FS_download_stop()" in you shutdown task, which you trigger after 5s -- so obviously the process will sit there idle for 5s.

You should schedule the FS_download_stop()-task when you receive DOWNLOAD_COMPLETED (you do nothing there right now).

Then, when you get DOWNLOAD_STOP, you (correctly) schedule a task to do FS_stop(). So with this tiny change, things should work fine.

Christian Grothoff

2015-07-08 15:29

manager   ~0009403

remibd: is this now resolved?

remibd

2015-07-10 20:11

reporter   ~0009412

Last edited: 2015-07-10 20:12

Not really: with the searching API, there’s no need to call FS_stop() (except to free the handle) because calling FS_search_stop() ends every task associated with a search. Even if I forget to call FS_stop(), which is the case when I let Guile’s garbage collector free the FS handle, the scheduler is eventually short of tasks and simply returns.

Having FS_download_stop() to fail killing every task associated with a download means that in the case of bulk download, for instance when retrieving a bunch of binary packages, I’ll have idling tasks stacking, forcing me to regularly re-allocate a new FS handle.

Anyway this doesn’t really complexify my task, thus it is only an unsignificant issue, or misunderstanding on my part.

Christian Grothoff

2015-07-10 21:19

manager   ~0009413

Again, I checked and could not find FS_download_stop leaving a dangling task.
In your C example, you're using the API wrong and thus FS_download_stop() is not called, hence it fails.

Please re-read my comment 9394 and compare with the C code you provided. If you fix it as I suggested, the C example will NOT hang -- at least not for me. So please try to fix the C version first (as per comment 9394), and let me know if you then still have a problem. (We can also Mumble or chat on IRC about this).

Christian Grothoff

2015-08-03 13:39

manager   ~0009523

Considering that 0003917 also suggests that you're really using 0.10.1 and not SVN HEAD, I'm pretty sure the issue is simply that you're using an old version. Please upgrade to SVN HEAD for development work.

Issue History

Date Modified Username Field Change
2015-07-02 21:26 remibd New Issue
2015-07-02 21:26 remibd File Added: download.c
2015-07-03 11:07 Christian Grothoff Assigned To => Christian Grothoff
2015-07-03 11:07 Christian Grothoff Status new => assigned
2015-07-03 11:51 Christian Grothoff Note Added: 0009394
2015-07-03 11:51 Christian Grothoff Status assigned => feedback
2015-07-08 15:29 Christian Grothoff Note Added: 0009403
2015-07-10 20:11 remibd Note Added: 0009412
2015-07-10 20:11 remibd Status feedback => assigned
2015-07-10 20:12 remibd Note Edited: 0009412
2015-07-10 21:19 Christian Grothoff Note Added: 0009413
2015-08-03 13:39 Christian Grothoff Note Added: 0009523
2015-08-03 13:39 Christian Grothoff Status assigned => resolved
2015-08-03 13:39 Christian Grothoff Fixed in Version => 0.11.0pre66
2015-08-03 13:39 Christian Grothoff Resolution open => no change required
2015-08-03 13:39 Christian Grothoff Target Version => 0.11.0pre66
2018-06-07 00:25 Christian Grothoff Status resolved => closed